Since I decoupled the sort order from the aggregations, there were some combinations that didn't made sense, for example grouping by date by sorting by something different.

Fix this situation by not allowing those combinations, and using sensible defaults instead.