SQL Child / Summarising Safely

Filtering Before And After You Summarise

Narrowing your data at different moments gives different answers, and both of them will look entirely reasonable.

There are two quite different moments at which you can exclude records, and confusing them is a rite of passage. The first moment is before anything is summarised: you throw away the records you do not care about, and then you summarise what remains. The second moment is after: you summarise everything, then throw away whole summary lines that fail some test. Both are legitimate operations. They answer different questions and produce different numbers, and the difference is not always visible from looking at the output.

An example makes it concrete. Consider customers and their orders, and you want to look at large customers. If you exclude small orders first and then total per customer, you get each customer's total from their large orders only, and small customers may still appear with modest totals. If instead you total every order per customer and then keep only the customers whose total exceeds a threshold, you get complete totals for genuinely large customers, and small customers vanish entirely. Same data, same threshold, wildly different lists, both correct for their own question.

The way to keep this straight is to go back to your written sentence. Is the condition about individual records, or about the group as a whole? Orders over one thousand dollars is a condition on individual records, so it belongs before you summarise. Customers whose annual spend exceeds fifty thousand is a condition on the group, so it can only be applied after, because the group total does not exist until the summarising has happened. Say the condition out loud and ask which thing it describes. That question resolves it every time.