A Total That Looks Right Can Still Be Wrong

Plausibility is the weakest possible evidence, and it is the evidence most people quietly rely on.

The most dangerous answer is not the absurd one. An absurd answer defends you: if revenue comes back as eleven trillion dollars you will investigate, and you will find the problem. The dangerous answer is the one that sits comfortably inside the range you were expecting, because your expectation is what you check against and a wrong number inside that range passes every test you were actually applying. You feel satisfied. You send it on. And the only thing that made it feel right was that it was not surprising.

This is worth sitting with, because it means your confidence is largely uncorrelated with your accuracy. A figure you feel sure about is a figure that matched your prior guess, which tells you about your guess and not about the data. The practitioners who get caught out are rarely careless people. They are people whose intuition was good enough to produce a believable expectation and who then used that expectation as verification. Intuition is excellent for noticing that something is wrong. It is worthless for confirming that something is right.

So replace the feeling with a procedure. Take one record you can verify independently, ideally something you can check against a document or a person, and follow it all the way through your working to confirm it lands where it should with the value it should. Then check the boundaries: the earliest date, the latest date, the largest and smallest values. Then confirm your counts at each stage. None of this proves the answer correct. All of it removes the most common ways of being wrong, which is the best that anybody can actually do.