War story

Two systems, two numbers. Which one is lying?

· Adrian Sullivan

It usually surfaces in a meeting. The finance system says one number. The operations system says another. They are supposed to be the same number, and they are not, and now an awkward hour is spent working out which one to believe.

Underneath, this is almost always a database problem wearing a business disguise. Two systems hold overlapping data. Something syncs between them, or someone assumed something synced between them, and over time they drifted apart. A failed integration nobody noticed. A field that means one thing here and another thing there. A nightly job that has been silently skipping rows for months.

One I remember had two systems that were a few percent apart, consistently, for over a year. Everyone had a theory. The real answer was a data feed that quietly dropped any record with a particular character in it, every single night, since a change long forgotten. Small, consistent, invisible, and slowly making one of the numbers a polite fiction.

Nothing had failed loudly. Both systems were up. Both were confidently reporting. They were just no longer telling the same truth, and no alert in the world watches for that.

We run a free, read-only check that looks at the integrity and the joins beneath your reporting, not just whether the servers are running. When two numbers disagree, the database usually knows which one is lying. Somebody just has to ask it.

Free health check

Want to know if this is sitting in your estate? We run a read-only check and hand you a graded report in plain English.

Get your free health check

← All posts