What actually breaks
Automated reports don’t usually fail in obvious ways.
They degrade—quietly, and over time.
1. Tracking changes without visibility
Websites evolve.
- new pages are added
- forms are updated
- events are renamed or removed
These changes affect how data is collected.
But your reporting layer doesn’t know that.
It continues to pull data based on assumptions that are no longer true.
2. Connectors drift or disconnect
Automated reports rely on APIs and third-party connectors.
Over time:
- authentication expires
- APIs change
- data schemas shift
Sometimes data stops flowing.
More often, it partially breaks.
A metric updates—but not correctly.
A dimension changes—but silently.
The report still looks complete.
It just isn’t.
3. Definitions change
Platforms evolve.
- attribution models change
- session definitions are updated
- conversion logic is adjusted
These changes rarely break reports outright.
Instead, they introduce subtle inconsistencies.
The same metric begins to represent something slightly different than before.
4. Assumptions drift
Automated reports depend on a few key assumptions:
- tracking is consistent
- sources align
- definitions remain stable
At first, they may hold.
Over time, they don’t.
The result isn’t a broken dashboard—
it’s a misleading one.
Why this goes unnoticed
Because the system keeps working.
There’s no clear failure point—
just a growing gap between:
what the data says
and what’s actually happening
The core issue
Automated reporting solves for efficiency.
It does not solve for data integrity.
It assumes the data is correct—and stays that way.
When that breaks, the reporting layer has no way to recover.
What reliable reporting actually requires
Accuracy over time comes from structure:
- defined tracking logic
- controlled data modeling
- consistent metric definitions
A system—not just a pipeline
A simple way to think about it
Automated reports:
pull and present data
A structured system:
defines what that data means
What to watch for
If your reports are degrading, you’ll start to notice:
- numbers that don’t match across platforms
- unexplained changes in performance
- more time spent validating data
- decisions being second-guessed
These aren’t reporting issues.
They’re system issues.
Final thought
Automated reporting doesn’t break.
It becomes unreliable.
And without a system behind it, there’s nothing to bring it back into alignment.
Doug McCaffrey
Designs and maintains analytics systems that remain reliable over time.
Explore how this connects across your data estate: