You spend weeks tuning an AI chatbot. The answers are accurate. Stakeholders sign and you send it. After three months, the system is confidently wrong in one-third of what users ask. No one changed the model and no one touched the instructions. The world has changed, prices have changed, policies have been updated, product specifications have shipped a new version, and the main knowledge store hasn’t moved with it.
This is not hypothetical. This is one of the most common production failure modes in enterprise AI right now, and regardless of how the AI system gets the data, most data engineering teams don’t have the right tools to catch it.
Failure that doesn’t seem like failure
The AI application does not care whether it is fetched from the vector store, document index, or API call. Whatever the mechanism, nothing in the default lookup pipeline checks that what it serves is still valid. The old grade document is retrieved with the same confidence as the current document because the system evaluates compatibility or availability, not correctness. A record with a silently missing field passes as cleanly as a complete record for the same reason.
So failure is invisible by design. Outdated or incomplete data still gets a high score for relevance or passes every check so that the data pipeline can be activated. The model responds with full confidence because the retrieved context appears to be authoritative. It stays green on every dashboard you look at. The system seems to be working. It’s just wrong.
I’ve watched a similar version of this happen outside of the AI context, in the fintech pipeline. The upstream system changed the field without notifying the downstream users. The pipeline did not fail; it was just propagating bad values to the dashboard because the system was only checking that the job completed, not whether the data was still correct. The problem only appeared when the customer saw something inappropriate. By then, the bad news had already moved downwards.
Whether it’s an out-of-date document or a silently lost field, the form of failure is the same: the absence of an error is not the presence of correctness, and nothing in the pipeline can identify the problem without the proper validation layers in place.
Why is this a data engineering problem?
Teams that fail to do this tend to make the wrong diagnosis, and they tend to do it twice.
Blames the model: The first instinct is to blame the model, try a different LLM, adjust the order. The real problem is higher up, at the level of data engineering, the same instinct behind the fintech failure above: monitoring built for the pipeline, not the data.
Blame the search layer: After the model is ruled out, the next instinct is to blame the search or context layer and buy a better one. The timing is no coincidence: as enterprises push these systems into the real production world, this gap is starting to become fully apparent, and the vendor response has been all over the place.
-
AWS simply entered "context layer" race with a knowledge graph that learns from agent usage.
-
Snowflake’s new Horizon Context and Cortex Sense target the exact symptom. opened with this piece: agents confidently give wrong answers because nothing is driving the business logic underneath them.
Both are a real answer to a real problem, but sit one layer above it; the graph of knowledge still depends on whatever feeds it.
The real problem lies further up, at the level of data engineering. Teams check to see if a job is done, to see if the data it’s transferred is still correct, an instinct that predates artificial intelligence by years. Monitoring is built for the pipeline, not the data.
What’s really missing: Data observability
Data observability is a popular concept, but not enough attention is paid to how it is actually implemented. The relevant metric isn’t percentage—it’s coverage: what fraction of critical data sets actually have a generation that can be queried versus just living in someone’s head.
Uber built it specific data quality and observability platform long before search augmented generation existed. Their Unified Data Quality platform supports over 2,000 critical data sets and detects approximately 90% of data quality incidents before they reach downstream consumers.
Netflix solved a different part of the same problem, establishing a company-wide data line system so anyone can answer where the data set came from and what touched it along the way. It maps dependencies across Kafka topics, ML models, and experiences, not just warehouse tables. Like Uber, the platform was built for people and is now becoming more important with the rise of AI/LLM applications.
Between them, Uber and Netflix cover two of the four things you need to set up. In practice, I think of it as four dimensions, each measurable on its own terms.
Correctness: Does each record conform to its intended form and rules, correct field types, unexpected spaces, values in range. tools like Great Expectations and soda handle it well: automated row- and column-level checking instead of manual checks after something breaks. Track the percentage of records that pass validation in each run.
Freshness: Is the data still current relative to the source, not just at the time of last check? Track the time since the last successful update for each source, as some sources require hourly updates while others do not.
Sequence: The same fact reads the same wherever it is stored or indexed. It fails silently, only appearing when two systems fed from the same source begin to disagree. Periodic cross-checking between downstream directions is sufficient to flag the degree of inconsistency above a threshold, to catch it early.
Generation: Can you trace any output back to its source and every transformation it goes through, the same question Netflix built its system to answer.
None of this requires infrastructure that most data teams don’t have. I know because I built it, I didn’t just argue for it.
Horse Socurecustomer data arrived in the form the customer intended to send it, and sometimes it was silently wrong. The challenge was to build a system that could identify false information before it spread downwards. The same principles apply: Validate what’s coming, understand where it’s coming from, and prevent bad information from becoming someone else’s problem.
High expectations became part of this foundation: schema and range checking on ingest, per-source SLAs for freshness, cross-system checks for consistency, and file-level generation. They all sat in the back write-audit-publish the instance in which the data enters the stage is validated and moved downstream only if it passes the required checks.
The result showed downstream: better accuracy in reporting, ML models and AI search built on the same data.
What to do on Monday morning
If you’re running search-based AI systems in production, the diagnostic question isn’t which model to try next or which search architecture to switch to. These four narrower questions are:
-
Is the master data verified according to the standards required by its consumers?
-
What is the oldest content currently served with high confidence?
-
Can two pieces of the same source agree with each other in the same search result?
-
If it turns out to be wrong, can you trace where it came from?
If you can’t answer these questions, then the gap is in the pipeline between your source systems and whatever your agent is reading. This is a data engineering fix, not a model change or vendor migration.
Whether you’re building reporting pipelines, ML systems, or AI agents, ensuring accuracy, freshness, consistency, and generational data reliability. AI simply exposes the weaknesses that exist in data engineering.
