A single AI agent conversation can appear flawless and still point to a broken product. This gap is driving a shift in how enterprises evaluate agents, away from evaluating individual footprints and comparing user cohorts to a baseline.
Horse VB Transform 2026, Harrison Chase, CEO of LangChain; Hui Zhang, CTO and Co-Founder of Conviva; and Emmanuel Turlay, director of engineering at CoreWeave, described the change as a parallel move to cheaper, narrower ruling models.
Agent as referee – judging the performance of one AI agent against another – did not replace LLM as referee, which Chase said remains the default. The bigger tension, Zhang said, is between automated judgment by an LLM or agent and human review.
"Whether it’s agents as referees or LLMs as referees, you’re scalable but groundless, you value the bottom line, you value the work. It’s still very difficult to justify and then you use people and it’s just not scalable," Zhang said. "The whole industry faces this, which poison do you want to pick."
Evaluation criteria already function as product specifications
That gap—the conversation that’s doing well but still points to a broken product—is what teams try to close by building a comprehensive set of evaluations before shipping anything. Chase said it wouldn’t work.
"Sometimes we see teams that are almost paralyzed," Chase said. "They say it’s an evaluation kit, I can’t get it to work. The best teams start and then iterate."
The Chase framework evaluation criteria is not a one-time test suite, but a living specification: a product requirements document — a standard software development specification of what a program should do. "Ratings are like new PRD," he said. "They define what your agent should and shouldn’t do."
Turlay described hitting the same failure from a different angle. "I was trying to get 100% coverage for my tests and still had errors in production," he said – a test suite that looks complete but still misses what’s important, the same loophole that Chase describes with evaluations.
Extensive, always-on monitoring, he says, results in more real failures than a comprehensive pre-launch test suite. Teams should first set up extensive online checks, use them to identify classes of failures as they occur, and then build targeted offline assessments around emerging issues.
Why is it wrong to list traces one by one?
Even a well-designed evaluation process can still produce erroneous results. Zhang’s objection is to how most teams conduct the evaluation: sampling tracks, 50 of them, or the entire population, evaluating each one individually. This approach only misses a signal that emerges when comparing cohorts of users to a baseline, Zhang calls contrastive analysis.
Zhang illustrated this with a retail example: a buyer asks an agent for running shoes before a half-marathon, the agent asks relevant questions, and the buyer buys the shoes. Individually, this interaction looks good. However, the accuracy rate of how many follow-up questions an agent asks before completing a task is three times higher than the baseline for this shoe category among the full user population. A second metric, the frequency with which shoppers completed their purchase outside of chat, was five times higher than the baseline for the same category.
No number is visible from a trace. Both point to a categorical problem that can eliminate errors. Zhang said the industry lacks a second source of data: what happens before, between and after the conversation, not just the track itself.
Measuring the judge for the case
Once the contrastive analysis has identified which category is actually violated, the next challenge is whether and at what cost. Tourlay’s rule was to start with the most capable model available and work up to prove the task was solvable. If it can’t be done with a high-end model, he said, it won’t work with a smaller one. When sampling is appropriate, teams can sample a subset of traffic instead of evaluating each interaction, and offload simpler tasks like binary classification to smaller open-source models.
LangChain has taken this even further by fine-tuning its model to detect a signal that the user believes the agent has made a mistake, which Chase calls a perceived error. "The model we reviewed was the Gwen model," he said, referring to Alibaba’s open source family. By combining hand label with distillation, the result worked well. "It’s the same with (Claude)Sonnet because it’s either 10-100x cost reduction depending on how we service it," Chase said.
Not every fuse needs a model. Chase pointed to Claude Code’s own preservers as evidence: regexes, a common programming technique for finding and validating patterns in code. "Most of the guards they had were just regex," he said. "They weren’t little LLMs, just regex."
An LLM as a judge does not mean that a person disappears from the circuit
The bigger question is whether using LLM as a referee will remove the need for a human in the circuit.
Turlay pointed to accountability, using his previous job at a self-driving car company. His team compressed data acquisition and reengineering into a two-week cycle to ship a new model car. Even then, someone had to sign.
"I made sure on behalf of the company that this model should go into the car," he said. The same logic applies to law, finance and healthcare. "Before removing a person, I confirm and take legal responsibility for it, it will be a while before agents can do it on their own."
Zhang agreed that while automation will eventually work at a scale that exceeds individual human accuracy, humans will remain custodians in the corner boxes—machines can see more at the sample level.
Chase went even further: this human check is more than just a safety net. "The human in the loop is really important for building trust in how these agent systems work, and also really important for learning from memory and systems." he said. "For the system to learn, there must be interaction."
