The coding expert, the logic expert, and the overall model request routing team assume that each will cover the others’ blind spots. A new study An evaluation of 67 frontier models from 21 providers shows that the assumption is mathematically flawed – and the flaw has a name: the joint failure ceiling.
The hypothesis works like this: if two models do not normally fail in the same instructions, combining them should create a safety net against failures.
The real limitation in orchestration is not how often the models disagree, but the percentage of propositions that each model in the pool gets wrong at any one time. By ignoring the shared failure ceiling, enterprises build complex, expensive routing infrastructure to chase performance gains that don’t exist. Fortunately, developers can use the same math to build a free test that determines when multi-model orchestration will actually pay off.
The hidden costs of a multi-model strategy
Developers typically rely on three architectures to organize multiple language models. Model routers act as a traffic cop by sending complex requests to expensive models and simple requests to cheaper models. Cascades send each offer to the low-cost model first, only moving up to the premium model if the initial system signals low confidence. Finally, who approaches A mix of agents (MoA) combines several models by asking them the same question and generating a synthesized answer from their combined results.
These architectures are a "shadow price" to deduct costs. Every time a development team implements routing or cascading, they pay a premium for additional system latency, complex infrastructure maintenance, and increased management risks across multiple API providers.
To justify these operating costs, engineers rely on “pairwise error correlation” to select their model pool. Imagine a developer has a Model A who writes great Python but fails at SQL, and a Model B who writes great SQL but fails at Python. Their pairwise error correlation is low because they fail at different types of cues. The developer believes that by placing a router layer in front of them, they have created a composite system that rarely fails to encode.
According to the study, lumping different models together based on low correlation can actually hurt performance if the models aren’t equally capable—when you’re voting between different but unequal models, the weaker ones often stack up and outperform the smartest.
The paper’s author, Joseph Chen, told VentureBeat that in his experiments "Naïve majority voting on unequal models yielded a negative mean gain (minus 10 points in our hard mix): diverse but weaker members outperform stronger ones." Here’s some actionable advice for developers "Only combine models in the appropriate quality range." If you can’t match the quality, consider a single model and spend your budget on the best model available.
The article has one bright spot for this approach regarding MoA architectures. Teams often use it when building ensembles "Self MoA," where they query the same premium model repeatedly to generate a synthesized response. The researchers found that generating an ensemble of diverse models with low pairwise correlations of suitable quality outperforms the highly correlated Self-MoA setup.
However, when teams use the same pairwise correlation metric to predict the absolute accuracy of their overall system, the math breaks down.
"So teams pay the orchestration overhead (latency, complexity, multi-provider operations) on the assumption that diversity dividends come later." Chen said. "This is usually not the case, because today’s best models agree, and worse, they fail the same queries… When the boundary disagrees, the indication of which model is correct simply carries little signal."
Why math fails: the joint failure ceiling
The main finding of the study is based on the so-called metric "joint failure rate" — the official name for the completely wrong scenario described above. No router, voting system, or cascade can ever achieve higher accuracy than the ceiling it sets.
Coding, logic, and the common pool show low pairwise correlations in regular cues—they rarely fail together. But the joint failure ceiling represents a dark, highly complex edge case that pushes the limits of current AI architectures. If the command is so difficult that all three models hallucinate or fail, it doesn’t matter how intelligently the router distributes the task. The entire pool is deleted at once.
The researchers tested a pool of 67 models, including the GPT-5.5, Claude Opus 4.8, and Gemini 3.1 Pro, on the open-ended MATH-500 math test. Based on standard pairwise correlations, statistical models predicted that the entire pool would clear only 2.3% of questions simultaneously. In reality, the combined failure rate was 5.2%.
Standard correlations underestimated the failure rate by about 2.25 times. The culprit is not just an independent difficulty, but a shared point of failure.
"The driver is what we call a common-mode atom: the slice of queries where the entire market fails together, which no pair of statistics can see," Chen said. "Adding a model 20 to your pool does not buy a tail cover. The tail is shared."
The researchers also found that task format directly contributed to joint failure. When they took graduate-level science questions from the GPQA benchmark and changed them from a multiple-choice format to a free-response format, the entire error tail widened to 12.7%.
Developers can engineer around the ceiling. "The engineering result is troubling: multi-model installations get the least where teams want it most, in open generation," Chen said. "Anywhere you can turn generations into validation or limited choice (structured results, verifiable responses, performance tests) you open the ceiling again."
Ultimately, the researchers found that this ceiling limits AI applications in two different ways depending on the domain:
-
Ceiling-related environments (eg open-ended math): The joint failure rate is high. The task is very difficult and all models fail simultaneously. No routing can bypass the lack of core capability.
-
Possible environments (e.g. science at master’s level): The joint failure rate is close to zero, meaning that at least one model in the pool usually knows the answer. However, the models disagree so finely that the routing layer cannot reliably choose the correct answer without an omniscient oracle.
$0 sanity check before posting
Before devoting engineering hours to building a router, teams can calculate absolute performance ceilings for free using a Clopper-Pearson-based mathematical formula.
Acts as a Clopper-Pearson worst-case scenario calculator. If you flip a coin ten times and get eight heads, you cannot guarantee that the coin will always land heads 80% of the time. The table takes a small sample of test questions and deduces a mathematically guaranteed ceiling.
Applied to language models, suppose a team tests a pool of five agents on 50 sample queries and finds that they all fail only two questions. A developer can assume that their multi-agent system will achieve 96% accuracy in production. The Klopper-Pearson formula corrects this optimism. He analyzes a small sample size and mathematically guarantees that the true joint failure rate may actually be as high as 12%.
To use this in practice, enterprises must create a stored database. For example, a fintech company might take 200 complex customer support tickets from the previous quarter and record perfect resolutions for human agents to serve as benchmarks. While this may sound like a heavy manual project, mature engineering teams can automate the entire ceiling calculation.
"Integration is trivial: it’s computational work on top of the evaluation logs that teams already produce," Chen notes, "thus, it runs in the same CI phase as the evaluation suite and restarts when the model pool or workload changes."
The engineering team then runs the candidate models once against these 200 tickets and records the results. When they want to evaluate multi-model configurations, they can use the joint failure rate measure to predict the maximum accuracy they can get from the system without making additional queries.
An important conclusion of the study is that in tasks where responses can be strictly verified, combining models rarely uses the best single model in the market unless the team has an extremely strong query-level routing signal.
In an enterprise environment, there is an objective, zero-tolerance answer to a strictly audited task. This includes creating an SQL query that must execute without error, extracting a custom invoice from a 50-page PDF, or formatting a JSON payload that perfectly matches a strict schema. For these tasks, businesses are usually better off paying a premium for the smartest border model than combining three cheaper models and hoping the router gets the right result. The study did not test subjective, non-assessed tasks such as developing marketing copy—the authors note that it remains an open question whether these findings fall outside the criteria being tested.
Because this mathematical validation is free, enterprise teams can track their collective failure rates as new models drop.
"Measurement costs nothing, so any team can track their failure rate between model generations and see if the tail is closing," Chen says. Finally, "The leverage buyers hold is not model number, but failure mode heterogeneity and market volatility."
