Enterprise AI faces an ROI paradox. While throwing more computation at the strongest foundational model works well in product experiments, the cost becomes prohibitive when the product is deployed in production.
A new paper Writer provides an accessible solution from researchers to engineering teams. The study takes a systematic look at optimizing various components of the orchestration layer surrounding the foundation model, such as AI harnesses.
By optimizing harnesses, researchers show a dramatic reduction in tokens per task, up to a 61% reduction in cost per successful task, and stable quality without changing the underlying foundation model.
Because the harness is fully under the control of the developer and no fine-tuning of the model is required, engineering teams can apply these findings to create highly cost-effective AI applications.
Tokenmaxxing’s ROI crisis
The current state of AI engineering suffers from this "tokenmaxxing," an industry trend where developers rely on massive context windows and brute force token consumption replacing good system design.
Instead of designing elegant workflows, developers have imported a reflex from traditional software development: build, run, fail, re-window the error and more context, and try again.
"Teams tokenmaxx because it’s the cheapest fix right now and that’s literally how most engineers work today," Waseem AlShikh, CTO and co-founder of Writer, told VentureBeat. Because this approach succeeds often enough in coding tasks, it has become the default reflex for every other agent workload. The danger is that price drops per token mask underlying inefficiencies.
"Your invoice is a token for each task and most teams only track the second number," Al-Sheikh said. "In agent workloads, token merges per task—each loop iteration retransmits increasing context—and merges faster than decrements. Price reduction becomes anesthesia. It hides the fact that the loop itself is bleeding."
Tokenmaxxing leads to several enterprise failure modes. Teams redirect simple tasks to premium boundary models by default. They use LLM as a lazy search index, filling the context window with raw documents instead of getting accurate answers. Most destructively, they create infinite agent loops that spiral out of control when the model encounters an error. Since exit tokens cost significantly more than input tokens in all major model providers, inefficient execution of tasks is a silent budget killer.
The industry has introduced several efficiency techniques to curb these costs, but they mostly fall short because they drive the model in isolation:
-
Quick compression compresses the input text to save space, but ignores how the system sequences these inputs across complex workflows.
-
Budget justification involves computational steps that the model can perform, and this often degrades the quality of the output if the workflow is not intelligently directed.
-
Reverse encoding forces models to output minimal code to store output tokens, but does nothing to address inefficient tool invocation.
-
Speculative decoding uses a smaller draft model to speed up text generation of a larger model by optimizing inference speed, unable to resolve bloated agent architectures.
These efforts fail because they optimize the engine while ignoring the transmission. They don’t look at the orchestration layer, leaving underlying architectural inefficiencies unresolved.
Trailer packaging: levers of efficiency
The harness is the orchestration layer that routes, formats, and transforms the underlying LLM into a working system.
Key levers of harness optimization include system-wide caching, interaction history compression, tool management, search strategies, and error handling. These are the most accessible intervention points for engineering teams looking to improve AI performance.
As the Writer researchers note in the study: “If the trailer model is the layer that calls for work, it is also the layer that determines the price of work.”
Historically, developers have treated attachments as a one-time glue code meant to simply attach an API to the user interface. Research suggests that the harness should now be treated as a first-class object: a core software artifact that requires its own testing, versioning, and rigorous design.
For businesses, it’s resurfacing "vs. self-rent" decision.
"Enterprises spend months on model evaluations and then rent their orchestration off the shelf – meaning they optimize the smaller arm and outsource the larger one." Al-Sheikh said. "Whoever owns the trailer owns the economics of your unit and is not tuned to your open frame build for demos."
Inside the experiments
To isolate the effect of the orchestration layer, the researchers ran experiments on six foundational models spanning multiple vendors and weights: Claude Sonnet 4.6, Gemini 3.1, Gemini Flash 3.5, Gwen 3.6, GLM 5.1, and Writer’s own Palmyra X6.
Their experiments compared a frozen, conventional production agent circuit with an off-the-shelf Writer Agent Harness on the same 22 locked-down enterprise tasks, including capabilities such as grounding and search, multi-step workflows, tool usage, and content creation. By keeping models and tasks fixed, they can isolate the effects of the orchestration layer itself.
The optimized harness led to significant cost reductions, reducing the blended cost per task by 41%, from 21 cents to 12 cents. This was mainly achieved by reducing token consumption, with the number of tokens per task dropping by 38% from 14.2k to 8.8k.
The trailer is designed to delegate tasks such as searching to specialized sub-agents. The sub-agent receives only the tool and the specific query it needs, retrieves the exact data, and returns a closed, clean summary to the main agent – keeping the main context window from filling up with raw search results.
Assignment success rates remained stable even as token usage declined—rising from 78% to 81%, a gain the researchers described as directional rather than statistically significant in sample size, meaning quality has not been lost even though costs have decreased.
End-to-end task latency was also significantly reduced due to faster caching and the resulting elimination of justification loops, reducing the average wall clock time by 44%, from 48 seconds to 27 seconds.
However, researchers also found limitations to multi-agent orchestration. Smaller models such as Gemini Flash 3.5 and Gwen 3.6 scored well below usable reliability thresholds in sub-agent delegation tasks (0.45 and 0.42, respectively) – a capability that is simply not reliable in lighter-weight models.
Sub-agent orchestration exceeded usable reliability thresholds only on the two most powerful models tested: The writer’s own Palmyra X6 (0.86) and Claude Sonnet 4.6 (0.85).
The developer’s playbook: impressive results and compromises
The research results become a textbook for enterprise developers building agent workflows at scale. The first step is to implement what AlShikh calls "Two Zone Advice" and "Context Unloading."
System query caching structure (Dual Zone Query): Modern LLM APIs offer operational caching, but to enable it, developers must properly structure their payloads. Developers should separate "stable zone" from "volatile zone." Place static, unchanging items (such as ground rules, major tool charts, and standard operating procedures) at the top of the survey. Dynamic elements should be added below, such as a custom user request or the state of the last conversational task. This ordering allows the trailer to reuse the cached prefix across hundreds of calls. "This single separation really enables operational caching and prevents you from paying for the same instructions over and over again at each of the agent’s thirty steps." Al-Sheikh said.
Manage context with context unloading: Avoid context padding where each turn of the loop adds to a monolithic query until the window is maxed out. Instead, copy the history and intermediate artifacts from the window to removable storage, and only pull back what is needed for the current step. If possible, delegate tasks to single-purpose sub-agents to avoid context bloat. As Al-Sheikh points out, "The biggest line item in agent spending isn’t reasoning—it’s resending what the model has already seen."
Build sustainable loops and redefine KPIs: Unmanaged agent loops quickly drain API budgets. Teams should start tracking Completions Per Million (CPM) to understand their true engagement costs, but the trailers themselves should have physical guardrails. "The main principle is that you should never ask the model to control her expenses." Al-Sheikh said. "The fence should live under the model in the code, on your side of the API." This requires three strict checks:
-
Hard token budgets per task: The run is stopped when the budget is spent, unless an exception is made.
-
Generation Fencing: Caps on steps, tool calls, and recursion depth to stop non-combining agents.
-
Unsuccessful spending management: After the first failed check, measure what the run could cost so that the failed task doesn’t become your most expensive job.
Avoid unnecessary complexity: Optimizing the orchestration layer comes with engineering costs. If you’re in the prototyping and exploration phase, this extra cost isn’t justified – iterate fast with a powerful model and a lightweight trailer. Once you reach millions of queries per day, the savings from harness optimization become significant.
But teams should be aware "trailer hitch." Adding structural scaffolding requires the model to capture and obey that context. If the model is too small, it will spend its limited resources on scaffolding analysis instead of completing the task, leading to reduced accuracy and higher tokens. The procedure for adding complex orchestral features is strictly mathematical: "If a feature adds more coordination tokens than removes task tokens for that particular model, cut it," Al-Sheikh said. "Nothing in a trailer is free."
The future of the enterprise
The era of tokenmaxxing and treating context windows as bottomless buckets is coming to an end. Throwing more compute into poorly designed systems is not a viable strategy for companies that need to demonstrate returns on AI investments.
As the underlying models evolve to accommodate planning, tool selection, and multistage reasoning, the role of trailers will shift from compensating for model weakness to enforcing enterprise policy.
"The “allowables” that are never included in the model are: budgets, permissions, data boundaries, audit trails, deterministic kill switches," Al-Sheikh said. "In five years, the harness will be more subtle but more important. There will be less scaffolding and more management. No matter how skilled a model is, someone outside of it still needs to determine what it can spend, see, and touch. That layer is owned by the enterprise and should never be rented."
