What actually slows down AI agents is legacy infrastructure, not the models themselves. This was the collective conclusion of three infrastructure leaders – From LinkedIn, Walmart and Zendesk — hour VB Transform 2026.
The panel brought together Animesh Singh, SVP of AI platform and infrastructure at LinkedIn, Desiree Gosby, SVP of enterprise technology services and technology strategy at Walmart, and Sami Ghoche, VP of applied AI at Zendesk, who each described what actually breaks when agents move from pilot to production. Each came from a different starting point to the same conclusion: None of the bottlenecks they hit were model problems.
Unifying their responses was a common thread: most enterprise infrastructure is built for how people work, not how agents work. The space between these two speeds is where the real engineering happens.
Gosby made this clear when asked what Walmart has learned scaling agents in its workforce. According to him, the goal is to be sure "engineering no longer becomes a bottleneck to what we are trying to do."
Where was the bottleneck actually
Each company hit a different version of the same wall: the infrastructure designed for how people work doesn’t stop when agents do the work.
The first bottleneck at LinkedIn wasn’t a model, it was Kubernetes, which assumed that containers were spun up on demand, a process that took seconds. Singh said it was too slow for agents. The fix moved from on-demand provisioning to pre-provisioned pools of containers that change agent workloads in real-time.
After allowing LinkedIn agents to manage their own orchestrations, a second, more difficult problem arose. The five-point grading system seemed clean, but the hallucinations kept showing up anyway. Singh said the problem is structural, an LLM evaluating another LLM’s performance shares the same failure mode as the one it is evaluating.
"We built our own harnesses, our own control flow, and pushed LLMs to the leaf instead of forming a loop," Singh said. About 80% of the workflow is now scripted, deterministic code, LLMs are used only where reasoning is required, and evidence of each step is written to disk before the system is run.
Walmart’s bottleneck stemmed from success. The agent trailer, which was put directly into the hands of employees, went viral internally, and what Gosby called it "citizen developers" they started building their own agents to solve problems that once required a formal engineering roadmap. The advantage was a real innovation. The downside was the duplication of dozens of agents following each other without coordination. The fix wasn’t about bridging trailers, it was about building controls to detect duplication, promote the best version of the agent, and get it into production before it becomes an engineering bottleneck.
Zendesk hit the bottleneck on the data side. Joined Ghoche Zendesk’s acquisition of ForethoughtClosing in March 2026, Zendesk described 20 billion customer conversations in its repository as sitting on what it calls public figures. The instinct is to hand that history to a big language model with a big context window and let it build the agents the business needs. Ghoche said it wouldn’t work. "You can’t really do that, so instead you have to really invest in the underlying data pipelines and all the data infrastructure that comes with it." he said.
The role of open source
In open source, all three leaders were built on a similar instinct: own what you can and trust frontier labs only where they still have a clear edge.
Ghoche said he believes most enterprises prefer to own their own models and infrastructure wherever possible, and that rationale drives Zendesk’s approach. The exception is boundary reasoning work, which is still led by labs, though he said the slice of use cases is shrinking relative to what enterprises are doing with AI now.
LinkedIn’s response was to build two subsystems specifically for independence. The first is what the company calls an AI gateway, which is a single interface through which every call to the model goes, regardless of provider. The second component is a memory subsystem built to store context independent of any model provider.
"Every outbound call to LLM follows the same semantics, the same API calls, whether in the public cloud or on-premise in our own data centers. We can quickly switch between different providers," Singh said.
Walmart built its internal gateway to remain vendor agnostic across three types of workloads: fully deterministic workflows, scheduler and scheduler workflows for open-ended tasks, and a hybrid of the two. Compliance-heavy work remains deterministic by design; management, security, and evaluation go through the gateway regardless of the model on the other end. Gosby said the choice between a boundary model and an open-weight model is not about a fixed policy, but about which is more effective for a specific workload.
Tips for a modernization journey
Three tips came directly, each tied to the wall the leader had already hit.
Above all else, invest in assessments. Ghoche called it something common to every internal or customer-facing use case.
"What they all have in common is the assessments. This will force you to solve the problem, and once you have a solid set of estimates, you can move much faster." said
Own your agent belt from day one. Gosby’s recommendation was to get the AI harness into the hands of workers early on, coupled with infrastructure to track what the trailer was producing.
"This will unlock a huge amount of innovation," he said.
Build for model and context independence. Ensuring flexibility is critical to success.
"Build for independence, whether it’s today’s frontier model or tomorrow’s open source model," Singh said. "Save this context in your enterprise so you can reuse it tomorrow when you ship a model or trailer," Singh said.
