Zillow, the real estate technology company, never once talks to its customers. They go from phone screen to loan officer, sometimes to real estate agent over months or years, and expect the context to follow. A single chatbot can never carry this topic.
Horse VB Transform 2026Zillow SVP of Engineering Toby Roberts and Glean co-founder and CEO Arvind Jain explained how they built their AI architecture to carry context throughout the journey and why context, not raw data, is a harder problem to solve. Zillow’s products touch about 80% of US real estate transactions each year, and the company used AI long before ChatGPT existed.
"We quickly identified that we needed a persistent context layer that would meet our customers and professionals wherever they were." Roberts said.
Data has never been the hard part
Zillow’s AI efforts begin where most enterprise AI efforts begin with data itself, Roberts said.
"We started with a big push to make sure our data had the right foundation," Roberts said. This meant a data grid approach, a clear data pipeline and a governance structure with permissions and identity attached to the data itself.
None of these were difficult problems. The challenge was to build something that remembers where the customer is in their journey and takes them forward, no matter what surface they appear on.
"This context layer must live to support you wherever you are on your journey," Roberts said. Zillow chose to own that layer itself rather than depend on a single external chat interface, which Roberts said the team arrived at quickly after looking at the shape of a real transaction rather than a single chat.
Why Zillow built its architecture and Glean fits it
Instead of routing customers through a single model API, Zillow built its own trailers. The team leveraged its 20-year machine learning history behind products like Zestimate, leaning toward smaller, task-specific fine-tuned models instead of one general-purpose model.
Internally, this plugin works with Glean. Roberts said Zillow currently has thousands of Glean agents in production, performing repetitive tasks with tens of thousands of executions within the company. According to Jain, Glean’s pitch centralizes this integration work once through the Glean MCP gateway, rather than allowing finance, legal and marketing to each maintain their own connections to the same systems.
That centralization is also a cost lever. Jain pointed to two mechanisms: model routing, which sends most tasks to smaller, cheaper models instead of defaulting to boundary models, and precomputed context, which prevents an agent from assembling its own context from scratch and burning tokens.
"Claude is also very slow because the first part of gathering this context actually takes forever," Jain said. Routing that demand through Glean instead could cut token consumption in half, he said.
What Zillow and Glean’s approach means for businesses
The session on data, pricing and permissions provided several practical suggestions for enterprises building agent AI into their systems.
Build your measurement base before, not after, the AI boost. Roberts said Zillow’s ability to reliably attribute a 40% increase in referral code to AI adoption was based on the DORA metrics the team posted years ago, not the AI implementation itself.
Centralize the context once instead of letting each team rebuild it. Jain’s main argument for the Glean platform is that the repetitive integration work between finance, legal and marketing teams is a hidden cost that most businesses don’t account for.
Don’t assume that permission inheritance is sufficient for regulated data. Even with a permission-aware context platform, Zillow overlaps with strict rules and constant compliance checks for the most sensitive categories, rather than relying on automatic management of the architecture.
Treat context as a cost lever, not just a capability. Model routing and pre-calculated context were two mechanisms Jain pointed to for reducing AI costs, both aimed at reducing wasted token consumption rather than adding new capabilities.
"Models are not enough to bring automation with AI in your enterprise," Jain said. "You should relate it to your enterprise context."
