There is an important difference between AI that just works today and AI that continues to scale. Many companies over-optimize for the former without asking if they’re building the latter.
Speed without discipline and strategic direction is a liability, not an asset. The hardest part of building AI at scale isn’t getting a model to work once. It builds systems that keep working, transcend individual teams and use cases, and continuously improve over time.
Today’s AI systems do more than predict and optimize. They talk, think and act more and more. An autonomous system that makes decisions on behalf of the traveler creates very different expectations around reliability, governance and accountability. As artificial intelligence takes over more of these roles, the principles of how these systems work are more important than ever.
We’ve spent years applying AI and machine learning (ML) to the traveler journey, from personalization, ranking, and recommendations to fraud prevention, customer support, and more recently, generative and agent AI experiences. This depth of experience has led us to develop a set of ML and AI principles to guide how we build, deploy and improve AI systems at our company.
The goal is simple: Make sure the systems we build deliver real business value, scale and operate securely. These principles define how we measure, design, operate and manage our systems.
From principles to practice
Publishing principles are the easy part. The harder and more important work is turning them into operational mechanisms: Recommendations, requirements, tools, and release processes that teams actually use.
We’ve started using ‘Agentic Release’ transitions: a set of recommended and in some cases required checks before agent AI features can be enabled. These fees translate principles such as clear ownership, risk-based management, evaluation, secure implementation and monitoring into concrete expectations for teams.
Some of these recommendations and requirements are already automated and integrated into the software development life cycle (SDLC). Over time, the goal is to incorporate these expectations into how we design, evaluate, validate, deploy, and monitor AI systems from the ground up.
Conclusions: Measuring what really matters
The first test for any model is whether it improves the bottom line and ultimately the traveler experience—not just whether it improves technical metrics.
-
Scale models with business impact: Every ML effort should be tied directly to a key business outcome or traveler experience metric. Technical optimizations are useful midpoints, not end goals.
-
Optimize for cost recovery: The value the model creates must justify what it costs to develop, train, and monitor, and the operational complexity it adds. Opt for solutions that have a lasting impact relative to their cost of use.
-
Justify complexity with strong rationale: Sophistication should be earned, not assumed. Start with a strong base: Existing generic model, simple heuristics, out-of-the-box solution. Only resort to specialized models or more complex architectures when simpler options fail to provide the true answer.
-
Request both an offline and online assessment: No model goes to wide deployment only in offline validation or goes directly to A/B testing. Each model must perform in both offline and online assessments. Over time, our offline assessments should reliably predict what we see online.
Design: building systems that go beyond the teams that build them
Hiring a model is a challenge. It’s harder to capture its value beyond a single command or use case.
-
Building on common foundations; specialize only when justified: Opt for a shared, platform-wide foundation for core capabilities, data representations, and model building blocks. Specialization should build on these foundations, not rotate isolated stacks, so when the foundation improves, gains flow throughout the organization.
-
Treat data as a premium product: The quality of a model is limited by the quality of its data. We need to maintain solid pipelines, clear generations, reproducibility and reusable features that other teams can rely on, built with documented ownership, clear schemas and SLAs.
-
Prefer generality over local optimization: Favor an approach where learnings, assets, and operational patterns are reusable across teams, brands, and use cases, when the two approaches are implemented similarly. We need to optimize not only for local performance, but also for how quickly improvements can be rolled out across the company and converged over time.
-
Minimize and sunset manual workflows: Manual rules are sometimes necessary for policy, security, or compliance, but they should be clear and reviewed regularly, never silent patches for weak models or a source of constant maintenance debt.
-
Reproducibility and tracking by default: Training data, features, configurations, evaluation results, deployment versions, and key decisions should all be documented and recoverable. This allows you to resolve a production issue months later and hand over ownership without losing institutional knowledge.
Trust: ownership, governance and working responsibly at scale
The bar for implementing artificial intelligence is simply not high "does it work" this "can we stand behind it?" Trust is not something you add on at the end; it is earned over time and maintained for the full life cycle of every model we ship.
-
Establish clear ownership and accountability: Each model needs defined ownership throughout its lifecycle – business owner, product owner, AI owner, and operations owner. It doesn’t have to be four people, but the responsibilities should be clear. Who is responsible for the results? Who will answer if the model slips? Who responds to the incident at 2 o’clock in the morning? If this is not in place, the models are orphaned and problems are revealed where there is no one to take ownership of them.
-
Adhere to standards and governance: AI and ML models must use validated platforms and adhere to established company standards, release gates, and governance processes. Operating outside these guardrails requires a clear, defined path to elimination or obsolescence, rather than an outright exception.
-
Manage in proportion to risk: The level of review, evaluation rigor, and human control should be measured against the impact of the model. A customer-centric model that affects the price or availability of millions of travelers requires a much higher bar than an in-house tool used by a small team. For high-impact, safety-sensitive, or highly autonomous systems, human control points are built into the loop from the start.
-
Design for fairness, privacy and transparency: When decisions meaningfully affect users, we proactively test for unintended bias, have strong data safeguards, and prioritize accountability. These are included from the beginning, not added.
-
Design for safe release, recoil and control: Deployments are progressive, with return paths, return mechanisms, and circuit breakers ready prior to activation. The ability to safely return a deployment is just as important as the ability to send it.
-
Continuously monitor and adapt: Once live, teams must actively monitor quality, churn, latency, cost, and business performance and retrain or recalibrate when data changes. The team should always be able to explain the current performance of the model, not how it performed when it was launched.
These principles do not define how we build. They define what we are willing to send and how we stand behind it. In a world where artificial intelligence systems are increasingly effective and making real decisions for real travelers and partners, these standards are essential. When applied consistently, they build responsible AI that is sustainable.
Xavi Amatriain is chief artificial intelligence and data specialist at Expedia Group
Xavier will share more details about Expedia’s architecture in his session VB Transform July 14 at 11:10 a.m. PT. He will discuss: "Expedia’s plan to build autonomous agents for high-stakes operating systems."
Interested in participating in VB Transform 2026? Sign up here. A select number of free passes are also available for senior technology leaders. Contact us to get yours.
