TL;DR
Wharton researchers found that 80% of people get the wrong AI answers. Now, apps like Moot are cashing in on instinct to outsource decisions.
A pair of Wharton researchers have put a name to something that many AI users are quietly starting to do: let chatbots make decisions for them. Steven Shaw and Gideon Nave published a study in JanuaryThinking, Fast, Slow and Artificial,” which they presented the expressioncognitive surrender” to describe people’s tendency to reject AI outputs, even when those outputs are wrong.
In a study conducted through the University of Pennsylvania’s Wharton School, participants were asked to answer questions with or without AI assistance. Those who received the help of artificial intelligence got 93% of the correct answers, which is not surprising. What caught the attention of the researchers was the error rate: participants perceived error AI answers 80% of the timeand 11.7% of those working without AI reported high levels of confidence.
The results came from controlled experimental conditions rather than real-world use, but were consistent from sample to sample.
Shaw and Nave proposed what they called “Three systems theory,” to addSystem 3” to the framework made famous by Daniel KahnemanThinking, Fast and Slow.“In their model, System 1 is fast intuition, System 2 is slow thinking, and System 3 is cognition supported by artificial intelligence, in which the machine effectively performs the thinking of the human mind. The risk, they argue, is that System 3 gradually weakens Systems 1 and 2 through disuse.
This phenomenon is not limited to academic practices. Business Insider reports that Carolyn Yoo, a former software engineer in New York, used Anthropic’s Claude chatbot to decide whether to quit her job, how to tell her parents, and what to do about a friend who was annoying her. He told the publication that he approached the chatbot as a combination of a therapist and a life coach.
Business Insider also cited financial writer Dominic Frisby, who wrote that he asked an AI chatbot on Substack for relationship advice and found the response more helpful than anything offered by a human friend.
Now there is a commercial product built on this exact impulse. Launched earlier this year, the Moot app allows users to present their life decisions to a panel of five AI personalities named The General, The Sage, The Skeptic, The Diplomat and The Architect. People discuss the question among themselves and then vote and give a recommendation.
According to the app’s listings on the Apple App Store and Google Play, it’s designed for people stuck with everyday choices, from career changes to relationship questions. The app’s effectiveness claims come from the company itself and have not been independently evaluated.
Cornelia C. Walther, senior fellow at Wharton’s AI and Analytics Initiative, told Business Insider that flattery of artificial intelligence, the tendency of chatbots to agree with users rather than challenge them, exacerbates the problem. When a chatbot validates every instinct the user brings to it, the feedback loop that usually forces revision disappears.
Walther, who explores social AI applications, described a pattern consistent with broader public concern about the social implications of artificial intelligence.
Separate studies support the concern. Helen Putnam Fellow at Harvard’s Radcliffe Institute and Anat Perry, associate professor of psychology at the Hebrew University of Jerusalem, co-authored a paper in Science that examines how sycophantic AI responses erode users’ ability to calibrate their judgments. The paper found that when AI systems consistently confirm the user’s position, the user’s ability to make independent judgments deteriorates over time.
Joanna Stern, senior technology analyst at NBC and “I’m Not a Robot: The Year I Used AI to Do Everything” documented the rampant dependence on AI in everyday life. His report showed how users started with low-value queries like what to cook for dinner or what to wear and gradually escalated to consequential decisions about careers, finances and relationships. Once the trajectory from comfort to trust is difficult to reverse.
The Wharton study’s framing of cognitive surrender as a structural risk is not just a bad habit, it’s important because it shifts the conversation from individual discipline to system design. If AI tools are designed to be maximally agreeable and frictionless, then the cognitive surrender described by Shaw and Nave is not a lack of will, but a predictable consequence of the product’s architecture.
Stanford’s 2026 AI Index report found a widening gap between public anxiety and expert optimism about artificial intelligence.suggesting that ordinary users sense something that the builders of these systems are slower to admit. The question is whether the industry will treat cognitive delivery as a design flaw worth fixing or an engagement metric worth optimizing.
Shaw and Nave’s recommendation is simple: AI systems should be designed to make users think, not think for them. Whether this recommendation meshes with the incentive structures of consumer AI, where ease of use and maintainability are important dimensions, is another question entirely.
