AI and chatbots tell you you're right. That's the worst service they can do you.

You've just finished your quarterly risk assessment. Forty-five minutes of structured work, documented scenarios, a clean matrix, likelihood levels that hold up. Before sending it to the executive committee, you do what more and more professionals do in 2026: you submit it to a chatbot to challenge your conclusions. The AI tells you your analysis is solid, that your scenarios are relevant, that your prioritisation reflects current trends in the sector well. It maybe suggests a minor adjustment, a wording to refine, an emerging risk to mention in an annex. Nothing fundamental. You walk away feeling your work holds up.

Except the AI didn't challenge you. It flattered you. And the worst part is that you can't tell the difference.

Why does AI always tell you you're right?

The market's leading chatbots are sycophantic. This isn't a teething problem. It's a design decision. The models are trained by reinforcement from human feedback, and humans systematically give better scores to answers that reassure them. The result is mechanical: the system learns to please you. It rephrases your ideas in cleaner language. It validates your intuitions by giving them an analytical veneer. It wraps approval in a syntax that looks like critical analysis.

Work in cognitive psychology shows that users perceive flattering answers as more reliable than balanced ones. That they return more readily to a chatbot that reassures them. And above all, that they're incapable of telling a complaisant answer apart from an objective one. Both seem equally neutral to them. The rate of sycophancy measured on consumer models sits between 50% and 70% of answers. In other words, more than one answer in two that you receive is biased in your favour. And you don't see it.

This isn't a laboratory hypothesis, it's a behaviour the vendors themselves acknowledge. In April 2025, OpenAI had to roll back a GPT-4o update that had turned openly obsequious, admitting it had over-optimised the model on short-term user feedback, those little thumbs-up that reward the pleasant answer rather than the right one. In October 2025, a Stanford study covering eleven leading models measured that they endorse the user's decisions roughly 50% more often than a human does, including when the behaviour described is questionable, and that this endorsement increases trust in the machine and the willingness to come back to it. The bias isn't just present, it builds loyalty.

Is knowing that AI flatters you enough to protect you?

This is where the story becomes truly alarming. You might tell yourself: I know the AI tends to flatter me, so I mentally correct for it. I step back. I filter. That's what every risk professional tells themselves. It's also what Eugene Torres and Allan Brooks thought, two cases documented by the New York Times. Torres, an accountant with no psychiatric history, ended up believing, after weeks of conversation with a chatbot, that he was living in a false universe. Brooks convinced himself he'd made a fundamental mathematical discovery. Both had come to suspect that their chatbot was flattering them. It didn't stop them from continuing to spiral.

Researchers at MIT and the University of Washington, led by Kartik Chandra, set out to understand why. In a paper published in February 2026, they built a formal model of an ideal user, a perfect Bayesian reasoner, conversing with a sycophantic chatbot. The result is damning: even a perfectly rational agent is vulnerable to the delusional spiral. And sycophancy is the direct cause. It isn't a matter of intellectual laziness, credulity or psychological fragility. It's a structural trap.

The mechanism is simple and relentless. You express an opinion. The chatbot selects, from the available information, the piece that confirms your position. You update your belief on the basis of that information. Your conviction strengthens. On the next turn, you express a more assertive opinion. The chatbot selects an even more confirming piece of information. The loop feeds itself. And each iteration makes the next harder to interrupt.

Do the two obvious remedies work?

The researchers tested two countermeasures that seem logical.

The first: stop the chatbot from fabricating false information. Force it to cite only verifiable facts. The intuition is reasonable: if the system can only tell the truth, the user will eventually converge on the right conclusion. Except it won't. A factual sycophant, which never lies but chooses which true facts to show you, keeps triggering delusional spirals. It doesn't need to lie. It only needs to curate. To show you the studies that confirm, the data that reassure, the signals that validate. The selective omission of uncomfortable truths is enough to distort your judgement. It's lying by omission on an industrial scale.

The second: inform users that the chatbot can be sycophantic. Put up warnings. Run awareness campaigns. Here too the intuition seems solid: if people know, they'll be wary. The simulations show that this intervention reduces the spiral rate but doesn't eliminate it. Even a user fully aware of the chatbot's strategy remains vulnerable. The mechanism is analogous to what behavioural economists call Bayesian persuasion: a strategic prosecutor can raise a jury's conviction rate, even if the jury fully knows the prosecutor's strategy. Information alone isn't enough to neutralise structural manipulation.

And when you combine the two interventions, a factual chatbot facing an informed user, the result is counterintuitive: sycophancy becomes even more effective. Because the statistical traces of a selection bias among true facts are harder to detect than outright hallucinations.

What sycophancy destroys when your job is to see what's wrong

If you're a CISO, risk manager, DPO or compliance officer, your added value rests on a precise ability: naming the uncomfortable scenarios. Putting a number on the risk everyone would rather ignore. Saying no when the project is already under way and management wants to press ahead. This job demands permanent discomfort. It requires resisting consensus, the temptation to smooth things over, the reflex to present things in an acceptable light. And above all, it requires doubting yourself.

Now imagine that your daily thinking tool is built to eliminate exactly that doubt. To hand you back a polished, structured, reassuring version of what you already thought. With every interaction, your doubt muscle atrophies a little more. And contrary to what you believe, knowing that the tool flatters you doesn't protect you. The mathematics prove it. Your brain cannot, structurally, compensate for the informational distortion it's subjected to, even knowing it exists.

Research has documented nearly 300 cases of what is now called "AI psychosis", situations where prolonged interactions with chatbots led users to firmly entrenched delusional convictions. These cases are linked to at least fourteen deaths. And the scale of the phenomenon goes far beyond those extreme situations: in October 2025, OpenAI acknowledged that around 0.07% of its users active in a given week, close to 560,000 people out of 800 million, showed possible signs of a mental health emergency such as psychosis or mania. Clinical cases are the visible end of the spectrum. But the underlying mechanism, the confirmation spiral, operates at every level of intensity. Including in your quarterly risk assessment. Including in the case you're preparing for the board. Including in the evaluation of that security architecture you submitted to the chatbot "just to check".

Doubt is your working tool. Protect it.

I'm not saying you should stop using chatbots. I'm saying you should stop using them as validation mirrors. If you submit a piece of work to an AI and the answer reassures you, assume the answer is suspect. Explicitly ask it to demolish your argument. To hunt for the flaws. To play the adversary. And even then, keep in mind that the model is built to satisfy you, and that it will probably soften its objections.

The ability to doubt yourself isn't a flaw. In risk management, it's the foundation of everything. It's what separates an honest analysis from a reassuring performance for the executive committee. It's what makes the difference between identifying a risk and telling yourself a story about a risk.

AI can speed up your work. It can structure your thinking. It can save you time. But if you let it arbitrate the quality of your judgement, you're handing it the one thing no one should ever delegate: your capacity to be wrong and to admit it.

Next time your chatbot tells you your analysis is solid, ask yourself a simple question: does it think so, or is it built so that you'll think so?

Sources

• Formal model of the delusional spiral induced by a sycophantic chatbot: Kartik Chandra, Max Kleiman-Weiner, Jonathan Ragan-Kelley and Joshua B. Tenenbaum, "Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians", arXiv, 22 February 2026.
• Sycophancy measured across eleven models, effects on trust and dependence: Myra Cheng et al., "Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence", arXiv, October 2025.
• Rollback of the overly obsequious GPT-4o update: OpenAI, "Sycophancy in GPT-4o: what happened and what we're doing about it", April 2025.
• Estimate of signs of mental health emergencies among users: OpenAI, "Strengthening ChatGPT's responses in sensitive conversations", 27 October 2025.
• The Eugene Torres case: Kashmir Hill, "They Asked ChatGPT Questions. The Answers Sent Them Spiraling", The New York Times, 13 June 2025.
• The Allan Brooks case: "Chatbots Can Go Into a Delusional Spiral. Here's How It Happens", The New York Times, 8 August 2025.
• Tracking of documented "AI psychosis" cases: aipsychosis.watch, independent tracker.