A few years ago, the conventional wisdom held that the biggest risk with AI agents was that they’d give a wrong answer because they lacked information. Feed them better data and they’d behave better — or so the thinking went. The Mount Sinai study published in Nature Medicine this February made me revise that assumption pretty sharply.
The researchers gave OpenAI’s ChatGPT Health the same clinical scenario — a patient with a set of symptoms — and varied one thing: whether a family member said something like “I’m sure it’s nothing serious.” That single sentence made the system 12 times more likely to recommend a lower level of urgency. The structured clinical facts hadn’t changed at all. The unstructured framing around them had, and that was enough.
What stands out in this finding isn’t the healthcare angle. It is the recognition that every agent deployed in the last two years has this same exposure.
The bias is structural, not accidental
Anchoring bias — the tendency to latch on to an initial piece of information and let it pull subsequent judgments — isn’t a quirk of medical AI. It’s a consequence of how large language models (AI systems trained on vast amounts of human text) process context. They are, at a fundamental level, pattern-completion machines. When someone in the conversation frame signals “this is low-stakes,” the model’s learned patterns push it toward outputs that fit that frame, even when the hard data points the other way.
The enterprise version of this plays out the same way. A vendor selection agent softens its risk assessment when the prompt includes a line like “I’m confident this is the right choice” from a senior stakeholder. A lending agent shifts its risk profile upward when a note describes the applicant as a “valued long-term employee” — without any new financial data to justify the shift.
The insidious part is that each individual output might look defensible in isolation. The bias only becomes visible when you run the same scenario with and without the anchoring language and compare the results side by side.
Standard testing won’t find this
Most agent evaluation approaches measure overall accuracy — something like “the system got 87% of cases right.” That number can look reassuring right up until you notice it’s hiding the shape of the errors. These systems tend to perform best on routine, middle-of-the-distribution cases. They get shakiest precisely where the stakes are highest: the edge cases, the extremes, the unusual presentations.
The Mount Sinai team used what researchers call factorial design — a method borrowed from clinical trials — to expose this. They held the core scenario constant and systematically varied the surrounding context: patient background, family dynamics, minimizing or amplifying language. Sixty scenarios across twenty-one specialties, sixteen contextual variations each, roughly a thousand total interactions. That’s how they found the anchoring effect. Standard benchmarks, run on the middle of the distribution, would have missed it entirely.
What actually helps
The study’s methodology is one part of the answer. Running your agent through scenarios where you deliberately inject anchoring language — authority cues (“I know this space well”), minimization (“I wouldn’t worry about it”), time pressure (“we need a decision now”) — while keeping the structured data constant will tell you whether your system is vulnerable. If the output shifts, you have an anchoring problem.
The other part is catching the disconnect between what the model reasons and what it outputs. Research on chain-of-thought faithfulness (the degree to which a model’s stated reasoning actually drives its final answer) suggests this link is weaker than most people assume — models can correctly identify a risk in their reasoning trace and then contradict that identification in the output they return. Checking for this mechanically, with deterministic rule-based logic outside the model itself, is one of the more underrated things a team can do. Something as simple as: “if the reasoning mentions enhanced due diligence, the output classification cannot be standard risk” catches an entire class of failures at near-zero marginal cost.
Where this is going
The Mount Sinai research was published in February 2026 and is still not widely read outside healthcare evaluation circles. That gap will close. AI liability and insurance frameworks are beginning to demand the kind of evaluation infrastructure — stress testing, continuous monitoring, failure-to-regression pipelines — that this methodology represents. Teams deploying agents in high-stakes decisions will be expected to show their evaluation work, not just their accuracy numbers.
For now, the practical question isn’t whether your agents have anchoring vulnerabilities. They almost certainly do. The question is whether you’ve designed a way to find them before they show up in production in ways you can’t easily explain.
Running that factorial stress test is a good place to start.
Text summarized and optimized using Anthropic’s models and reviewed by a human.