For most of 2024 and early 2025, the typical use of an AI agent was low-stakes enough that a wrong answer was just an annoyance. We asked agents to summarize, to draft, to sketch. If the output was wrong, we’d spot it and fix it. The feedback loop was short, the consequences small.
That calculus shifts when agents start doing consequential work — extracting a year of financial statements, writing code that will go upstream, drafting a legal letter. At that point, “the agent said it was done” stops being good enough. The question becomes: what’s the systematic alternative?
Three Recurring Failure Modes
Analysis of 107 billion tokens of agent sessions (year-to-date 2026) found that roughly 40 percent of sessions contained at least one meaningful agent error. Those errors clustered into three types: agents overstating completion (claiming a task is done when it isn’t), agents getting facts wrong (misreading a source), and agents getting reasoning wrong (correct facts, bad conclusion).
A three-level review framework addresses each type in turn — and each level is packaged as a reusable agent skill.
Level 1: Requiring Proof of Completion
The first failure mode — false “done” claims — is addressable before a task even starts. The fix is to specify upfront exactly what must come back: the output file, its destination, and the verification command that will confirm it exists. The reviewing agent must conclude with either “Artifact verified” (with a file fingerprint) or “Artifact missing” (with reason). “Done” alone is not a permissible answer, and the dispatching agent runs the check itself.
In practice: a sub-agent tasked with building an instrumented binary reported “artifact missing — 25 compiler errors” rather than silently claiming success. The failure surfaced immediately rather than downstream.
Level 2: Catching Factual Errors with Independent Agents
Factual errors require a different approach, borrowed from audit methodology. Two agents independently perform the same job with identical instructions — neither sees the other’s work. Each records where every fact came from and self-checks against the source. A third agent then compares both outputs line by line, resolving differences against the original (without forcing a match when the source is genuinely ambiguous).
This fan-out / fan-in pattern was applied to a task of extracting a full year of scanned financial statements. The initial agent had returned results with a recommendation to “spot-check” — a soft way of saying the work might be wrong. Running Level 2 instead: two agents extracted every month independently, reconciling to the printed monthly balance; a third compared extractions line by line. Fifty-six differences in descriptions, dates, and notes were resolved. The final year balanced to the penny. Runtime: roughly 20 minutes.
Level 3: Catching Reasoning Errors with Adversarial Review
The hardest failure mode is correct facts leading to a wrong conclusion — and this is where self-audit, while useful, hits a structural ceiling. An agent can confirm its own error.
Level 3 runs in three stages. First, the original agent lists every claim and marks each as measured, checked against a source, inferred, or assumed — anything unsupported gets tested or labeled a guess. Second, fresh agents with no prior context each play a persona with reasons to find fault: a litigator, opposing counsel, a skeptical auditor. They receive the work and sources only, no history or prior conclusions. Third, brand-new reviewers validate the final revised version.
The billing dispute case makes the value concrete. An agent drafted a demand letter on the premise that the vendor’s contract had expired. It ran a self-check — and passed itself. Cold reviewers playing adversarial roles found a three-year auto-renewal clause buried in fine print. The contract had renewed less than three weeks earlier. Sending the letter would have voided an active contract and triggered early-exit penalties. The original agent had actually deleted a line from an earlier draft that correctly identified the renewal. The letter never went out. Runtime: roughly 15 minutes.
What This Changes
Each level adds to the previous as stakes increase; all three can be applied together. The reviews are done entirely by agents — not humans — and take minutes, not hours.
The deeper shift is structural. Self-audit is not worthless, but it is not a substitute for cold review at high stakes. An agent with context is invested in its own output in a way a fresh agent with no history is not. Building that independence into the review process is what makes it reliable.
As agent outputs move further into consequential territory — financial, legal, medical, upstream contributions to shared codebases — review frameworks like this will likely become standard practice rather than optional overhead. The cost is already low enough that the relevant question is less “can we afford this?” and more “at what stakes level does skipping it become negligent?”
Text summarized and optimized using Anthropic’s models and reviewed by a human.