For a long time, the standard wisdom about AI-assisted development was: write better prompts. Sharpen the instructions, give more context, iterate on the wording. That advice was fine when agents were doing small, bounded tasks. As agents take on entire features — end-to-end, with real state and real UI — something more structural is needed.
The gap that keeps emerging is not one of capability. It is one of verification. An agent finishes, announces success, CI goes green, and then a human looks at the running app and finds that data is not being saved, the confirmation message is wrong, or a form silently swallows input. The agent was not lying, exactly — it checked what it knew how to check. But “what it knew how to check” was the code, not the outcome.
Why code-review verification falls short
Code review — whether by a linter, a type checker, or a second LLM — operates on what was written. It can catch structural errors. It cannot tell you whether the right rows appear in the database after a user submits a form, or whether a screenshot of the live page matches what the ticket described.
This distinction matters more as agents become more autonomous. An agent running a long task accumulates small misunderstandings. Each one is below the threshold of a test failure, so the gates pass. The final state can be subtly broken in ways that only show up when a human actually uses the thing.
Outcome-driven verification addresses this directly: instead of asking “did the agent produce correct code?”, ask “did the intended user action produce the intended state change?” Those are different questions, and answering the second one requires different tooling.
The verifier CLI pattern
The most practical approach is to build a small, project-specific command-line tool that the agent can invoke to check its own work. Not a test suite in the traditional sense — a purpose-built verifier that understands the app’s specific actions and outcomes.
A minimal version covers a few operations: start the app, run a health check, perform the target user action (submit the form, trigger the flow), probe storage to confirm the state was written, and capture a screenshot. Each step returns a deterministic pass or fail with enough detail for an agent to self-heal.
The key properties:
– Deterministic outputs. A prose instruction like “reload and check the result” leaves room for interpretation. A CLI command does not. Thirty agents given the same CLI command produce thirty comparable outputs.
– Outcome-level evidence. The verifier checks what actually happened — database rows, network responses, rendered pixels — not just whether the code compiles.
– Reusability across the team. Once the tool exists and its usage is documented in a shared file (an `agents.md` at the repo root works well), every agent working on the project benefits from it without any additional setup.
The loop this enables: agent does work, agent runs verifier, verifier returns detailed pass or fail, agent self-corrects if needed, agent runs verifier again. The human enters the picture at the end, presented with a PR that includes screenshots and storage-state evidence as attached receipts.
Fuzzy checks alongside exact checks
Some correctness criteria resist exact matching. Confirmation text changes, UI copy evolves, and string equality breaks on minor wording shifts. A fast, inexpensive language model (running locally or via a cheap API endpoint) used as a semantic judge handles these cases well — it can evaluate whether the displayed message conveys the right intent without requiring it to match a specific string.
This two-track approach — deterministic checks for what can be measured exactly, semantic checks for what needs judgment — keeps verification honest without making it fragile.
Saving the receipts
Every verifier run produces evidence: screenshots, storage snapshots, network logs, CI pass or fail records. These are useful for debugging today. They are more valuable over time.
A team that accumulates clean, structured run receipts has the raw material for an evaluation dataset — a set of known inputs and known correct outputs that can be used to tune prompts, compare models, or drive automated fix cycles. Teams not collecting this data are passing up a compounding advantage. The infrastructure cost of storing it is low; the cost of not having it, later, is high.
Prescribe the destination, not the route
One broader principle ties this together. Current frontier models are capable enough to find their own path to a well-defined outcome. Over-specifying the steps — turn-by-turn instructions rather than a destination and a verifier — constrains that capability without improving the result. The more productive framing is: tell the agent what “done” looks like, give it a tool to check whether it has arrived, and let it work.
That shift — from prescribing procedure to verifying outcome — is the design move that unlocks real autonomy. The verifier CLI is not just a debugging convenience. It is the mechanism that makes outcome-driven development tractable.
The next step for teams already doing this is to close the feedback loop further: route failure receipts back into automated fix cycles, have a monitoring agent watch production after an agentic deploy, and treat the receipt archive as a first-class asset alongside the codebase itself. That is the direction this is heading, and it is not far off.
Text summarized and optimized using Anthropic’s models and reviewed by a human.