Why AI-Built Apps Break Under Use — and How Investigation Closes the Gap

A year or two ago the workflow seemed straightforward: describe a feature, let an AI coding assistant generate it, iterate until it looked right. The results were often impressive — screens that matched the brief, flows that held up in a demo. Then came the actual use.

The pattern that emerged was consistent enough to have a name: vibe-coded apps. They look convincing because the tools that build them are very good at producing the visual surface of software. What they do not automatically produce is the layer underneath — the rules that govern what data is valid, the relationships between records, the arithmetic that produces a report, the edge cases that exist because of how a business actually runs. That layer has to come from somewhere, and if it does not come from a precise specification, the agent fills in the gap with plausible defaults that may be wrong.

The gap is a specification problem

The hard part of building software has always been knowing what to build. AI coding tools have dramatically lowered the cost of generating an implementation once the requirements are clear. They have done almost nothing about the cost of getting the requirements right in the first place.

One approach that has gained traction is to start from an existing implementation rather than from a blank description. The practice — call it investigation-before-specification — is to study a working product to surface the decisions already embedded in it: what fields it tracks, how it calculates, what it does at the boundary cases. That understanding then becomes the foundation for writing requirements, rather than a vague feature request.

What REA does

REA (Reverse Engineer Anything) is an open-source tool that gives AI coding agents the ability to investigate software — including software whose source code is not available. It connects static analysis (via tools like Ghidra or Hopper, which decompile compiled programs into readable code), runtime observation, and browser inspection through a common interface that agents like Claude Code and Codex can call through MCP (a standardized protocol for connecting external tools to AI assistants).

Setup is a single command: `npx rea-agents@latest setup`, which registers the configuration and installs a bundled workflow skill. The only prerequisites are Node.js version 22.19 or higher (versions 23 and 25 are not supported). After restarting the agent, a small inspection prompt verifies everything is working — an integrity warning on first use with an Electron application (the runtime many desktop apps are built on) is expected behavior, not a sign of a broken setup.

The value is not automation. REA supplies capabilities; the investigation still requires domain knowledge. Knowing which behavior to investigate — and why — comes from understanding the problem well enough to spot the awkward cases a generic build request would miss.

How an investigation actually runs

The recommended sequence is narrow by design. Name one behavior and the decision it informs. Follow the information from user input through the records it touches to the result it produces. Let the requirements evolve as the implementation becomes visible. Then define a check: a predicted input-to-output pair that would reveal a wrong answer.

That last step — checkable understanding — is the part that closes the loop. An agent can describe how a clipboard operation works in general terms; it is harder to be vaguely right about a specific account balance that has to match to the cent.

Concrete investigations show where the approach earns its keep. A project called Ledger Reborn used REA together with Ghidra to trace inventory arithmetic, budget sign handling, and reconciliation balances inside QuickBooks Desktop for Mac. The resulting Python implementation then compared its own output against three sample-company reports account-by-account: a net profit figure of 113,560.69 dollars, total assets of 630,033.93 dollars, and trial-balance columns that agreed to the penny. A separate production portal investigation found something simpler but immediately actionable: the login form was reachable over plain HTTP on the tunnel hostname — a deployment divergence invisible from the source files, fixed with a single Cloudflare setting.

Neither of those findings would have surfaced from a feature description alone. They required looking at what was actually running, not what the project files described.

Where this points

The pattern points at something larger. As AI coding tools get faster and cheaper, the bottleneck in a build shifts further upstream — toward requirements, toward knowing what the correct behavior actually is. Tools that help close that gap, by investigating existing implementations or by making verification easier, fill a need that has been present since the beginning and is becoming more visible as the first generation of AI-assisted apps reaches production and breaks.

The quality bar has always been: does it work when someone actually uses it? Getting there is still the hard part. The tools for investigating that question are now better than they were.

Text summarized and optimized using Anthropic’s models and reviewed by a human.