When the Hard Task Goes Better Than the Easy One
For years, the working assumption among practitioners has been that AI capability maps roughly onto human difficulty ordering. If something is genuinely hard for humans — reconstructing a codebase from scratch, proving a novel theorem — it should be harder still for AI. The easier, more familiar tasks would be reliable ground.
That assumption is worth revisiting carefully.
A recent case illustrates the gap precisely. One frontier model autonomously reimplemented a roughly 16,000-line bioinformatics toolkit — a body of specialized scientific software — by observing its behavior alone, without ever seeing the source code. The result was estimated to represent several weeks of skilled human work. The same model, given a comparable inference budget, then failed at a task estimated to take a human only a few days: customizing an existing data version control library for METR (an AI safety evaluation organization). The output contained obvious bugs and was unusable.
The question of why turns out to matter more than the result itself.
The Feedback Loop That Drives Improvement
The critical difference between the two tasks was not their objective complexity. It was how easy it was to tell whether any given attempt was getting better.
Reimplementing a codebase from observed behavior comes with a clear success signal: does the new implementation produce the same outputs as the original? Behavioral tests can verify this automatically, cheaply, and at scale. Each iteration gets honest feedback that the optimization process can use.
Customization requires something different. The developer has to decide how new behavior fits the existing design. There is no oracle to check against. The question is not “does this match a known answer” but “does this make sense given intent and context we have to infer.” That kind of judgment is harder to check and harder to reward during training.
This is the core of what researchers call checkable feedback asymmetry: tasks with cheap, reliable correctness signals — formal proofs, test suites, game scores — improve much faster than tasks requiring human judgment or long-horizon observation to evaluate. Requirement interpretation, design coherence, detecting that a specification is subtly wrong — these do not have that kind of feedback signal, and so they improve more slowly per unit of investment.
A Profile That Gets Spikier, Not Smoother
One intuition that deserves revision: as models get more capable overall, the expectation has been that capability profiles would converge — that strengths would bring up weaknesses, evening things out. The evidence suggests the opposite can happen.
When training rewards certain kinds of checkable success heavily, those domains improve fastest. Capabilities that rely on contextual judgment and incomplete-specification detection improve too — but from a weaker starting point, and more slowly. The absolute gap between strongest and weakest capabilities can widen even as the system improves across the board.
This pattern has a name: narrow superintelligence. It describes a system that is demonstrably extraordinary in specific activities while remaining dependent on human support for tasks a human would consider comparatively simple. The bioinformatics rebuild and the METR failure are not a calibration error or a fluke. They are what a spiky capability profile looks like in practice.
It is worth noting that mid-2025 benchmark analysis found broadly similar improvement rates across software and reasoning tasks — real evidence against pure spike divergence. But broad improvement and convergence are distinct things. Similar rates of improvement can persist alongside large absolute differences in capability level.
What This Changes
The economic implication is worth sitting with. If execution — the ability to carry out a well-specified task — gets dramatically cheaper while judgment — the ability to verify that the specification was right in the first place — remains scarce, the bottleneck shifts. It moves upstream: to the quality of the question, the clarity of the goal, the ability to detect when a requirement conflicts with what the project actually needs.
There is also an underappreciated risk on the other side. A stronger model that misunderstands a requirement does not just fail faster — it may implement much more of the wrong design before anyone notices. Capability amplifies both good and bad outcomes. The gap does not protect against misjudgment; it just changes where misjudgment surfaces and how much work it takes to undo.
Where the Practice Settles
The most honest position right now is that we are early in understanding these dynamics. What would settle the question is more demanding: a system that, unprompted, identifies when a stated requirement conflicts with the purpose of a project, investigates the conflict, and revises its plan without losing ground it had already established. Until that capability is reliably observable, the practice worth adopting is treating AI execution and AI judgment as distinct resources — one increasingly abundant, the other still genuinely constrained.
Text summarized and optimized using Anthropic’s models and reviewed by a human.