When that benchmark first surfaced a few months ago, it warranted a second read. Same model — Claude Opus 4.5, identical weights — scored 78 percent on one evaluation harness and 42 percent on another. That is not a rounding error. That is nearly double the performance from the same brain with different scaffolding around it.
That gap has been worth thinking about ever since, because it changes what we should be measuring.
What a “harness” actually is
“Harness” is jargon. In plain terms: the model is the reasoning engine, and the harness is everything else — the environment where it runs, the memory system it leans on, the tools it can reach, the way it hands work across sessions, and the loop of checks and corrections around each output. Two tools built on the same model can behave as if they were built on fundamentally different intelligence, just because the harness is different.
The benchmark in question — the CORE evaluation from the AI Engineer Summit — compared Claude Code and Smolagents, both running Claude Opus 4.5. Claude Code’s harness produced 78 percent accuracy; Smolagents’ produced 42 percent. If you were benchmarking models, you would have concluded the models were different. They were not.
Two philosophies, not two tools
What struck me about comparing Claude Code with Codex was not the feature list. It was the underlying philosophy.
Claude Code bets on the agent doing the remembering. It writes structured progress files, commits descriptively, and stores capabilities as small files the model loads on demand. The human stays close — working alongside the agent, routing tasks, making the calls the model should not make alone.
Codex bets on the codebase doing the remembering. It runs in isolated cloud containers, treats the repository as the single source of truth, uses linter errors as remediation instructions, and has background tasks automatically scan for deviations and open refactoring pull requests. The model operates more autonomously; the human reviews rather than co-pilots.
Neither philosophy is obviously wrong. They are different architectural bets about how work should flow between people and machines.
The compounding problem nobody is pricing
Here is where the real risk sits: it is not the initial tool choice. It is the automation you build on top of it.
One team built six layers of workflow automation, each one depending on harness-specific capabilities of the layer below it. Individual commands grew into scripts, scripts became part of the daily process, and the process became the workflow. None of those layers translate to a different harness. Switching is not learning a new interface — it is rebuilding every layer from scratch.
Claude Code represents about four percent of all public GitHub commits today. Independent analysis puts that figure above twenty percent by the end of 2026. Whether those projections land precisely or not, the direction is clear: teams are compounding workflow investments on specific harness architectures every single week. The architectural commitment accumulates whether people are conscious of it or not.
The cloud infrastructure analogy from 2010 keeps coming back to me. At that moment, Amazon Web Services and Microsoft Azure both offered virtual machines and object storage — technically comparable. Organizations that understood the architectural differences early made better long-term decisions. The rest spent years and significant budget migrating.
What to think through before choosing
Any team evaluating AI developer tooling right now should push for three things before signing up for anything.
First, map actual task types: Claude Code fits collaborative, context-heavy work where the model needs a deep understanding of a large codebase. Codex fits many independent, parallelizable implementation tasks in sandboxed environments. These strengths are not interchangeable.
Second, audit the automation already in place: if your team has written custom scripts, aliases, or workflows around a tool, count them. That is your current compounding investment — and those layers go to zero when you switch harnesses.
Third, treat harness philosophy as organizational philosophy: the real question is not “which tool scores higher” but “which model of how humans and AI divide work fits how we actually want to operate?” That question deserves a deliberate answer, not a free-trial instinct.
The 36-point performance gap is striking precisely because it is not about the model. It tells us that the architectural decisions made around an AI — the harness — are doing a large fraction of the work we tend to attribute to the model itself. As the market matures and model capabilities converge further, the harness is going to be where differentiation actually lives. That makes it worth treating as a first-class strategic decision, not an implementation detail.
Text summarized and optimized using Anthropic’s models and reviewed by a human.