For a while, the conversation inside engineering teams focused on how much AI was costing. Token spend dashboards appeared. Some teams quietly competed to use less. A few organisations put usage leaderboards in front of engineers.
That framing is understandable — AI tooling is a new and often visible budget line — but it points at the wrong thing. The parallel that makes this clear is lines of code, or LOC. LOC is a useful signal: very low output from an experienced engineer warrants a conversation; a sudden spike might mean something went wrong. But the moment a team starts optimising for LOC — rewarding more lines, or penalising fewer — the metric turns toxic. Token spend works the same way. Tracking it is reasonable; minimising it as a goal is not.
The right question is ROI, not cost
The more useful frame is return on investment: what value are those tokens producing relative to what they cost? That shifts the conversation from “how do we spend less?” to “how do we understand where tokens are generating value and where they are simply burning?”
One company working through this concretely is Ironclad, a legal AI company. Their engineering team arrived at what they call “trusted throughput” as the proxy metric for AI engineering value. The phrase is doing real work: “throughput” signals that volume of shipped code matters, and “trusted” signals that quality does too.
Trusted throughput is validated across three dimensions. First, objective checks: test coverage, security scans, canary rollout compliance — things a machine can measure. Second, human judgment in code and architecture reviews — the harder-to-quantify assessment of whether a change actually fits the system. Third, customer validation: production incidents, rollbacks, the bug and usability tickets that only appear after something reaches users.
This is a more honest definition of value than any single proxy, and it is deliberately multi-layered because no single number captures engineering quality.
How the metric evolved
Getting to trusted throughput was not a straight line. The progression went roughly: lines of code (too gameable), open pull request count (inflection points visible but PRs are not all equal), merged pull request count (better), and finally merged PR count weighted by AI-estimated complexity — a t-shirt sizing prompt that asks a language model to assess how significant a change is. That last approach is still evolving, and the team acknowledges it openly.
What is useful here is not the specific metric — every team’s codebase and review culture is different — but the trajectory: moving toward measures that capture both volume and quality, while staying sceptical of any single number.
The bottleneck has moved
The less obvious implication of AI-assisted code generation becoming routine is that it shifts where the constraint is. When generating code becomes cheap and fast, the bottleneck is no longer writing code — it is reviewing it and getting it through the pipeline.
This creates a predictable failure mode. When CI/CD (continuous integration and delivery — the automated systems that test and ship code) is slow, engineers naturally respond by batching more changes into each pull request to avoid the overhead of many small submissions. Larger pull requests take longer to review and get reviewed less carefully. The problem compounds.
The two investments that address this are straightforward but easy to defer: using AI tooling as a first pass on code review, so human reviewers focus on architecture and judgment-heavy decisions rather than style and missing tests; and actually fixing CI infrastructure — removing flaky tests (tests that sometimes pass and sometimes fail without a clear reason), reducing wall-clock time from PR-ready to merged, and tracking how often each PR needs to retry before it passes.
Running AI agents in a loop to babysit flaky CI is not a solution. It wastes tokens and erodes the team’s confidence in the system.
Practical levers worth institutionalising
A few token-side techniques hold up in practice. Structuring prompts with a fixed system prompt at the top and variable content at the bottom allows model providers to cache the common prefix efficiently, cutting costs without changing output. Capping the number of steps in agentic loops — where an AI generates code, runs tests, fixes failures, and retries — prevents runaway spend. Saving well-crafted prompts for recurring tasks (bug fixes, new features, refactors) in shared internal playbooks beats letting each engineer reinvent the same prompt from scratch.
On build versus buy: the principle that holds is to buy commodity tooling — IDE integrations, CI infrastructure — and build the context-specific playbooks that encode the team’s own domain knowledge. That internal knowledge is the differentiator; the scaffolding around it is not.
Where this goes
The organisations that will get the most from AI-assisted engineering in the next few years are probably not the ones that minimise spend. They are the ones that instrument the right metrics early, invest in review and CI infrastructure before bottlenecks become acute, and treat token ROI as an ongoing management discipline rather than a one-time optimisation exercise. The goal is not austerity — it is understanding what the tokens are actually buying.
Text summarized and optimized using Anthropic’s models and reviewed by a human.