For the last few years, “AI-assisted development” has largely meant one thing: a smarter autocomplete inside someone’s IDE. Claude Code on a laptop, a code review bot at the PR gate, maybe a Copilot subscription across the team. That counts as progress. But there’s a meaningful gap between that and what is starting to be called a software factory — a gap that most engineering organisations haven’t crossed yet, and that is worth understanding clearly before trying to cross it.
What a software factory actually is
A software factory is an agentic loop — a repeating cycle of issue, triage, spec, implement, review, QA, deploy, and monitor — that runs in the cloud rather than on individual developer machines. The key word is loop: not a set of tools scattered across a team’s workflow, but a connected, observable system that can be measured, improved, and incrementally automated over time.
The contrast with today’s typical setup is sharper than it sounds. Most teams are what you might call IDE-shaped: agents live inside the editor, the developer is the integration point, and the only shared infrastructure is the CI pipeline at the end. A software factory moves the integration point up a level. The loop itself becomes the unit — and because it lives in the cloud, every run is traceable, every bottleneck is visible, and automation can be added step by step without rewriting the whole workflow.
Why most organisations haven’t built one
The honest answer is that the tooling problem is mostly solved. What remains is an organisational problem.
A software factory requires product and engineering to co-own the loop. Not just to coordinate on requirements — to literally share accountability for what the loop produces. In most companies, those two functions operate in separate reporting chains with separate goals. The factory model collapses that separation. That is both its power and the reason it stalls: it surfaces exactly the places where ownership is unclear or contested.
A second, subtler blocker is psychological safety. Moving from a private inner loop — where an engineer drafts, experiments, and discards privately before pushing anything visible — to a public cloud loop means working in the open. Teams where that shift feels risky default back to local tools, and the factory never gets used.
How to start (and how not to)
The failure mode to avoid is the big-bang redesign: deciding to automate everything at once and spending months building infrastructure before anyone ships differently. This is the same pattern as the six-month product build in isolation — and it fails for the same reason.
The alternative is to instrument before automating. The first step is simply to get everyone contributing to the cloud loop interactively, even if every step still has a human in it. Slack prompts triggering agents, Jira tickets kicking off triage, humans reviewing every output — none of it automated yet, but all of it tracked and measurable. Once the loop is observable, the question of where to remove human steps becomes empirical rather than speculative.
A useful framing: measure “automation index” as human interactions per pull request, and treat it as a continuous variable to bring down incrementally — not a binary switch to flip.
Metrics that matter (and one that doesn’t yet)
Once the loop is running, measurement becomes the driver of improvement. A few categories are worth tracking:
Cycle time and deployment frequency (DORA metrics) are the baseline — they tell you whether the factory is actually faster than what it replaced.
Cost by stage — spend per engineer, per pipeline step, per model — surfaces where the economics are working and where they aren’t.
Online evaluations: running a model as a judge over sampled recorded sessions to answer specific questions (“what share of agent runs generate redundant tests?”) is a more reliable signal than subjective engineering opinion. It also lets teams run controlled comparisons across models.
Custom benchmarks from real work: tagging completed tasks and using them to build an organisation-specific benchmark suite — essentially a private SWE-bench built from the team’s own codebase — is a more meaningful quality signal than public leaderboards. It requires environment isolation to avoid the model simply reading the finished pull request, but the investment pays off quickly in teams with enough throughput to generate a useful benchmark corpus.
The one metric to defer: business outcomes. Revenue impact and conversion lift are real signals, but they lag by weeks or months and are confounded by too many other variables to be useful for tuning the loop early on. Teams at very early stage (pre-product-market fit) should skip factory metrics entirely and watch user sessions manually instead — the factory model is suited to scale, not discovery.
Code review, reframed
One of the quieter shifts the factory model produces is in how code review works. When low-risk pull requests can auto-merge under a defined policy — say, anything under a size threshold that doesn’t touch core modules — code review shifts from line-by-line reading to risk management.
The side effect observed at teams that have adopted this approach is instructive: once engineers know that tighter, smaller pull requests flow through faster, they voluntarily start writing tighter, smaller pull requests. The policy shapes the behaviour, not just the automation. The classic asymmetry holds in reverse: a focused, well-scoped change gets serious attention; a sprawling diff gets waved through.
Where the practice is heading
The near-term frontier is not more powerful models inside existing tools — it is making the loop itself a first-class engineering artifact: defined in code, versioned, testable by branching. Teams that treat their software factory the way they treat their infrastructure — as something that can be reviewed, forked, rolled back — are building a compounding advantage that individual tooling improvements can’t replicate.
The harder question, and the one that will take longer to answer, is whether the organisational changes the factory model demands are ones most companies are actually willing to make. The track record of good tooling waiting for culture to catch up is long. The technology is rarely the constraint.
Text summarized and optimized using Anthropic’s models and reviewed by a human.