Ten years ago, teams adopting microservices were navitating with trial and error. The patterns that are now obvious — Kubernetes, service meshes, circuit breakers — were developed as a coherent framework in years of practitiong. They crystallized and were documented slowly, through production failures and trial and error. Something similar is happening right now with AI agents.
A team building and running agents at scale on AWS (at Navan) has lived through enough production pain to map out where the field actually stands. Which of the layers are mostly solved, which are still difficult to address, and which remain open problems. The assessment is that agents in 2026 are roughly where microservices were around 2015: patterns are not yet settled.
What has stabilized
A few layers have matured to the point where practitioners can stop reinventing them. Cloud providers now offer solid agent runtime infrastructure, covering the stateful, session-persistent lifecycle that agents need and that traditional API services never did.
Tool calling has converged around MCP to the point where it is became industry-standard. Memory architecture — the component that lets an agent remember across sessions — has moved from simple retrieval-augmented search toward a structured pipeline: ingestion, extraction, consolidation, retrieval. These problems are not entirely closed, but reasonable answers exist.
Where the work is still going on
Observability is the layer that still does not fit neatly. Traditional logs collapse under the volume of an agent’s reasoning trace. One of the approaches is intercepting at pre- and post-tool-call hooks to auto-emit structured traces. This is practical, but what signals actually matter remains an open question.
OpenTelemetry (the emerging industry standard for distributed tracing) is the direction most teams are pushing, but whether it fits the shape of agentic calls is still being understood.
Testing is the other hard one. Agents are non-deterministic: the same goal produces different execution paths on different runs, which makes conventional pass/fail test assertions invalid. One approach gaining traction in research is trajectory evaluation — measuring how far along the path from start to goal the agent travels rather than asserting the exact steps taken. It is workable, but requires a meaningful shift in how engineering teams think about correctness.
Cost remains largely unsolved. Token usage by an agent is difficult to predict or cap in advance, vendor incentives run in the opposite direction, and replay for debugging is still emerging — one promising idea being to use a separate agent to help make sense of another agent’s trace, reducing the cognitive load on the human trying to debug.
The orchestration trap
Perhaps the most transferable lesson from the microservices era applies directly here: if a team could not build a well-structured monolith, adding microservices made things worse, not better. The agent equivalent holds. A single reliable agentic loop is harder than it looks; multi-agent orchestration compounds every unsolved problem.
The pattern that has worked at Navan is a single master agent that progressively loads skills — self-contained bundles of instructions and tool execution — composing context dynamically rather than front-loading everything into a fixed prompt. For large organizations where separate teams or systems need to hand off work across boundaries, the Agent-to-Agent (A2A) protocol offers a way to establish formal contracts between agents. But orchestration remains the area where over-engineering is most common and the distance between what looks clever in a demo and what holds up in production is widest.
What the next few years probably look like
The microservices analogy is useful precisely because it is honest about the timeline. The infrastructure that microservices practitioners now take for granted took the better part of a decade to mature. Agent infrastructure will not take that long — the field is moving faster and the lessons are being accumulated more deliberately — but anyone expecting a settled stack in the next twelve months will be disappointed.
What is more likely: the layers closest to production hardening (runtime, memory, tool calling) will consolidate quickly, while observability, testing, and cost control will take longer and may require purpose-built tooling that does not quite exist yet. The teams that will be in the best position are the ones building the foundation carefully now — one reliable loop at a time — rather than chasing the most elaborate orchestration pattern available.
Text summarized and optimized using Anthropic’s models and reviewed by a human.