For most of the past two years, the default picture of “working with an AI agent” has been the same: open a terminal or chat window, type a prompt, watch it run, nudge it along, repeat. That model felt cutting-edge when agents first appeared — and it still works. But it has a ceiling, and that ceiling is the person sitting at the keyboard.
The bottleneck hiding in plain sight
The shift happening now is a recognition that when the model’s capability has crossed a threshold — where it can handle a whole engineering task end-to-end, not just autocomplete a line — the human-in-the-loop stops being a quality gate and starts being a queue. Every pause waiting for your attention is latency introduced by the process, not by the technology.
The useful mental model is a spectrum with three positions. At one end: the human writes every prompt and watches every step (“in the loop”). In the middle: the agent does the work independently, the human reviews and approves at defined checkpoints (“near the loop”). At the other end: the agent runs on a schedule or in response to events, entirely without a human needing to be present (“out of the loop”).
Most practitioners today live at the in-the-loop end. The interesting engineering work has moved to the other two.
What “near the loop” actually requires
Moving one step along the spectrum — to the point where a human only reviews, not drives — turns out to require three things that most off-the-shelf agent tooling does not provide by default.
The first is durability: an agent that survives a closed laptop, a crashed session, or a network hiccup. Standard interactive coding assistants don’t qualify here — they are built for a conversation, and a conversation ends. A durable agent persists its state and resumes.
The second is isolated compute: a dedicated environment (a cloud virtual machine or container, always on) where the agent can run without competing with your local setup or dying when you walk away.
The third is a trigger layer: something that starts the agent without a human typing a message. A scheduled job, a webhook from a monitoring system, a new ticket in a queue, a log event — information arriving in the world, not a person deciding to engage.
Put those three together — durable agent, isolated sandbox, trigger — and the result is a system that can work for you while you are asleep.
Where the real engineering advantage lives
One pattern worth noting from practitioners who have moved down this spectrum: the biggest gains come not from choosing a more powerful model, but from building a better harness around a good-enough one. Out-of-the-box agent configurations tend to converge — everyone using the same defaults gets roughly the same results. Differentiation comes from custom tool definitions, well-engineered system prompts, and purpose-built evaluation steps baked into the loop itself.
An evaluation step worth understanding is the LLM-as-judge pattern: rather than checking agent output manually, a second model call scores it against a criterion and feeds the score back into the loop. That lets a system hill-climb toward a target — whether the target is visual similarity to a design mockup, signal quality in a news feed, or correct root-cause diagnosis in an incident — without a human scoring each iteration. The agent tries, evaluates, and tries again until a threshold is met.
The practical question
The frame that tends to move thinking in this area is concrete: what specific task currently requires a human to start it that does not actually require a human to do it? Routine monitoring, first-pass investigation, content classification, boilerplate generation, test generation — many engineering tasks fit that description if the system around the agent is built with care.
The near-loop and out-of-loop patterns are not exotic. They are closer to the standard cron job or the CI/CD pipeline than to science fiction: automated systems that run on a schedule, hand results to a human at a decision point, and wait. The difference is that the work inside the job is now language-model-shaped rather than deterministic-script-shaped.
What changes as more teams reach that point is not just productivity — it is also what the interesting engineering problems are. Writing prompts well enough that an agent can run without supervision, building evaluation loops that catch failures before the human review step, and designing triggers and handoffs that surface the right decisions at the right moment: those are the skills that matter on the far side of the in-loop bottleneck.
Text summarized and optimized using Anthropic’s models and reviewed by a human.