Multi-agent AI systems fail in ways ordinary software doesn’t. A pipeline can call the right tool with valid syntax, stay inside every permission boundary, get every API call green, and still produce the wrong outcome — because the failure is in the reasoning, not the traffic. Log aggregation and stack traces don’t catch that. Teams shipping agents into production keep discovering this the hard way: something goes sideways, and there’s no trace of what the agent actually intended to do, which tools it invoked in what order, or who approved the action along the way.

I built this course because “add some logging” isn’t an answer to that problem, and most engineers — including teams at companies already running agents in production — don’t have hands-on intuition for what agent observability actually requires. You can read about wide events, tracing, and control planes, but the concept doesn’t land until you’ve watched a pipeline you built go wrong and had to dig through its actual event stream to find out why.

The course is built around Cogniflow, a real multi-agent orchestration stack I wrote specifically to teach this, shipped in two parallel flavors so students can see the contrast between execution models directly on disk: a DAG flavor where pipelines run as a fixed sequence of agent stages start to finish, and a cyclic flavor where agents can loop back, message each other, and converge iteratively — the harder case, closer to how real multi-agent systems misbehave (deadlocks, runaway back-edges).

Each flavor pairs an orchestrator (the runtime that actually executes the pipeline as agent subprocesses) with a web UI that fuses two tools into one app: the Observer and the Configurator. From the Observer, a student opens a pipeline’s board, watches live agent status, drills into any agent’s exact context and tool calls, and can start, stop, pause, resume, reset, or approve a run — with a versioned history of prior runs if they want to compare. The Configurator is the authoring side: edit each agent’s system and task prompts, view the pipeline’s graph topology as a rendered DAG, validate that prompts include required sections, and use a “Specialize” button that hands a prompt off to the Claude CLI to refine it against a meta-prompt template. Both ship as double-click .exe/.app builds for non-technical students, with curated example pipelines that auto-seed into a working folder on first launch without ever overwriting a student’s own work.

The curriculum itself runs 17 modules across 8 parts with slides, narrated video, and seven hands-on labs of increasing complexity — from a 4-agent Proposal Writer DAG through a 7-agent pipeline, up to cyclic labs like an Article Review loop and a full Code Review Team with back-edges. Students aren’t reading about observability, they’re staring at their own pipeline’s event stream deciding whether the thing they just built actually did what they meant it to.

Building two reference implementations instead of one was the right call — the DAG-versus-cyclic contrast is nearly impossible to explain well in prose, but instantly clear once a student can point at two folders and see that one has back-edges and the other doesn’t. The seeding architecture (bundled example pipelines overlay onto a live, mutable data folder, additive-only, versioned by a marker file) turned out to be the unglamorous but load-bearing piece: without it, every course update either nukes student progress or leaves them on stale prompts, and getting that invariant right up front saved a lot of support pain later.

The deeper lesson came from the research I fed into the course itself: naive tracing degrades fast. OpenTelemetry’s flat attribute types force you to JSON-stringify structured agent output, which makes traces unqueryable and pushes teams toward selective sampling — which defeats the point, since observability is only valuable in hindsight, for the failure mode you didn’t know to look for. And agents themselves are bad anomaly detectors; they’re much better used to investigate a signal a deterministic check already flagged. That reframing — instrument everything, trace like a typed system, use agents for investigation not discovery — shaped the labs more than any UI feature did.

The Observer/Configurator split is basically a lightweight version of what companies pay LangSmith, Langfuse, or Datadog’s LLM Observability for once they have more than one agent pipeline in production: a place to see what’s running, what it’s costing, and what it’s touching, next to a place to change how it behaves. A team that’s already committed to a vendor platform could still use Cogniflow’s pattern internally — pairing a cheap, self-hosted execution/inspection loop with their production tracing — as a training ground before agents ever touch customer data.

The cyclic flavor’s UI is still unbuilt, which is the obvious next increment, but the more interesting direction is treating the two triage questions from the research — when did this start, and what percentage of users does it affect — as first-class fields in the Observer rather than something a student has to reconstruct by hand from the event log. That’s the same discipline production on-call teams already use; baking it into the tool would make the course’s lesson also the tool’s default behavior.