Systems Problem Formulation

A course-in-progress on how to formulate problems well enough that an AI agent can actually solve them.

OngoingMedium priority
The problem

The problem

Agents are now good enough at writing code that the bottleneck has moved. It used to be “can you build this.” Now it’s “do you actually know what to build, why it matters, and how to break it down so an agent can execute it correctly.” I kept noticing this pattern across interviews with people building agentic systems at real scale — the failures weren’t about model capability, they were about the human side: vague specs, wrong decomposition, not understanding your own system well enough to direct anything toward it.

This isn’t just a personal-productivity problem. Any team that’s handed execution to AI agents runs into the same wall: junior engineers who can prompt but can’t architect, senior engineers whose job quietly shifts from writing code to writing specs and reviewing decomposition, and orgs that don’t have a name yet for the skill they most need to hire and train for. If execution is cheap, the scarce resource inside a company becomes the same thing this course is about — people who can think clearly about a complex system and state the problem correctly.

The approach

The approach

This is a course, still in the ideas-and-source-gathering stage rather than a finished curriculum. It’s built inside my CogitOS pipeline, which treats a course as a project record: a manifest with an id, status, and priority; a running list of “ideas” pulled from source material and attached to the project; and reference summaries filed under the project so the raw material stays traceable back to where it came from.

Concretely, right now the project holds four sourced ideas pulled from talks by Andrej Karpathy, Geoffrey Litt, the Phaidra team (on why LLMs fail at large-scale entity lookup — “semantic blindness”), and Benoit Schillings of Google DeepMind — each one distilled into a full-paragraph note plus a link back to the original talk. The throughline across all four, and the organizing idea for the course itself: understanding is the bottleneck, not syntax; systems that scale by width (more instances, e.g. more GPUs) need to be represented differently to an LLM than systems that scale by depth (more hierarchy); and the “first day at a new company facing a 35-million-line codebase” moment is the actual test of whether someone can formulate a problem well. The curriculum being planned from this material teaches tradeoffs and systems-thinking over implementation detail — the parts that don’t get commoditized by agents doing the typing.

What I learned

What I learned

Building the source base before writing a single lesson forced a discipline I don’t normally have: don’t outline a course from what you assume the topic is, outline it from what the strongest people actually building agentic systems say goes wrong. The Phaidra example was the clearest case — “semantic blindness” isn’t a prompting problem, it’s a representation problem (linearize by hierarchy depth, not instance width), and no amount of clever prompting fixes a wrong data structure. That’s exactly the kind of tradeoff this course exists to teach, and I wouldn’t have found it by introspecting alone.

Where this could go

Where this could go

The next step is turning four sourced ideas into an actual module structure — likely one module per failure mode (context saturation, wrong representation, decomposition, spec-writing) rather than one per source. At team scale, the same material becomes an onboarding or L&D asset: instead of training new hires on a stack’s syntax, you train them on how to formulate problems for the agents they’ll be directing, and on the concrete failure patterns — like semantic blindness at scale — that syntax-focused training never covers. That’s a gap most engineering orgs don’t have curriculum for yet.

Takeaway

Takeaway

The course doesn’t exist yet, but the reasoning behind why it should is already sharper than most finished curricula I’ve seen — because it started from evidence, not from a syllabus template.

Text summarized and optimized using Anthropic’s models and reviewed by a human.