Enhanced RAG
A living, source-cited knowledge base that tracks why naive RAG is dying and what’s actually replacing it.
The problem
I kept hitting the same problem watching AI Engineer talks and reading vendor posts: a dozen different people (Turbopuffer, Pinecone, Microsoft’s Azure AI Search team, WorkOS, Neo4j) were all describing pieces of the same architectural shift in retrieval, but each source only gave me one fragment, and none of it stuck. I’d watch a great talk, nod, and then re-derive the same insight three months later from a different talk because I had no durable place to put it.
That’s not a personal quirk — it’s the standard failure mode for any team tracking a fast-moving technical debate. Engineering orgs sit through conference talks, vendor briefings, and design reviews, extract a good insight, and then lose it in a Slack thread or a forgotten doc. The next person on the team re-litigates the same question (“is RAG dead?”, “vector vs. graph vs. hybrid?”) from scratch because there’s no accumulated, citable answer sitting anywhere.
The approach
Enhanced-RAG isn’t a deployed retrieval system — it’s a structured knowledge base built by my own CogitOS project pipeline, and what it actually contains is the more interesting part. Every entry in it is an “idea” captured via a `capture_idea` tool call: a dated ID, a full argument written out in prose (not a bullet fragment), and a citation back to the exact source — a specific YouTube talk with timestamp anchor, or a specific article bundle. Right now it holds over 20 of these, each one a distinct, well-formed claim: why bigger context windows make retrieval quality harder rather than irrelevant (Jeff Dean’s “you need the right million, not a trillion” point from the Turbopuffer talk), why vector RAG can’t do aggregation or multi-hop reasoning and should route those queries to a graph database instead (from an AWS talk on hallucination), why hybrid retrieval (vector + BM25 + structured filters) beats any single method by a wide margin in Microsoft’s internal evals, and why enterprise retrieval increasingly means querying governed structured data — not indexing a PDF export of it.
The project also has a `manifest.yaml` (id, status, priority, and the full list of source references) and a `refs.md` file that syncs local copies of every cited source’s summary, so the knowledge base is self-contained — you can read the argument and check its source without hunting down where the original bundle lives. The organizing thesis that ties the whole thing together, and that a reader gets by skimming the top of the file: “RAG is dead” is the wrong framing. What’s actually dying is the 2023-era pattern of one embedding lookup stuffed into a prompt. What’s replacing it is an iterative loop — multiple retrieval calls, reasoning over partial results, mixing semantic and lexical search, pulling only what the sub-task needs.
What I learned
The concrete lesson is the content itself: RAG going away is a myth, but naive one-shot RAG going away is real, and the actual replacement pattern is retrieval-as-a-loop combined with the right retrieval unit for the job — chunks for prose, tables for numbers, graph traversal for multi-hop and aggregation, governed source-of-truth queries for anything that has to be operationally correct.
The meta-lesson, about the project itself, is that a knowledge base only stays useful if capture is cheap and provenance is non-negotiable. The moment I let myself write an idea down without pinning it to its exact source, the entry becomes unverifiable noise within a month. Forcing every capture to carry a citation back to a timestamp or an article section is what makes the difference between a knowledge base and a pile of half-remembered notes.
Where this could go
The same pattern generalizes past personal research hygiene into something teams and companies need structurally: a durable, per-topic, source-cited knowledge base that survives staff turnover and doesn’t force every new hire to re-litigate settled architectural debates from a cold start. Instead of a wiki page that goes stale the day it’s written, this is closer to an append-only, cited ledger of “here’s what we currently believe about X and why, with receipts” — which is exactly the kind of institutional memory that AI-heavy engineering orgs are bad at keeping and good at needing, especially as the field moves fast enough that last year’s best-practice architecture is this year’s anti-pattern.
At team scale, I’d extend this to other fast-moving technical questions the same org faces — agent memory design, orchestration patterns, evaluation methodology — each as its own tracked knowledge-base project rather than scattered meeting notes. The interesting next step is closing the loop: right now capture is manual per-source; the natural evolution is an agent that watches a team’s actual inputs (talks attended, PRs reviewed, incidents worked) and proposes captures automatically, the way this project already treats YouTube talks and Substack articles as raw material.
Text summarized and optimized using Anthropic’s models and reviewed by a human.