LLM Wiki

A personal research wiki that reads arXiv papers and Substack articles and writes itself into a growing, cross-linked map of a field — instead of a pile of unread PDFs.

OngoingMedium priority

The problem

I read a lot of AI research — papers, technical reports, newsletter deep-dives — and almost none of it stuck in a form I could actually use later. I’d read a paper, form an opinion, and three weeks later re-derive the same understanding from scratch because the insight lived in my head or in a PDF I’d never reopen. That’s not a reading problem, it’s a compounding problem: knowledge that doesn’t accumulate into something queryable is knowledge you keep re-buying at full price.

That’s a company problem as much as a personal one. Teams generate huge volumes of AI-touched material — meeting notes, research summaries, strategy docs, Slack threads — that gets written once and never read again. Most orgs are making the “compounding asset vs. growing pile of noise” decision by accident, with no deliberate architecture behind it. The same failure mode that costs me a re-read costs a team a re-discovery meeting.

The approach

This is built on Andrej Karpathy’s “LLM wiki” pattern: drop a raw source into a folder, and an agent reads it, extracts the entities and claims, and updates a set of interlinked markdown pages rather than just filing the document away. My version runs as three layers — `raw/` (immutable source PDFs and snapshotted articles, never edited), `wiki/` (the agent-maintained markdown pages, one concept or paper per file), and a `CLAUDE.md` schema file that is itself the spec the agent follows for every operation.

Three operations, run by asking the agent directly, no separate app: **ingest** a new source (read it fully, update or create the 10-15 wiki pages it touches, cross-link them, flag contradictions inline with `⚠️`, update the catalog); **query** the wiki in plain language and get an answer with citations back to specific pages and source PDFs; and **lint**, a periodic health check that finds orphan pages, broken links, and stale or contradicting claims. In practice this has ingested over 20 arXiv papers (DeepMind’s AutoHarness, Meta/Yale on LLM-judges, JPMorgan’s QueryBandits, the Darwin Gödel Machine line, MCP benchmarks, and more) into roughly 90 pages and 290+ cross-links, with lint logs tracking page/link counts and zero broken links at each check. It also does its own editorial work: when a lint pass flagged a content gap — five papers all touching “where does agentic intelligence come from” with no page tying them together — the agent wrote a synthesis page for it, explicitly flagged as synthesis rather than a primary source.

The one nuance I built in deliberately: this wiki isn’t the only memory system I run. I also keep Open Brain, an MCP-backed structured store for preferences, decisions, and daily state. The schema encodes explicit routing rules — “what do we know about X” (shared, ingested subject knowledge) goes to the wiki, “what do I know” / “what’s my…” (first-person, personal state) goes to Open Brain — so the two don’t collide or duplicate.

What I learned

The real lesson came from reading about the pattern I’d already built: a Substack piece I ingested (ironically, into this very wiki) named the flaw precisely — every ingest is an editorial act. The agent decides what to keep, what to drop, how to frame a connection, and those choices are invisible unless something forces a re-check. That’s why lint isn’t optional decoration; it’s the only thing standing between “compounding understanding” and “confidently wrong prose nobody re-reads.” The schema itself turned out to be the highest-leverage artifact in the whole project — more valuable than any individual wiki page — because it’s the thing that keeps 90 pages internally consistent instead of an agent improvising conventions differently every session.

Where this could go

The natural extension is the hybrid the ingested article argued for: a structured database (Open Brain, in my case) that holds raw, queryable facts, feeding a scheduled synthesis job that compiles the wiki from it — rather than me manually dropping sources into `raw/`. At team scale this is the same architecture a knowledge-management or research function needs: a query-time store for “show me every doc from Q1 that mentions X” plus a write-time compiled wiki for “how has our understanding of this space evolved,” with a recurring drift-and-contradiction audit as a standing job, not a one-off cleanup. Most teams have the raw-material half (Confluence, Slack, a wiki nobody trusts) and skip the audit loop entirely — that’s the piece worth stealing from this project.

Takeaway

A wiki that writes itself is only as trustworthy as its willingness to be checked — the lint loop, not the ingest loop, is what makes this an asset instead of a liability.

Text summarized and optimized using Anthropic’s models and reviewed by a human.