Open Brain
A self-hosted memory layer that gives every AI tool I use the same persistent, semantically-searchable context — instead of five sticky notes on five separate desks.
Every AI platform I use — Claude, a coding agent, whatever tool I pick up next month — starts from zero. Each has its own “memory” feature, but that memory is locked inside that one product. I’d re-explain the same project context, the same preferences, the same decisions, every time I switched tools or opened a new session, and none of it carried forward or connected to anything I’d captured before.
That’s not just a personal annoyance — it’s the same lock-in problem enterprises are walking into right now, just at bigger scale. A company standardizing its agent workflows on one AI platform’s built-in memory is making a long-term architectural bet by accident: twelve months of an agent connecting your CRM data to your engineering decisions to your Slack debates creates a layer of accumulated understanding that cannot be exported. Unlike a SaaS data lock-in (still just a CSV export away), that comprehension lock-in only gets more expensive to walk away from the longer the platform runs. I built Open Brain to test whether that dependency is actually necessary, or whether memory can be owned infrastructure instead of a vendor feature.
Open Brain is a fully local, self-hosted knowledge base: Postgres with pgvector for storage, running in Docker, talking to any AI client through a standard MCP server over stdio. There’s no cloud API in the loop for storage or retrieval — the whole thing runs on my own machine, and the embedding model (a 384-dim sentence-transformer) and re-ranker (a cross-encoder) run on CPU, so no thought I capture ever leaves the box unless I explicitly turn on LLM synthesis.
Concretely, it exposes eight tools to any MCP-connected client — capture_thought, search_thoughts, browse_thoughts, get_thought, update_thought, delete_thought, list_tags, and stats — so from inside a Claude Code or Claude Desktop session I can just say “remember this” or ask a question and get back semantically relevant past notes, not just keyword matches. Search runs a two-stage pipeline: pgvector’s HNSW index pulls the top 20 candidates by cosine similarity, then a cross-encoder re-ranks them down to the 5 most relevant before anything gets returned — a retrieval-then-rerank pattern that’s standard in production search but rarely wired into a personal note store. An optional third stage pipes the top results through whichever LLM CLI I have configured (Claude, Gemini, Copilot, or OpenAI — I built the adapter as a generic CLI shell-out so swapping providers is a one-line config change) for a narrative synthesis, but that’s off by default, so the base system works 100% offline after the one-time ~160MB model download.
I also built a self-contained installer (a stdlib-only Python script plus platform bootstrappers) that sets up the whole stack — Docker container, Python venv, models, MCP registration, and three Claude Code hooks (session-start context load, pre-prompt consult, post-turn auto-capture) — on a fresh machine with one command, idempotent and fully backed-up on every file it touches. And because MCP tools are namespaced by server name, I designed it so a second, fully independent instance (say, a domain-specific knowledge base separate from my personal one) can run alongside the first with no collisions — useful for keeping, e.g., a team or client knowledge store isolated from personal notes while both stay queryable from the same agent.
The biggest lesson was that retrieval quality is decided before the model ever sees the prompt — in chunking, metadata, and the extra re-ranking step — not in which embedding model you pick. Raw vector similarity alone was noticeably worse than vector-retrieve-then-rerank, even with a small, fast cross-encoder. I also learned the hard way that Windows has its own landmines that have nothing to do with the AI part: localhost resolving to native Postgres over IPv6 instead of my Docker container, mojibake from an un-forced UTF-8 stream on stdio, and quoted values in .env silently breaking psycopg2 — all failure modes that would never show up on the demo machine a tutorial was written on.
The deeper architectural lesson, reinforced by everything I researched while building this, is that memory-as-feature and memory-as-architecture are different things. A vendor’s built-in “memory” is passive accumulation you don’t control; a real memory layer needs deliberate separation — vector search for semantic recall, structured metadata for filtering, explicit provenance on what’s stored so an agent can’t later treat its own inference as fact. Getting that separation right up front is what makes the store something I can extend safely later (add a tool, add a column, swap the embedding model) instead of something that has to be rebuilt.
The natural next step is treating this as infrastructure rather than a single install: multiple isolated instances (personal vs. domain-specific vs. team) coexisting behind the same protocol, each with its own database and access boundary, which the MCP namespacing already supports cleanly. I’ve also worked through — though not yet shipped — moving it off a home PC entirely, onto a proper HTTPS endpoint behind a reverse proxy and tunnel, with bearer-token auth, so any AI platform I use, anywhere, can query the same store without my machine needing to be reachable directly.
At team or company scale, the same shape solves a sharper problem: instead of each employee’s agent starting cold, or the org standardizing on one vendor’s memory and inheriting that platform’s lock-in, a shared, owned knowledge store with per-consumer scoped access (this tool gets read access to project memories, that tool gets only calendar preferences) lets multiple agents and multiple people draw on the same accumulated context without any one of them owning it. The database and the protocol are the durable layer; the model behind any given agent becomes swappable.
Open Brain is a bet that the part of an AI workflow worth owning isn’t the model — it’s the memory, because the model changes every quarter and the accumulated context is what actually compounds.
Text summarized and optimized using Anthropic’s models and reviewed by a human.