The problem

Semantic code search only earns its keep if it’s cheap enough to run by default. The naive version isn’t: every time a developer opens a repo, you either re-embed the whole thing from scratch, or you let an agent grep through source files at query time and eat the token cost over and over — Turbopuffer’s own numbers put agentic grep traversal at roughly 6k tokens per sub-step, multiplied across every step, every developer, every session, even when two people are asking the exact same question about the exact same code.

That’s not just a personal annoyance — it’s the same bill an engineering org pays at scale, just bigger. A company with fifty engineers sharing a monorepo is running that same “re-index or re-grep” cost fifty times over for codebases that are 99% identical between checkouts. The waste compounds with team size, and it’s the reason semantic search stays an expensive opt-in feature instead of a default that’s just always on.

The approach

This project is a research study, not a shipped tool — I went looking for how the best-in-class implementations actually solve the re-embedding problem and wrote up what I found, with an eye toward building on it later. The centerpiece is Cursor’s approach, as described by Turbopuffer (the vector backend Cursor uses): a Merkle tree computes a compact cryptographic fingerprint of a codebase and identifies exactly which files changed between two versions, or between two developers’ checkouts of the same repo. When someone opens a repo that’s already indexed for a teammate, Cursor doesn’t re-embed anything — it copies the existing embeddings wholesale and only re-chunks and re-embeds the files that actually differ. That collapses the cost of indexing from “the whole codebase, every time” to “the delta since the last known version.”

I also looked at two adjacent approaches to see how the pattern generalizes. Deus Data’s Codebase Memory MCP treats the codebase as a structural graph rather than embedded text — it indexes an average repo in milliseconds, handles the full Linux kernel (28M lines) in about 3 minutes, answers structural queries in under a millisecond, and claims roughly 120x fewer tokens than having an agent read files directly. And a Recursive Language Models talk made the case that codebases aren’t documents at all — they’re directories, import graphs, dependency declarations, and test suites, and an agent that can execute against that structure (traverse imports, diff files, run tests) does things pure text-similarity retrieval can’t. The throughline across all three: index once, query cheap, and treat the embedding step as compute you cache — not data you rebuild.

What I learned

The clearest lesson is that “should we build a vector index” is the wrong question — the right one is “can we make indexing incremental.” Turbopuffer flags that the Claude Code team reportedly tried a local vector DB early on and dropped it, and reads that not as proof indexing is a dead end, but as evidence that the hard part is doing the incremental-diff mechanics well, not the embedding step itself. Content-addressed hashing is what turns “expensive full rebuild” into “cheap delta update,” and that distinction is the whole ballgame for whether semantic search is viable as an always-on default versus a feature nobody turns on.

Where this could go

The natural next step is to stop studying this and build a minimal version against a real repo — Merkle-fingerprint a directory tree, diff it against a stored snapshot, and confirm the re-embed set really does collapse to just the changed files. From there, the generalization is the more interesting part: the same content-addressed-hashing pattern applies to any corpus that evolves incrementally, not just code — a knowledge base, a set of internal docs, an inbox of tickets or emails. Any team maintaining a live semantic index over something that changes daily is solving this exact problem, whether they call it “codebase embedding” or not, and most of them are still paying the full re-embed cost because nobody’s bothered to make it diff-aware.

At org scale this stops being a nice-to-have and becomes a cost-control decision: a company running semantic search over a shared monorepo, a growing docs corpus, or a support ticket archive is either paying to re-embed the whole thing on some schedule, or it’s paying an agent to re-grep it every session — and both costs scale with headcount and corpus size in a way that incremental indexing doesn’t.