There is a finding from neuroscience that doesn’t make the AI coverage rounds, though it probably should.
Researchers at the University of Toronto and, more recently, a March 2026 MEG imaging study from Nottingham have found that “forgotten” memories aren’t actually gone. The brain retains a memory key — a compressed neural activation pattern that can reconstruct the full experience on retrieval. Think of it like a shorthand note that, when read later, unlocks the full memory of a conversation you thought you’d lost. Forgetting, it turns out, is a retrieval problem, not a storage problem.
A Google Research paper published March 25 did something mathematically analogous for transformer models — the architecture behind ChatGPT, Claude, Gemini and every other major AI system. TurboQuant compresses the KV cache — the working memory of any large language model — by 6x, with zero measurable accuracy loss. Instead of storing full-precision vectors (think of these as highly detailed, expensive snapshots of every word the model has processed), it stores a radius and a set of angles in polar coordinates: the minimum structure needed to reconstruct the original with high fidelity. Not everything. Just enough to remember accurately.
The neuroscience parallel isn’t decorative. It points to something structurally true about how intelligence — biological or artificial — handles the problem of bounded memory under the pressure of unlimited context.
To understand why this matters, it helps to know what the KV cache actually is.
Every time you have a conversation with an AI, the model doesn’t just read your last message. It holds the entire conversation in memory — every word you’ve written, every word it has responded with — and consults all of it every time it generates a new sentence. This running record is called the KV cache. It is, literally, the model’s working memory. The longer the conversation, the more memory it consumes.
Working memory has always been the binding constraint on the quality of thought — not raw processing power. A chess grandmaster who can hold twelve moves ahead in their head will consistently outplay someone who can only hold five, regardless of how fast either of them can calculate. The same applies here.
Andrej Karpathy — one of the founding researchers behind modern AI systems — once described the transformer as “a general-purpose differentiable computer.” A startup called Percepta took this seriously enough to compile an actual programming language interpreter directly into a transformer’s weights. The result: the model could execute real programs through its own reasoning process. To test it, they gave it the world’s hardest Sudoku puzzle. Rather than guessing, the model ran an actual systematic backtracking search — the same kind of exhaustive, logical elimination a human expert would use — and solved it with 100% accuracy.
The point isn’t the Sudoku. The point is what it reveals about the architecture: if the model’s weights are the processor, the KV cache is the RAM. And anyone who has watched a computer grind to a halt with too many browser tabs open knows what happens when RAM runs out.
Which means compressing the KV cache 6x isn’t just a cost improvement. It is equivalent to upgrading every GPU — the specialised chip AI models run on — from 80 GB to 480 GB of effective working memory, with no new hardware required. On a practical level, a server that could handle 9 simultaneous conversations can now handle 50 or more.
But the more important change is qualitative, not quantitative. More working memory enables multi-step reasoning chains that don’t collapse under their own length — the AI equivalent of being able to hold a complex argument in your head all the way to its conclusion, rather than losing the thread halfway through. It enables AI agents that maintain coherent awareness of what they’re doing across thousands of individual actions. It enables a coding assistant that can hold an entire software project in working memory while reasoning about a subtle bug buried three architectural layers deep.
This is not 6x more of the same. It is a different class of capability.
The infrastructure coverage frames TurboQuant as a GPU economics story, and the numbers are striking: nine concurrent sessions becomes fifty, revenue per chip multiplies, hardware contracts look different before you sign them.
All of that is true and worth knowing.
But the more interesting frontier is cognitive. The compression algorithms advancing simultaneously right now — TurboQuant, ShadowKV, DeepSeek’s latent attention redesign — are not just making existing AI cheaper. They are making longer, denser, more coherent chains of reasoning physically possible on hardware that already exists. The constraints that forced AI systems to “forget” earlier parts of a conversation, or to lose track of context mid-task, are being dissolved — not by building more powerful chips, but by learning to use the ones we already have more intelligently.
In that sense, the machines are learning to remember in the same way we do: not by storing everything, but by storing exactly what’s needed to reconstruct what matters.
The race everyone is watching is the chip race. The variable actually moving fastest is the one that determines how deeply a system can think before it forgets where it started.
Supply moves in years. Demand moves in quarters. Compression moves in days.
Text summarized and optimized using Anthropic’s models and reviewed by a human.