StatusOngoing
PriorityLow
Has UIYes
Sourcerich

Headroom Compression

A drop-in compression proxy that cuts Claude Code’s token spend 60-90% without touching answer quality.

01 — THE PROBLEM

Agentic coding sessions burn tokens fast — every grep result, log tail, and file read gets stuffed into context and paid for again on every subsequent turn. I hit this directly running Claude Code day to day: my Anthropic quota would exhaust mid-session on ordinary work, not exotic workloads. That’s not a personal quirk, it’s the standard failure mode of any team running agentic coders against a metered API — the more useful the agent gets at using tools, the faster it eats its own budget, and nobody on a team wants to ration prompts to stay under a quota.

The naive fixes are bad ones: truncate tool output and lose information, or summarize it and lose precision. I wanted the actual savings without either tradeoff, and I wanted to see the numbers, not take a vendor’s word for it.

02 — THE APPROACH

This project is an integration, not a from-scratch compression algorithm — I evaluated the open-source Headroom library (headroom-ai on npm, Apache 2.0) against my own real usage and built the pieces to actually run it. The core is `claude-headroom`, a CLI wrapper (`claude-headroom.bat`/`.ps1`/`.sh` plus `launch.py`) that starts a local compression proxy on port 8787 and points Claude Code’s `ANTHROPIC_BASE_URL` at it — you type `claude-headroom` instead of `claude` and everything else about your session is unchanged. There’s also `launch-split`, which opens Claude Code and a live terminal dashboard side by side.

That dashboard (`perf_dashboard.py`, a Rich-based TUI with a `–watch` live-refresh mode) is the part I’d point people to first: an overview panel with total requests, tokens in versus out, tokens saved and the percentage, and cache hit rate with read/write token counts, plus a per-model breakdown table (requests, in, out, saved, savings%, $/Mtok). Running `headroom perf` gives you the same numbers from the CLI after any session. I also built a Pi agent extension (my own agent framework) that hooks compression into the tool-result and provider-request boundaries directly, with its own slash commands: `/headroom` for status and last 10 events, `/headroom-on`/`/headroom-off` to toggle mid-session, `/headroom-learn` to mine failed sessions for corrections it writes back to CLAUDE.md, and `/headroom-stats` to inspect the persistent JSONL log.

The compression itself routes each message through a content-specific compressor — AST-aware code compression, a JSON/structured-data compressor, and a model trained on agentic traces for natural language — and a CacheAligner component that specifically protects Anthropic’s prompt-cache prefix, so you don’t compress your way into losing a bigger cache discount. Nothing is silently thrown away either: compressed originals are cached locally and retrievable, so the model (or you) can always pull back the full content on demand.

03 — WHAT I LEARNED

The number that mattered wasn’t the vendor’s benchmark table, it was what happened in my own sessions: real Claude Code runs against real repos landed at 60-79% token savings with zero fallbacks, and the model still completed the tasks correctly using the compressed content. I only trusted that once I’d run it myself rather than citing the README.

The bigger lesson was about cache interaction, which the marketing material glossed over: compression that touches the system prompt can invalidate Anthropic’s prompt cache, and on a large-context call that cache loss costs more than the compression saves. The safe rule I landed on was to never modify the system message and to compress only the trailing tool-result and history content — cheap to state, easy to get wrong, and the actual reason a naive “just compress everything” approach would have been a net loss rather than a win.

04 — WHERE THIS COULD GO

The proxy design generalizes past my own terminal: point-a-baseURL-at-it works for anyone’s existing Anthropic or OpenAI-shaped client (LangChain, LlamaIndex, raw SDK calls) with no code change, which is exactly the shape a platform team would want if they’re trying to control agentic-coding spend across a whole engineering org rather than one developer’s machine. Run one shared proxy, tag traffic per team via the stack header it already sends, and you get per-team cost attribution for free out of the same dashboard.

Unshipped pieces I’d build next: a provider-registration mode so compressed models show up as a normal model choice without installing anything locally, output-token shaping (Anthropic’s Opus pricing makes output tokens 5x more expensive than input, and this is untouched so far), and exposing the reversible-compression retrieval as a callable tool so the model can pull back an original on demand instead of it just sitting in a cache. Any of those turns this from “one engineer’s proxy” into infrastructure a team plugs into once.

05 — TAKEAWAY

Most of the token bill on agentic coding isn’t reasoning, it’s redundant context — and once you can see that split on a dashboard instead of guessing at it, cutting it 60-90% without losing answer quality stops being a tradeoff and starts being obvious.

Text summarized and optimized using Anthropic’s models and reviewed by a human.