StatusOngoing
PriorityHigh
Has UIYes
Sourcerich

Pi Integration

A compression proxy and CLI-agent troubleshooting toolkit that cuts LLM token bills by up to 92% and keeps agentic coding tools (Claude Code, Pi) actually running.

01 — THE PROBLEM

Agentic coding tools are token-hungry — every file read, search result, and tool output gets replayed back into the context window on every turn, and it adds up fast. I was watching Claude Code sessions burn through quota on nothing more exciting than re-sending the same JSON blobs and log dumps turn after turn. That’s not a personal quirk of my setup — any team running multiple engineers against Claude Max/API quotas or per-token API billing hits the same wall, and it gets worse as agentic workflows (multi-step tool use, RAG, long sessions) become the default way people write code.

On top of that, the tooling itself is fragile in ways that quietly kill adoption: OAuth callbacks that fail silently on Windows, CLI wrappers that don’t resolve the right executable, billing pools that are separate from what you think you’re paying for. None of that is a research problem — it’s the kind of friction that makes people give up on a tool in the first ten minutes, which is exactly the failure mode that matters when you’re trying to get a whole team onto new infrastructure, not just yourself.

02 — THE APPROACH

The core of this is `claude-proxy`: a launcher (`launch.py`, plus `.bat`/`.ps1`/`.sh` wrappers so it works the same from PowerShell, cmd, or bash) that starts the Headroom compression proxy on localhost, points Claude Code at it via `ANTHROPIC_BASE_URL`, and then runs Claude normally. Nothing about the Claude CLI itself changes — traffic just gets intercepted and compressed in flight. Headroom picks the right compressor per content type automatically: SmartCrusher for JSON, an AST-aware CodeCompressor for source code, Kompress for natural-language/agentic traces, and a CacheAligner that specifically protects Anthropic’s prompt-cache prefixes so compression doesn’t cannibalize the cache-hit savings you’re already getting. On the benchmarks I checked (code search, incident debugging, issue triage, codebase exploration) the reported savings ran 47-92% with accuracy held flat on GSM8K/TruthfulQA/SQuAD v2/BFCL.

To make the savings visible rather than theoretical, I built a `rich`-based terminal dashboard (`perf_dashboard.py`) with four sections — Overview (requests, tokens in/out, tokens saved, cache hit rate), By Model, By Transform (which compressor ran and how effective it was), and Throughput (p50/p95 latency for input, compression, and generation) — plus a `–watch` mode that live-refreshes it. It’s wired into Claude Code as a `/headroom-perf` slash command, so from inside any session you can just type it and get the numbers without leaving the chat. There’s also a split-pane launcher (`launch-split.ps1`/`.sh`) that opens the live dashboard in one pane and an interactive Claude session in the other, since a `rich` panel can’t host an actual interactive TTY program inside itself — that has to be two OS-level processes.

The other half of the project is Pi (pi.dev, a separate coding-agent CLI) integration and troubleshooting: fixing an OAuth login that failed on Windows with `EACCES` because Pi’s random callback port kept landing inside a Hyper-V/WinNAT-reserved range, clarifying that Pi’s Claude Max OAuth draws from a separate pay-as-you-go “extra usage” pool rather than your normal plan quota, and wiring up Gemini model support (API key vs. a custom `models.json` provider entry) so Pi isn’t locked to one vendor.

03 — WHAT I LEARNED

The most concrete lesson was operational: the compression model has a real cold-start cost (Kompress’s ML model takes 30+ seconds to load on first proxy start), so any health-check or launcher timeout has to be sized for that or it fails with a false negative on the very first run — subsequent launches, with the proxy already warm, are instant. I also hit a hard architectural boundary with `rich`: it can render live dashboards beautifully, but it cannot host an interactive program inside its own panel — that split has to happen at the OS/terminal level, not inside the rendering library, which is why the split-pane launcher exists as a separate mechanism rather than a fancier dashboard layout.

The broader pattern, reinforced across both the Pi debugging and the Headroom build, is that “wrap it in a proxy and swap a base URL” is a remarkably durable integration point — it’s how the Headroom proxy attaches to Claude Code, and it’s conceptually the same shape Pi itself uses for third-party model providers. When a tool exposes an OpenAI/Anthropic-compatible HTTP surface, you don’t need to fork or vendor anything to sit in the middle of it.

04 — WHERE THIS COULD GO

The build notes already call out the next phases: registering Headroom as a proper provider-proxy inside Pi (not just Claude Code), and documenting the direct SDK-wrapper integration path (`withHeadroom(new Anthropic())`) for cases where you don’t want a subprocess/proxy at all — useful for services rather than CLI sessions. There’s also an unresolved rough edge (Windows Terminal not reliably on PATH for the split-pane launcher) that’s a small but real adoption blocker worth closing.

The bigger direction is turning this from “my launcher script” into shared infrastructure: at team scale, the same proxy pattern generalizes to a shared compression gateway that every engineer’s agent CLI routes through, with the dashboard becoming a fleet-wide cost/savings view instead of a single-session one — the kind of thing a platform or DevEx team would run centrally so cost governance and cache-hit visibility aren’t left to each individual’s local setup. The Pi debugging work points the same direction: once you’re running more than one coding-agent CLI across a team, “which agent, which billing pool, which provider” stops being a one-off troubleshooting session and becomes a standing configuration decision someone needs to own.

05 — TAKEAWAY

Most of the value here wasn’t a clever algorithm — it was making an existing compression library actually observable and boring-reliable to run day to day, which is the unglamorous work that decides whether a token-saving tool gets used or abandoned after one bad launch.

Text summarized and optimized using Anthropic’s models and reviewed by a human.