Token Compression Is the Wrong Way to Cut Your AI Bill

I understand why some of the teams we work with have asked us for token compression. RTK's GitHub page promises 60 to 90% fewer tokens on common dev commands while Caveman advertised 65% fewer output tokens. If the bill fell by anything close to that, I'd want it too.

Over the past year I've read a number of analyses that priced these tools on a coding agent's actual bill. None of them found savings close to the advertised numbers. The most recent ones find that compression can make the bill bigger. The best description I've heard for these tools is "AI bro science."

If we built compression because people asked for it, I'm fairly sure we'd raise some of their bills and take the blame for a poor compressor when the problem is the idea.

#How the tools work

RTK, the Rust Token Killer, rewrites the shell commands an agent runs so that git status or a test run comes back terser. Caveman began as a skill that makes the agent's own replies terse ("Why use many token when few do trick") and now compresses some of what the agent reads as well. Headroom is a proxy that sits between the agent and the model provider. It compresses tool output, logs, files and search results before each request goes out.

The tools that compress what the agent reads share a design. They swap part of the content for a shorter version, leave a pointer to what they took out and keep the original beside themselves. When Headroom compresses a tool result, it stores the original under a hash in a local SQLite file and puts a marker like [1000 items compressed to 20. Retrieve more: hash=abc123] in its place. It also adds a headroom_retrieve tool to the model's tool list so the model can ask for the original back. Its proxy keeps originals for 30 minutes by default. RTK does the same when a command fails or a filter cuts a list short. It writes the raw output to its own SQLite database and prints a hint such as [full output: rtk recall 3f9c2a81d4e7b05a] for the agent to follow.

So the model works from a summary of what its tools returned and has to decide, from that summary, whether it needs the rest. I call it a hack because a second program now decides what the model sees and the agent pays for every mistake that program makes.

#What the benchmarks found

JetBrains has been running paired A/B tests of these add-ons on SkillsBench, 86 coding tasks run in Claude Code with and without the tool. Forced on for every reply, Caveman cut output tokens by 8.5% against the 65% it advertised. Most of what a coding agent writes is code, diffs, tool calls and error messages. Caveman leaves all of that alone. Per task, the smaller output still made Caveman close to 10% cheaper. That's the best independent result I've seen for a compressor, though one runaway task erased the saving in the raw totals of JetBrains' full run. Caveman's own README now calls "Be concise" the real baseline. Against an instruction to answer concisely, its default mode writes 3% fewer output tokens at the median and its terser ultracave mode writes 35% fewer.

RTK did worse. In JetBrains' RTK test, the median task cost 7.6% more at low reasoning effort and took 13.8% more turns. At high effort the difference vanished. Quesma spent over $1,500 testing RTK on Terminal-Bench 2.1. With Claude Fable 5.0 the total bill fell 5%, but nearly all of that came from one task. The average task cost 1% more. With DeepSeek V4 Pro the average task cost 17% more.

JetBrains called RTK's filters "real and often elegant" and found no drop in task quality from either tool.

#Counting tokens instead of dollars

Most of the big numbers come from the people who build these tools. They count tokens per command or per payload (X tokens instead of Y) and leave the reader to assume the bill falls by the same share.

RTK's own counter shows how far a token count and a bill can drift apart. In JetBrains' low-effort run, rtk gain reported 96.2 million tokens saved while the bill for the same tasks went up. It counts a command's full raw output as what the agent would otherwise have read. One cat of a 1.2 MB CSV logged 320,000 tokens saved, though Claude Code cuts off a tool result long before it gets that big. Quesma caught the counter crediting 120.5 million tokens, twice, to a head -1 that would never have returned the whole file. To its credit, RTK's documentation now says a command showing 90% fewer output bytes doesn't make a session 90% cheaper. Headroom reports its savings per payload, from 21% on a code search result to 57% on an incident-debugging prompt. Its limitations page lists multi-turn agent conversations as not independently measured.

The bigger gap is the prompt cache. An agent resends the whole conversation on every turn. The provider reads most of it from its prompt cache at a fraction of the input price. On Claude Opus 5.5 fresh input costs $4 per million tokens, writing new input into the cache costs $5, reading from the cache costs 20 cents and output costs $20. Once a tool result is in the cache, every later turn reads it back for a twentieth of the input price and a hundredth of the output price.

To see where the money goes, I modeled one coding session on Opus 5.5 at Anthropic's prices as of November 9, 2026. The session runs 46 turns. The first writes 31,200 tokens of system prompt, tool definitions, project instructions and task into the cache. On each turn the model generates 1,150 output tokens (reasoning included) and gets back a 920-token tool result. Both join the conversation. Every later turn reads the conversation so far from the cache and writes the newest 2,070 tokens into it. Tool results end up as 27% of the session's input tokens, more than the 11% Quesma measured for Claude Code on Fable, so for a Claude Code session these numbers favor compression. The whole session costs $2.37. I rounded the results to keep them readable.

Cache reads are 95% of the session's tokens and 29% of its cost. Output costs more than all the cache reads, yet RTK's filters and Headroom's default compression leave it untouched.

#The best case against one extra turn

Now give compression every advantage. Say it shrinks every tool result in the session by 60% and the agent never misses what was cut. Each compressed result saves half a cent, counting the cache write and every later turn that reads it back. Across the 45 tool results the agent reads, that comes to 23 cents, a 10% saving.

The RTK versions JetBrains and Quesma tested saw only part of the tool output, the results of shell commands. Claude Code reads and searches files with its own tools, which skipped RTK's hook. JetBrains found the hook saw about a fifth of the tool output. Compress a fifth of the tool output by 60% and the session saves 5 cents, or 2%.

One extra turn costs this session 6 cents, more than that entire saving. A compressor saves tenths of a cent per tool result and bets that nothing it drops will cost a turn, though plenty of things can. The model misreads a compressed result, or the compressor has a bug. A line it dropped as noise turns out to matter and the agent goes back for it, through the retrieval tool or by running the command again.

At low reasoning effort, JetBrains counted 13.8% more turns with RTK. At high effort the count didn't change. Quesma counted 18% more with DeepSeek, where the average turn carried 7% less input but the extra turns more than made up for it. Now drop the assumption that the agent never misses anything. Give the best-case compressor six more turns (close to the 13.8% JetBrains measured) and the session costs $2.44, 3% more than not compressing at all.

#A second program in the loop

The extra turns are only the part you can count. A pointer asks the model to judge something it can't see. Shown [1000 items compressed to 20. Retrieve more: hash=abc123], the model has to decide whether any of the missing 980 matter. If it decides they don't and it's wrong, it carries on with partial data. If it fetches them, it pays for another model request and then carries both the summary and the full output for the rest of the session. RTK ships a counter for this, rtk gain --recalls, which its docs describe as the signal that a filter's cap is too aggressive for your workflow.

Pointers can also outlive what they point to. Headroom's proxy keeps originals for 30 minutes unless you raise the limit. RTK keeps its last 200. A long session can end up holding a marker that no longer resolves.

The layer also makes mistakes harder to trace. When compression runs in a proxy, the agent's transcript keeps the original output while the model saw something else.

Bugs in the compressor land inside the agent loop. In one of Quesma's runs, RTK rewrote a find command that used a flag rtk find didn't support. The rewritten command failed and so did every retry, because each retry was rewritten the same way. The agent hit 339 errors in a row before it timed out and still passed the task, at about nine times the cost of the matching run without RTK. RTK has since fixed that bug.

On our own agents, we took RTK out. In-line compression through Headroom gave us bad results. A tool that trimmed output as the agent ran each command saved a low single-digit percentage over several months, which wasn't worth keeping another tool working on every machine our agents run on.

#Good agents already trim their own output

A well-steered agent already does much of what compression promises. Frontier models often trim their own tool output. Quesma found that about half of Claude Code's shell commands already limited their output with head, tail or wc. Terminal output was only 7% of Fable's context. Caveman's README shows a go test -v run producing 10,561 tokens raw and 397 the way agents already run it, piped through tail -20. On the output side, a plain instruction to answer concisely gets most of what Caveman gets.

The rest comes from steering, which costs a few lines of instructions and hides nothing from the model. Project instructions can tell the agent how to run the test suite quietly and to read the first failure before the whole log. Tools can return what the agent needs in the first place.

#Where the money is

The prompt cache comes first. On Opus 5.5 a cache read costs 5% of the input price. On Fable 5.1 it's 2.5%. In the session above, a cache miss at turn 30 means writing all 91,000 tokens of the conversation back into the cache. That one turn costs 48 cents instead of 5, an extra 43 cents that's nearly twice what perfect compression saves over the whole session. The cache misses when a session sits idle past its five-minute lifetime or when anything changes the earlier part of the conversation. A compressing proxy has to leave that earlier part unchanged byte for byte. Headroom's docs list frozen cache prefixes among the things it won't change. For agents that pause while someone reviews their work, Anthropic's one-hour cache costs twice the input price to write and stays warm through the wait.

Sonnet 5.5 costs half of what Opus 5.5 does at every rate, so the same session on Sonnet costs $1.19. Sonnet isn't the right model for every task, but on the tasks it handles, picking it cuts this session from $2.37 to $1.19. We priced this out over a month of agent traffic in an earlier post.

Reasoning effort is the third big setting. Reasoning is billed as output, which is 45% of this session's bill. Quesma found that output, reasoning included, was 56% of DeepSeek's cost in its RTK runs. Caveman's own biggest number comes from model choice and effort. Its Auto mode picks the model and the effort level for each request. In a run that's half finished, it cost 54% less than Opus 5.5 at extra-high effort on every request. The README says most of that saving came from lowering effort on easy requests.

Work that can wait, such as summarizing a backlog of tickets overnight, can go through the Batch API at half price on input and output.

#Before you install one

If someone pitches you a compressor anyway, ask for a paired benchmark on your own sessions, priced on the invoice. Run the same tasks with and without it and compare what each finished task costs, which a token counter on a dashboard can't tell you. Caveman now ships a caveman trial command for that kind of A/B run on your own work. I haven't seen a compressor come close to its advertised number when someone ran that test.