A Free Tool That Tells You Exactly Where Your AI Agent's Token Budget Went
If your AI agent costs more than expected and nothing visibly errored, the most likely culprit is a broken prompt cache. The tokens your agent already paid for get re-billed at write prices, and nobody tells you which turn caused it. Replay Doctor is a new tool that reads the transcripts sitting on your disk and names the exact turn, the cause, and how many tokens got re-billed.
Prompt caches let language model providers reuse previously computed context instead of reprocessing it. When the cache holds, you pay less. When it breaks, you pay full price for tokens you already sent. The problem is that nothing in the output tells you it happened. Costs go up, the agent keeps working, and you have no way to trace the increase to a specific moment in the session.
Replay Doctor parses existing transcript files and identifies cache invalidation events. It reports the turn where the break occurred, what changed to cause it, and the token cost difference. The most common cause is cache expiry, but the tool can name other causes as well. One user reported that 98.8 percent of their re-billed tokens traced back to a single cause, which they reduced to 4.2 percent the same day after using the tool to identify and fix the problem.
Four Commands, Generated From the Binary
The tool ships as a single Go binary. Four commands cover the main use cases. One identifies cache breaks and names the turn. Another ranks what is filling your context and shows how much the ranking overstates after a compaction event. A third prices a model switch and names the turn where the cheaper model becomes cheaper. A fourth produces a compact contribution file for the public pool.
That pool is how the project builds aggregate data. Each contributing machine writes a file under 600 bytes containing sixteen counts and ratios plus three strings identifying the build that produced them. No file paths, no account identifiers, no repository names. The files are posted to a public endpoint and summed on a roster page where every row is a downloadable, re-hashable file. The current pool shows data from 26 installations across 1,363 sessions as of September 7.
Contributing requires explicit opt-in. A consent file must be created before the contribution command will run, and the contribution command itself is a separate step from diagnosis. The tool prints what it measured and refuses what it did not measure.
Free Diagnosis, Expensive Forensics
Diagnosis is free and stays free. The paid offering is a forensic engagement: a week-long analysis where every turn in your agent sessions gets annotated with cache behavior, break causes, and layout changes worth making. As of September 13, 2026, the engagement is priced at $22,000, quoted between $18,000 and $25,000 depending on scope, with none sold yet. The author notes that sentence changes the day one is.
The project also includes a concrete recommendation for avoiding the problem in the first place: setting promptCacheTtl to one hour in Claude Code settings and keeping the tool list stable mid-session. If that describes your setup, the tool is not for you. For everyone else, the author suggests running it once and reading the diff to understand whether the wall you are hitting is output, compaction, or silent re-reads.
The tool installs in one line and requires Go 1.22 or later. The repository is at github.com/RedRobotKK/Replay, and the diagnostic output is generated from the binary itself, so what you see is what the code actually does.