How CliffCompaction Makes Long-Horizon Coding Agents 50% Cheaper Without Sacrificing Performance
Coding agents solve increasingly complex tasks by interacting with tools and environments over contexts spanning millions of tokens. But there is a hard ceiling: every LLM has a finite context window. When a trajectory exceeds that window, the agent must compact its context to continue working. The problem is that compaction is expensive, error-prone, and often degrades performance. Summarization compounds loss across summary-of-summary chains. Sliding windows invalidate the prefix cache at every step, driving up inference costs. LLM-based compression introduces hallucinated content that silently corrupts the agent's understanding of the codebase.
CliffCompaction, introduced by Trang Nguyen, Eulrang Cho, Bingqing Chen, and Tim Dettmers at Carnegie Mellon University and the Bosch Center for AI, proposes a radically different philosophy: never rewrite, only truncate. The technique reduces compaction cost by up to 50% while maintaining or improving agent performance, makes test-time scaling economically viable, and sets a new state-of-the-art result for continual learning on KernelBench. At its core is a deceptively simple insight: the reason compaction degrades performance is not that it removes information, but that it rewrites it.
The Hidden Cost of Context: Where Agent Tokens Actually Go
Before understanding CliffCompaction's design, it helps to understand where agent context tokens actually accumulate. When a coding agent works on a complex problem, the conversation is dominated by tool results and tool calls. On Terminal-Bench 2.0 with GLM 5.1, tool results account for 56% of all context tokens, and tool calls account for another 28%. Together, 84% of the tokens spent on a trajectory come from tool interactions. The model's own reasoning thoughts are a surprisingly small fraction.
But the bigger issue is not the volume of tokens, it is the cost of them. On the backend, cached input tokens correspond to KV-cache reads. While each cache read is individually cheap, for long-context sessions they dominate the total inference cost because every request bears the cost of all past tokens. API costs for uncached input tokens that require re-prefilling are roughly 5 to 6 times as expensive as cached tokens. Any modification to the context invalidates the KV-cache and forces a re-prefill from scratch.
This creates a tension: the more aggressively you compact, the more re-prefills you trigger, and the more you negate the savings. CliffCompaction resolves this tension by making compaction events rare and dramatic rather than frequent and gentle. It lets the context grow naturally until it hits a preset threshold, then performs a single decisive compaction.
The Three Design Principles
CliffCompaction makes three deliberate design choices that distinguish it from every existing compaction strategy.
First, it compacts when the context crosses a token threshold, not proactively. Rather than actively managing context throughout a trajectory, CliffCompaction leaves it unchanged and lets it accumulate until it reaches a predefined budget, at which point a compaction event fires. Between compactions, the existing context is never modified, so the KV-cache remains valid across each entire growth segment. The cache is invalidated only at compaction points themselves.
Second, it retains verbatim excerpts rather than summaries, and drops only the token-intensive portions. When compaction triggers, tool results exceeding 500 characters are dropped entirely, while shorter results like grep matches, exit codes, and concise script output are kept verbatim. Tool calls are reduced to compact signatures containing the tool name, target file, and essential arguments, but the call itself is preserved verbatim. Agent thoughts are truncated to 300 characters. The system prompt and the first user message are kept in full. The most recent turns remain unchanged.
This is the crucial departure from prior work. If most information is summarized correctly, the model assumes it has all the information and does not need to revisit past state such as documentation or code files. This leads to subtle but continuous context drift where the summary does not preserve the right information. CliffCompaction calls this problem low compaction precision and solves it by never rephrasing or rewriting anything.
Third, and perhaps most controversially, each compaction event discards all previous compacted history and constructs a new compacted block only from the current live session. The algorithm never compacts a compaction. At each compaction trigger, the previous compacted history is discarded entirely rather than nested into the new compacted block. This creates a sharp cliff-shaped drop in context length after every compaction event, and each cliff is equally faithful to the turns it condenses.
The Trade-Off: Precision Over Recall
CliffCompaction makes a deliberate trade-off between two competing properties. Compaction precision measures how much information is preserved verbatim, while compaction recall measures how much information is retrievable over the course of many compactions.
Summaries have high recall, since they can retrieve information from many past compactions. But they have low precision, because each successive summary distorts the original content. CliffCompaction has the opposite profile: high precision but low recall. After two compactions, all direct information from earlier turns is gone. However, the authors argue that full-history recall is not needed for coding agent performance, and they support this claim empirically.
There is a subtle mechanism that softens the loss in recall: residual propagation. Every turn of the live session is generated with the previous compacted history in view. The agent's actions and reasoning carry an implicit influence of the discarded context. This chain extends back to the start of the trajectory, so no information is ever carried forward directly, but residual knowledge leaks from each session into the next through the agent's own behavior. This allows CliffCompaction to remain effective at long-context sessions over millions of tokens even with a compaction window of only 128K tokens.
Results: Matching or Beating Full-Context Performance
CliffCompaction is evaluated on SWE-bench Verified, Terminal-Bench 2.0, and Terminal-Bench 2.1 across multiple models including Kimi K2.5, Kimi K2.6, GLM 5, GLM 5.1, GLM 5 Turbo, and GLM 5.3 Flash, using both mini-swe-agent and OpenHands scaffolds as well as the Terminus-2 scaffold and Claude Code.
On Terminal-Bench 2.0, CliffCompaction matches or exceeds full-context performance at moderate thresholds. For Kimi K2.6, both the 32K and 16K settings achieve 61.42% resolution rate, exceeding the full-context baseline of 59.16% by 2.26 percentage points while cutting cost by 50%. GLM 5.1 shows a similar benefit, improving from 49.83% at full context to 54.33% at 16K. Removing stale context sometimes improves agent performance rather than simply reducing memory pressure.
On SWE-bench Verified, CliffCompaction preserves most full-context performance at moderate thresholds. For Kimi K2.6, reducing the threshold to 32K changes the resolution rate only from 73.87% to 73.27%, and the 16K setting remains within 2 percentage points of the full-context baseline. GLM 5 Turbo actually improves from 69.80% to 70.53% at the 32K threshold.
On Terminal-Bench 2.1 with Claude Code, at a matched mean peak context of approximately 45K, CliffCompaction reaches 76.69%, exceeding both Claude Code's own auto-compaction at 70.97% and its default 200K configuration at 73.03%, with markedly lower variance. Notably, CliffCompaction has no knowledge of Claude Code's tool schema, reducing every tool call to a generic signature, yet still outperforms a scaffold-native summarizer.
Making Test-Time Scaling Economically Viable
Test-time scaling, where multiple agent rollouts are run and the best solution is selected, has remained largely academic due to its exorbitant cost. Without compaction, three rollouts of Kimi K2.6 on all Terminal-Bench instances buy only 4.8 points of performance gain over a single run at three times its cost of $91.65, which exceeds the cost of a single run of a proprietary model that scores 5.5 points higher.
CliffCompaction inverts this trade-off. Three Kimi K2.6 rollouts with CliffCompaction at a 16K context limit cost $58.01 in total, less than a single run of GPT 5.3 Codex at $64.63, while reaching 69.7% resolution rate, matching Anthropic's Opus 4.7 at 69.4% and exceeding GPT 5.3 Codex at 64.7% and Opus 4.6 at 62.9%. This represents a gain of 10.5 percentage points at only 1.9 times the cost of a single uncompacted Kimi run, compared to 4.8 points at 3.0 times cost without compaction.
The gains persist under more conservative budgets. With only two rollouts, CliffCompaction reaches 65.9% at $38.67, cheaper than a single run of either GPT 5.3 Codex or Opus 4.6, while already exceeding the uncompacted three-rollout baseline at 42% of its cost. GLM 5.1 shows the same pattern, raising practical resolution from 55.1% to 60.7% while reducing scaled-inference cost by 40%.
Continual Learning Over Sessions Exceeding a Million Tokens
Beyond standard agent benchmarks, CliffCompaction excels at continual learning, where a single trajectory can exceed one million tokens. On KernelBench Level 3, where agents iteratively optimize GPU kernels until reaching a speedup target, CliffCompaction achieves CUDA kernel speedups of 2.23x after 200 steps and reaches 3.58x after 400 steps.
These results are remarkable because they surpass specialized search algorithms and trained agents despite CliffCompaction being a general-purpose compaction technique. The specialized baselines include DR.Kernel, Iterative Refinement, OpenEvolve+Memory, AdaExplore, and CUDA-Agent, all of which are purpose-built for kernel optimization. CliffCompaction outperforms them by 25% while requiring no external memory or task-specific design.
The bounded context is also cheaper to scale. Because each compaction event dramatically reduces context size, the cost of scaling to longer sessions grows sublinearly. CliffCompaction reaches the best results on KernelBench while requiring no task-specific adaptation, making it applicable to any long-horizon coding task.
Deployment: Scaffold-Agnostic and Open-Source
CliffCompaction is implemented as a scaffold-agnostic API proxy usable with any harness, including Claude Code, Codex, and others. It requires no changes to the underlying model, no training, and no scaffolding modifications. The implementation is open-sourced at https://github.com/nguyenvuthientrang/cliffcompaction.
For practitioners, the practical implications are significant. If you are running a coding agent on complex tasks that require long contexts, CliffCompaction can cut your inference costs in half while potentially improving success rates. The technique works across models, scaffolds, and task types, requiring only a single configuration parameter: the compaction threshold.
Limitations and Trade-Offs
CliffCompaction makes a deliberate sacrifice of full-history recall, which could matter for tasks that require precise recall of events from many compactions ago. The 8K setting shows larger performance drops across benchmarks, suggesting it is best viewed as a stress test rather than the default operating point. The authors note that the optimal threshold depends on the task and model, and that tasks requiring extensive cross-session memory may need further investigation.
There is also a subtlety in the residual propagation mechanism: while it preserves useful behavioral patterns across compactions, it does not guarantee that every piece of discarded information is recoverable. For coding agents working on problems where exact code snippets from hours ago are needed, the loss of full-history recall could occasionally be costly.
The Bottom Line
CliffCompaction introduces a fundamentally different approach to context management for long-horizon coding agents. Rather than trying to summarize, compress, or remember everything, it embraces a disciplined policy of only truncating, never rewriting, and always discarding past compactions. This simple philosophy, grounded in the empirical observation that 84% of agent context consists of tool interactions that are recoverable on demand, enables dramatic cost reductions while maintaining or improving performance across every benchmark tested. The results on test-time scaling are perhaps the most surprising: by making individual rollouts cheap enough, CliffCompaction turns a technique that was previously impractical into one that can beat frontier proprietary models at a fraction of the cost. For anyone deploying coding agents on complex, long-running tasks, it establishes a new baseline for what is possible with rule-based context management.