A developer running independent benchmarks on AI coding harnesses has published results that show wide disparities in how different tools use tokens to solve the same problems. The test, conducted on GLM 5.3 using ten SWE-bench verified tasks selected for difficulty, puts 3code at the top for both task completion and token efficiency, while Claude Code consumed far more resources without a corresponding advantage in results.
The benchmark setup
The test used ten tasks from the SWE-bench verified set, chosen representatively to cover a range of difficulty. Each harness ran against the same model, GLM 5.3, on the same tasks. The results track tasks solved, total tokens consumed, cache hit rate, and token usage relative to the most efficient harness. The author notes that the benchmark is not perfect, that unpublished model improvements mean results are not deterministic, and that it serves as a rough evaluation rather than a definitive ranking.
The harnesses tested include Claude Code, OpenCode, pi, zcode, Hermes, and 3code. Each represents a different approach to structuring the interaction between a language model and a coding agent, with different strategies for system prompts, tool calling, retry logic, and token management.
3code and the token efficiency gap
3code solved nine of the ten tasks using five million tokens total. The runner-up in task completion, OpenCode, also solved nine of ten but used roughly twice the tokens. Pi solved six tasks in the allotted time with lower total token usage than OpenCode but could not match 3code's completion rate. Claude Code solved eight tasks but consumed significantly more tokens than any other harness in the test.
The author attributes 3code's performance to model family-specific system prompts and fine-tuning that target GLM 5.3 specifically. The current version of 3code also benefits from fine-tuning work originally done for zcode, but without zcode's aggressive task completion policy, which causes it to spend excessive tokens on tasks it ultimately fails to solve.
Zcode's results illustrate this pattern. On tasks where it succeeded, zcode was hyper-efficient, using minimal tokens per solution. On tasks where it failed, it continued generating output until timing out, burning through tokens without producing results. That behavior inflated its total token count despite its efficiency on successful cases.
Cache rates and their misleading appeal
OpenCode's cache hit rate of 99.2% stands out as the highest in the test. But the author argues that cache rate alone is a misleading metric. OpenCode uses twice the total tokens of 3code, and even with a high cache rate, those cached tokens still cost money. The argument is that reducing overall token usage matters more than optimizing cache utilization, because every token that gets cached was still generated and still billed.
For developers choosing between harnesses, the distinction matters. A harness with a lower cache rate that uses fewer total tokens may cost less in practice than one with a perfect cache rate that generates twice the volume. The benchmark suggests that 3code's approach, focused on minimizing total token consumption, produces better cost efficiency than OpenCode's approach, which generates more tokens but caches a higher percentage of them.
Pi's performance is notable for a different reason. It achieved six solved tasks without any model-specific tuning, relying on a generic approach that works across different models. The author suggests that model-specific optimization, as 3code does for GLM 5.3, produces better results than the one-size-fits-all strategy, though at the cost of portability.
What this means for harness selection
The author is clear that these results should not drive anyone to switch harnesses based on numbers alone. If you have an unlimited budget and prefer Claude Code's interface, use Claude Code. If you want maximum token savings and are comfortable with a terse command-line agent, 3code is worth evaluating. The benchmark is most useful as a data point for developers who are cost-sensitive and want to understand where their tokens are going.
The broader takeaway is that harness architecture matters more than many developers realize. The same model, on the same tasks, produces dramatically different results depending on how the harness structures the interaction. System prompts, retry policies, tool-calling strategies, and token management all affect both completion rates and cost. The differences are not subtle. They span an order of magnitude in token usage for the same set of problems.
The author plans to continue running benchmarks and is working on funding for more extensive tests across different models and task types. For now, the GLM 5.3 results on ten hard tasks suggest that the field of coding harnesses is far from settled, and that smaller, more targeted tools can outperform well-funded alternatives when they are designed for a specific model and a specific workflow.