Ternary large language models pack every weight into one of three values: -1, 0, or +1. The information-theoretic floor for storing those choices is about 1.585 bits per weight, and the standard five-trit packing scheme used in practice lands at 1.625 bits. A paper posted to arxiv on 14 September argues that the real number should be lower, because zeros dominate the weight distribution in ways the current format ignores.
The researchers measured the symbol distribution across 29 ternary LLM models and found that zeros account for up to 51.5% of all weights. That skew means the three symbols are not equiprobable, and a storage format that treats them as equal wastes space.
A layout that adapts to the model's own distribution
The paper introduces BITCOS, a storage format that pairs a dense presence bitmap with a compacted sign vector. The bitmap marks which positions hold non-zero weights, and the sign vector stores only the signs of those non-zero values. The total cost comes to 2 minus the zero density per weight element. When zeros make up half the weights, that works out to about 1.5 bits per weight, below the conventional five-trit packing.
BITCOS outperforms five-trit packing on 26 of the 29 tested models. On the sparsest model in the set, it reaches 1.485 bits per weight.
Unpacking on real hardware
The format is designed for efficient unpacking on modern processors and GPUs. The paper presents optimized unpacking sequences for AVX-512, AVX2, and Intel Xe2 GPUs. When measured against production ternary matrix-vector multiplication kernels, the realized performance gain at the zero densities found in real models reaches up to 1.28x.
End-to-end inference results across five platforms, including client and server CPUs and integrated and discrete Xe2 GPUs, show decode throughput improvements of up to 1.18x on CPUs and 1.27x on GPUs.
What the savings mean for deployment
Ternary models already appeal because they reduce memory footprint and arithmetic complexity compared to full-precision weights. Shaving another tenth of a bit per weight might sound small, but at the scale of modern language models, with billions of parameters, it compounds. A model that fits in less memory can run on cheaper hardware or serve more concurrent users on the same machine.
The practical constraint is that the gain depends on how sparse a model's weights actually are. Models with fewer zeros see less benefit. The paper's 29-model survey suggests that most real-world ternary models have enough zeros to make BITCOS worthwhile, but the format is not a universal win.
Evangelos Georganas, listed as the corresponding author, posted the paper with optimized kernels and platform-specific results. The approach does not require retraining a model, only reformatting how its ternary weights are stored in memory before inference.