A $22.25 million seed round does not normally move the needle in AI. But PrismML, a startup founded by Caltech researchers and led by compression expert Babak Hassibi, is betting that the next generation of AI models will not need to be large at all. On Thursday, the company released Bonsai 2 27B, a compressed version of Alibaba's Qwen3.8 27B that runs from a 25GB model down to 5.9GB, small enough to fit on a consumer PC or a high-end smartphone.
Why Model Size Is the Real Bottleneck
The AI industry has spent years scaling models upward. Bigger parameter counts, longer training runs, more data. The assumption was that intelligence scales with size, and for a long time it did. But size carries costs that go beyond compute. Large models require cloud infrastructure to serve. They add latency on every API call. They send user data to remote servers, raising privacy concerns. And they lock advanced capabilities behind services that charge per token.
The alternative, running models locally on user devices, has been limited by memory. A typical 27-billion-parameter model in 16-bit precision consumes around 54GB of memory. That does not fit on any phone, and it barely fits on most laptops. The models that do fit on devices tend to be too small to be useful for complex reasoning tasks.
Ternary Weights: Three Values Instead of Sixteen Bits
PrismML's compression technique, called ternary weights, attacks this problem at the weight level. In a standard neural network, each weight stores a 16-bit floating point number. PrismML replaces those continuous values with three discrete options: +1, -1, or 0. Each weight shrinks from 16 bits to roughly 1.58 bits, a reduction of more than 10x.
The technique is not entirely new in the research literature, but making it work at production quality is the hard part. Aggressive quantization typically destroys model performance, especially for reasoning tasks where subtle weight differences matter. PrismML's first release, Bonsai 1 in March 2026, matched 95% of Qwen's aggregate benchmark scores after compression. Bonsai 2, released Thursday, closes that gap to 98%.
The practical difference between 95% and 98% benchmark parity is larger than it sounds. Benchmarks are imperfect proxies for real-world use, and a 5% gap on aggregate scores can translate into noticeable failures on specific tasks. A 2% gap is much harder to distinguish from the original model in practice, especially given that the software harness surrounding a model affects accuracy as much as the weights themselves.
What 5.9GB Actually Enables
A 27B parameter model compressed to 5.9GB changes where AI can run. It fits in the memory of most modern laptops without requiring external GPU hardware. It fits on phones with 8GB or more of RAM. That means reasoning-capable models can run entirely on-device, with no cloud round-trip, no API charges, and no data leaving the user's hardware.
Ion Stoica, PrismML's adviser and co-founder of Databricks, frames this as a shift in who controls intelligence. Models that run on-device are free to use after the initial download, private by default since no data is transmitted, and available without an internet connection. For developers building AI-powered applications, this opens up deployment scenarios that cloud-dependent models cannot reach.
The download numbers suggest demand already exists. PrismML says the original Bonsai has been downloaded over 11 million times, with its smaller model variants adding another 2.6 million downloads. Those numbers indicate significant interest in running capable models locally, even before the technology reaches full parity with uncompressed originals.
A Caltech Lineage and What Comes Next
PrismML was founded by a group of Caltech researchers and is led by Babak Hassibi, a Caltech professor specializing in compression technologies. The company's academic roots show in its approach: the ternary weight technique is rooted in theoretical work on quantization, and the team publishes its models openly on Hugging Face for community testing and validation.
The company is backed by Khosla Ventures, Cerberus Capital, and Caltech. It faces competition from Multiverse Computing, a well-funded startup also working on LLM compression, founded by a prominent physicist from Spain's Donostia International Physics Center. Multiverse Computing has raised significantly more capital, but PrismML is betting that its compression quality gives it an edge.
The next target is larger models. Hassibi told TechCrunch that PrismML plans to release compressed models in the several-hundred-billion-parameter range within the next few months. His reasoning is counterintuitive but sound: larger models have more redundancy, meaning there is more room to compress without losing intelligence. As models grow, the compression problem actually gets easier, not harder.
There are also unconfirmed rumors of talks with Apple. Hassibi declined to comment on that, but the logic is clear. A compression technology that can shrink a 27B model to 5.9GB with minimal quality loss is exactly what a device manufacturer would need to ship on-device AI without requiring users to buy new hardware.
The Practical Question for Developers
For teams building AI applications, compressed local models change the economics and architecture of deployment. Instead of routing every inference call through a cloud API, applications can run the model on the user's device. Latency drops to near zero. Costs drop to zero after the initial download. Privacy concerns disappear because no user data is transmitted.
The tradeoff is that compressed models are not identical to their uncompressed originals. A 2% benchmark degradation means some edge cases will produce different results. For most applications, that difference is acceptable. For safety-critical or high-precision tasks, developers will need to evaluate whether the compressed version meets their requirements.
PrismML's trajectory suggests that the gap will continue to narrow. Each release has improved over the last, and the company is now targeting larger models where compression headroom is greater. If the trend holds, the question is not whether local models will match cloud models, but when.