Tiiny AI wants to put a 120-billion-parameter language model in your pocket. The company, founded in 2024 by a team from MIT, Stanford, HKUST, SJTU, Intel, and Meta, debuted the Tiiny AI Pocket Lab at CES 2026 and secured a Guinness World Record for the smallest mini PC capable of running a 100B-parameter LLM locally. The device weighs 300 grams, fits in a palm, and runs open-source models completely offline with no cloud connection, no subscription, and no data upload.

The Pocket Lab is not a developer board or a reference design. It is a finished product with a custom heterogeneous module combining a system-on-chip and a dedicated neural processing unit, delivering roughly 190 TOPS of inference performance. It ships with 80GB of LPDDR5X memory, a 1TB SSD, and a 35-watt thermal design power. The company claims real-world decoding speeds of 20 or more tokens per second on models up to 120B parameters, which puts it within reach of usable interactive inference on a device that runs on battery-class power consumption.

How it runs large models on small hardware

The engineering challenge is obvious: 120B parameters at FP16 would require 240GB of memory, far beyond what any portable device can hold. Tiiny's approach relies on two sparse inference techniques developed at SJTU's IPADS lab and published as open-source research: TurboSparse and PowerInfer.

TurboSparse applies aggressive activation sparsity to transformer models, running only a fraction of the feed-forward network's neurons for each token. PowerInfer extends this idea with a heterogeneous computing strategy, routing activations between the CPU and a neural processing unit based on which hardware handles that neuron's computation more efficiently. Together, these techniques reduce the memory footprint and computation required to run a large model while maintaining output quality within acceptable bounds.

The result is a device that does not try to be a GPU cluster. It does not compete with NVIDIA hardware on raw throughput. It competes on a different axis: portability, privacy, and zero-dependency operation. A developer or researcher who needs to run inference on sensitive data, in a location without reliable internet, or simply without sending prompts to a third-party API, can do so on a device that fits in a jacket pocket.

Software ecosystem and developer tooling

The hardware ships with a software stack. The Tiiny client, available for Windows and macOS, provides one-click deployment of open-source language models and AI agents. Users without coding experience can download a model, load it, and start interacting through a chat interface. The company positions this as lowering the barrier to local AI adoption beyond the developer community.

For developers, the TiinyOS SDK is an open toolkit for building, deploying, and customizing AI agents locally. The SDK supports the creation of localized personal agents and AIoT hardware demonstrations. Combined with the device's CDP and network capabilities, developers can build agents that interact with local services, process documents, or automate workflows without any data leaving the machine.

The company's GitHub repository hosts the PowerInfer codebase, giving developers visibility into the sparse inference engine. This is not a black-box accelerator. The architecture is documented, the techniques are published, and the implementation is auditable. For teams evaluating local AI deployment, that transparency matters.

Privacy as a design constraint

The core proposition is data sovereignty. Every prompt, every file, every interaction stays on the device. There is no telemetry, no cloud sync, no usage reporting. The company frames this with a line from founder and CEO Miles Mi: "Your AI should be as private as your diary."

This is not a marketing abstraction. The device has no reason to phone home. It runs offline by design. The 1TB SSD stores model weights, conversation history, and working data locally. For organizations handling regulated data, financial records, legal documents, or medical information, this eliminates the compliance overhead of sending content to cloud AI providers. The device does what a self-hosted server does, but in a form factor that travels.

The privacy model also applies to the model weights themselves. Open-source models loaded onto the device are not transmitted anywhere. The device does not report which models are running or how they are being used. For developers building commercial products on top of open-source LLMs, this removes the question of whether the inference platform is observing their work.

Pricing and availability

The Pocket Lab launched on Kickstarter with a super early-bird price of $1,399, with standard early-bird pricing at $1,299. The retail MSRP is $1,999. The company notes that the 80GB of LPDDR5X memory alone has a standalone market value exceeding $900 at current pricing, positioning the device as aggressive on price-per-gigabyte of fast memory.

The Kickstarter campaign ran in early 2026, with the company planning direct sales through its website starting in September 2026. Tiiny AI will also demonstrate the Pocket Lab at AI Infra Summit 2026, with live demonstrations of the device running language models, AI agents, and document intelligence workloads.

For developers evaluating local inference options, the competitive landscape now includes cloud APIs, GPU-equipped desktops, Apple Silicon with unified memory, and portable devices like the Pocket Lab. Each option trades off cost, performance, portability, and privacy differently. Tiiny's bet is that the combination of pocket-sized form factor, zero cloud dependency, and open-source model support fills a gap that the other options do not reach. Whether that gap matters to enough developers will determine whether the Pocket Lab is a curiosity or a category.