Running AI models in the browser is no longer theoretical. WebGPU, WebAssembly, and ONNX Runtime have made it possible to execute inference entirely client-side, with no server round-trip. The problem is that hardware capabilities vary wildly across devices. A developer cannot ship a single model and expect it to work on every laptop, phone, and tablet. Browser LLM Fit, a new open-source project, addresses this by auto-detecting client hardware and matching it against available in-browser AI models.

The Hardware Fragmentation Problem

Browser-based AI faces a constraint that server-side deployment avoids: you do not know what hardware your users have. A desktop with a dedicated GPU can run a 7B parameter model at acceptable speed. A laptop with integrated graphics might manage a 1B model. A phone with 4GB of RAM might not run anything larger than a quantized 300M model without exhausting memory.

Developers currently handle this by shipping multiple model variants and guessing which one to load. Or they ship the smallest model and accept that users with powerful hardware get a degraded experience. Or they ship a large model and accept that users with weak hardware get timeouts and crashes. None of these approaches are satisfactory.

Browser LLM Fit automates the selection. The library probes the client's hardware: available memory, GPU capabilities through WebGPU, CPU cores, and WASM support. It then matches those capabilities against a catalog of available models, each tagged with minimum hardware requirements. The user gets the best model their device can run, without the developer making assumptions.

Three Runtime Backends

The project supports three inference backends, each with different trade-offs. WebGPU provides GPU acceleration for devices that support it. This is the fastest path for models that can fit in GPU memory. WASM provides CPU-based inference that works on any modern browser, including mobile Safari. ONNX Runtime Web brings Microsoft's optimized inference engine, which supports both WebGPU and WASM execution providers.

The hardware detection layer abstracts these differences. A developer defines their model catalog once, specifying which backends each model supports and what hardware each backend requires. The library handles the runtime selection. If WebGPU is available and the device has enough GPU memory, it uses WebGPU. If not, it falls back to WASM. If ONNX Runtime is available and provides better performance for the specific model, it uses that instead.

This matters because the performance characteristics are not uniform. A model that runs fast on WebGPU might run slowly on WASM for the same hardware. A model optimized for ONNX Runtime might underperform on raw WASM. The library benchmarks at load time and picks the combination that gives the best throughput for the specific device.

How the Detection Works

The hardware probe runs at library initialization. It queries available system memory through the browser's device memory API (where available) and estimates available memory from the performance API. It checks WebGPU support by attempting to create a GPU adapter and querying its capabilities: maximum buffer size, compute shader support, and memory limits. It counts CPU cores through navigator.hardwareConcurrency. It tests WASM support by attempting to compile a small module.

The results produce a capability profile: approximate available memory, GPU type and memory, CPU core count, and supported runtimes. This profile is compared against the requirements of each model in the catalog. Models are scored based on how well they fit the hardware, with penalties for models that exceed memory limits or require unavailable runtimes.

The selection is not just about whether a model can run. It is about whether it can run well. A model that technically fits in available memory but leaves no room for the browser's own processes will cause swapping and poor performance. The library applies a safety margin, typically 20-30% below the theoretical maximum, to ensure the browser remains responsive.

For WebGPU specifically, the detection goes further. It checks whether the GPU supports the required compute shader features, whether the maximum workgroup size is sufficient for the model's operations, and whether the GPU memory is large enough for the model weights plus intermediate activations. A GPU that supports WebGPU but lacks sufficient memory is not a valid target for GPU-accelerated inference.

Practical Deployment Patterns

The typical usage follows a predictable pattern. A developer hosts model files at known URLs, tagged with metadata about their size, backend requirements, and hardware minimums. The library loads the catalog, runs hardware detection, and selects the best model. The model loads from the URL and inference begins.

For applications that need to work offline, the library can cache selected models in the browser's IndexedDB or Cache API. Subsequent visits skip the network fetch entirely. The hardware detection still runs each time, so if the user switches devices, the library selects the appropriate model for the new hardware.

Error handling is built into the flow. If the selected model fails to load, the library falls back to the next best match. If no model in the catalog fits the hardware, it reports the hardware profile to the developer, allowing them to add a smaller model variant. The fallback chain is configurable: a developer can specify that certain models should never be used as fallbacks, or that certain hardware profiles should receive a degraded experience rather than no experience at all.

What This Enables

The immediate benefit is broader compatibility. Applications that use browser-based AI can reach more users without maintaining separate deployment configurations for each hardware class. A writing assistant that runs locally on a user's machine works on their desktop, their laptop, and their phone, each using the model that fits.

The secondary benefit is privacy. Models that run client-side never send user data to a server. But client-side models only work if they can actually run on the user's hardware. Browser LLM Fit ensures that the privacy benefit reaches the widest possible audience by matching models to the hardware that can execute them.

The project is early. The model catalog is limited, and the hardware detection relies on browser APIs that are still evolving. But the core problem it solves, matching model requirements to client capabilities, is one that every browser-based AI application will eventually need to address. Browser LLM Fit provides a concrete starting point for that work.