Rachel Lee Nabors spent years on the React core team and at FAANG before returning to engineering with Mima, a local-first social aggregator. The project emerged from frustration with walled social networks and the desire to aggregate content from X, LinkedIn, Bluesky, and Mastodon in one place without sending personal data to cloud servers. Nabors began experimenting with small language models running on-device after noticing that most hackathon projects immediately became cloud wrappers, transmitting user information to remote servers where it could be used to train competitors' models. The technical insight driving Mima is that small language models like Llama 3.2B can perform many tasks previously thought require large cloud LLMs, while avoiding high API costs, reducing latency, and protecting personally identifiable information.
Nabors has been following LLM evolution since GPT-3 and observed that developer experience consulting work exposed her to many "wrapper" architectures that round-trip every user input to a remote server. She found this wasteful both energetically and financially; research she cited indicates multimodal model inference costs four times higher than necessary for most workloads. Beyond cost, the practice introduces privacy risks—private personal information has appeared in public sources after being sent to models. The technology has advanced enough that small language models now handle summarization, tool selection, and yes/no decisions that previously required cloud endpoints. Nabors set up evaluations using the Phoenix library and found Llama 3.2B could perform as well as Claude Sonnet for summarizing conversation threads, while consuming far less energy and eliminating the need to transmit messages off-device.
Model selection with golden datasets
Systematic evaluation using golden datasets became central to Nabors' approach. Rather than relying on community hype—Gemma 4 was "really hot right then" according to her—she ran tests against real conversation data and used a large model as a judge to pick the best option for each task. This method revealed that Llama 3.2B provided better summarization than Gemma 4 while maintaining lower latency. The evaluation process also surfaced model strengths and weaknesses: Llama 3.2B, trained on user data, excels at summarizing human conversations but may not be optimal for toxicity detection or image analysis. For moderation tasks, smaller lighter models perform better, and all such features remain opt-in within Mima. Users can choose to download additional models for specific capabilities rather than having them installed by default.
Client-side AI for moderation and context
Nabors is a strong advocate for client-side moderation rather than relying on platform owners to handle abusive content. She argues platform operators treat moderation as a cost center and "cheap out on it," creating environments she describes as "Lord of the Flies, the kids are running the schoolroom." Mima includes models specifically trained to check for abusive or unfriendly content, giving users control over trigger warnings and content filters. The same principle extends to personal context: once AI assistants reach the consumer market, they lack the personal context needed to be genuinely helpful. Connecting an assistant to email alone is insufficient—users also want integration with LinkedIn for job searching, Facebook for keeping up with children, and other walled gardens. The power of such data is "only really unlocked when it's cross-contaminated, which is a huge security risk," but personal agents would become "much more powerful for having that context, not even necessarily to post, but they would just know more about us and be better able to help us as individuals."
OS-level personal agents and UI standards
The conversation turned to the future of web and application architectures. Nabors mentioned A2UI, a dynamic UI rendering standard that could replace traditional web apps and native interfaces for consumption. OS-level personal agents paired with standardized UI schemas may eventually change how users interact with social networks and information. For software engineers, the skill set is moving beyond basic code generation toward computer science fundamentals, system design, and AI collaboration. Nabors emphasized that the user should always be able to opt in to enhanced experiences, particularly when those experiences require additional storage space or model downloads. The default should remain lean, with users choosing whether to add capabilities.
Nabors also discussed the architectural philosophy of local-first applications. Building apps this way delivers fast, offline capability for users while keeping cloud infrastructure costs low for developers. Small language models running directly on-device avoid the high API costs of cloud-hosted wrappers, reduce latency, and protect PII. The Mima client uses native web APIs under the hood, and all models are opt-in—users download additional models only for features they enable. This approach aligns with a broader movement away from servers holding personal information, particularly given geopolitical risks around data requests.
Subscribe on: The Future of Web Apps podcast