A GitHub Gist posted on September 20, 2026 strips a language model down to its barest form: roughly 15 lines of JavaScript that can generate text from a handful of training words. The code, published by skorotkiewicz under the title "the smallest LLM," is worth examining not for what it produces but for what it reveals about how language models actually work.
What the code does
The training corpus is a single string: "mama dad mama cat dad mama dog". The model splits that string into words and builds a lookup table mapping each word to the words that follow it. "mama" is followed by "dad," "cat," and "dog." "dad" is followed by "mama" twice. Every other word in the vocabulary appears exactly once with no successor.
When you ask the model to generate text starting from "mama," it picks a random word from the list of followers. If the current word has no entry in the table, it falls back to the full vocabulary and picks at random. The result is a sequence of words that looks vaguely plausible because it was assembled from patterns extracted during training.
This is a bigram model. It looks exactly one word back to decide what comes next. That is the minimum viable version of the idea behind GPT, Claude, and every other modern language model, which do the same thing but across billions of parameters and millions of tokens of context instead of seven words and a single lookback.
Why this matters as a teaching tool
Modern LLMs feel like black boxes because their scale makes the mechanics invisible. You cannot hold a 70-billion-parameter model in your head. You can hold this one. The entire state is a JavaScript object with seven keys. The entire inference loop is a for-loop that picks a random array element. The entire training step is a single pass through a string with split and push calls.
The core insight is that language modeling is prediction. Given a sequence so far, guess what comes next. Every token a modern model outputs is the result of that same operation, repeated thousands of times, with the "guess" refined by massive training data and learned statistical relationships. This Gist shows the operation in isolation, without the noise of neural networks, attention mechanisms, or GPU clusters.
What the limitations teach
The fallback behavior is the most instructive part of the code. When the model encounters a word it has never seen as a context, it picks from the entire vocabulary at random. This happens because the training data is too small to cover every possible context. In a real LLM, this same problem manifests as hallucination: the model generates confident-sounding text that is statistically plausible but factually wrong, because it has no reliable signal for the current context.
The model also has no notion of meaning. It does not know that "mama" is a family member or that "dog" is an animal. It only knows which words tend to appear after which other words. This is the same mechanism at work in models that produce fluent paragraphs about topics they clearly do not understand. The difference is that a bigram model's mistakes are obvious, while a large model's mistakes can be subtle enough to fool an expert.
Training on seven words also means the model has almost no generalization. It can produce sequences that look like the training data, but it cannot produce anything genuinely new. Scaling up training data and model size is what enables generalization, but the fundamental operation does not change. The Gist is a reminder that complexity is not the same as magic, and that even the most sophisticated language models are built on a single, simple idea.