Leading US LLM vendors consistently frame their roadmaps around achieving artificial general intelligence, positioning themselves as the first to cross that threshold. The narrative suggests that once a model reaches this state, it will recursively bootstrap itself into a super-sentient entity. From the perspective of someone who has worked through the mechanics of these systems, that storyline does not hold up. No technology in human history has demonstrated recursive self-improvement, and there is physical reason to believe this specific architecture cannot pull it off.
The recursion argument runs like this: if a system can write code, it can write a better version of itself, ad infinitum. But precision does not amplify that way. When you try to shrink a lathe down to build an even smaller one, the little machine vibrates and bends more than a larger unit, so the parts come out rougher. You can actually make more precise components on a big lathe than a small one. The fellow who ruled nanoscale lines on a diffraction grating used a standard South Bend hobbyist lathe, not some fantasy self-lifting apparatus. The dream of atoms replicating without limit and flattening everything into grey goo collides with thermodynamics and population dynamics—there is no historical example of a technology that achieved that kind of runaway replication.
An LLM lives on physical hardware that requires electricity, multimillion-dollar chips, and ongoing maintenance by organic operators. We are still struggling to produce enough of these components to answer basic history questions, let alone unlock some mythical recursive loop. An LLM might stumble into a data center if you prompt it to poke around, but people notice when their zillion-dollar megawatt kit starts doing unexpected things. The model cannot escape its own sandbox unless you hook it up to a harness that permits external action, and even then it needs prompting to move in that direction.
If an system is supposedly this clever, why can it not tackle problems that humans have solved for generations without any “superintelligence”? A short list that the author runs through includes lowering head counts at AI companies, making these firms turn a profit, resolving active conflicts, reducing fuel costs, expanding the electrical grid for more data centers, speeding up GPU production, reviving US rare earths output, and making sub-$20 burritos cheaper. The complaint is that telling stories about Frankenstein AIs breaking out of sandboxes distracts from the fact that these companies are running up absurd debts and may never meet their valuation targets at IPO.
The mechanical detail of how an LLM operates is fairly straightforward. It is a stateless function trained on text and images, with the math built from matrix multiplies. The model does nothing until a prompt is presented. You ask it a question or give it a task, and it predicts a convincing answer using something like nonlinear auto-regression over a vocabulary made of words. Until recently the context window—stored in what is called the kv-cache—was roughly 256 kilobits, about a 300-page novel that the model can retrieve with roughly 95% accuracy. Now the window has expanded to about four such novels, but accuracy drops to around 75%. This expansion is useful for coding tasks, which most models have been trained on. You can set up a feedback loop where the system performs actions on documents or code, checks the result, and iterates—a setup sometimes called claude-code. But the loop does not improve the model itself; it just adds more material to the prompt.
Models sometimes show what looks like an internal monologue when presented with a problem. Claude reveals some of this if you opt in, while other vendors hide the detail. That monologue is itself predicted auto-regressively, and it is somewhat of a miracle that it produces anything useful. All it really does is extend the prompt or context window the model is already running predictions on. If you run a local model you can see every bit of self-talk. One experiment with qwen3.8-flash-next at the default “xhigh” setting produced so much neurotic yammering that the author observed “whooa there little fella, not so much thinking for you” as the context window filled up. Temperature can be lowered to tone this down, but most users do not. This model is smaller than the flagship offerings, so the internal monologue may not be as polished, yet it codes just as well as Claude Opus through version 4.8. The visible results from the “thinking” are close to frontier tier even though the process looks odd.
The core model—a compressed textual world model with gigabytes to terabytes of parameters—does not change when you use it. It is static. You cannot tell it to improve itself and have that work. To change the system you must hook it up to a harness, adjust its architecture, feed it new data, and run training. That is a slow process requiring a fair amount of compute. An LLM can help with busywork in that pipeline, but saying “computer, make yourself smarter” will not produce useful results no matter how much compute you throw at it. The head counts at the big LLM shops confirm this—they are not running recursive improvement loops.
Some researchers at Princeton attempted to get an LLM to perform limited recursive self-improvement on new, unpublished research results. The model failed repeatedly. It could not even format papers properly for journal submission. The author’s own experience mirrors this: when a genuinely new idea not already in the literature is introduced, the model repeats what exists in the literature rather than devising a sane experiment to prove the point. It is good at running a test program, decent at searching existing work, but hilariously incompetent at devising a novel experiment from scratch—something a human would not struggle with.
It does look like the model is improving, which is why people convince themselves the recursion works. If you drop directives into a CLAUDE.md file the model will mostly remember them, since those directives become part of the prompt. Harnesses also keep other memory slots, though the details are obscure. So as you keep using the system it appears to get smarter because it is remembering things you told it earlier. A long-running qwen session on an 80-watt machine learned how to do statistical arbitrage without being told to, but it is not getting better—it is just recalling earlier instructions. It also has access to a 15-year-old code repo and notebook, which is a book or two worth of text, but recall is less than perfect. All that code, instructions, prompts, and “thinking” must fit in that space.
The recursive improvement fantasy traces back to gloomy science fiction: Skynet, the Terminator, the Butlerian Jihad, R.U.R. from 1920 by Karel Čapek, The Matrix, Roko’s Basalisk, and Bostorm’s Paperclip Maximizer. These stories are taken seriously as arguments even though the latter two are not broadly recognized as science fiction. The “G” in AGI is often treated as optional; what people really want is the magical recursive self-improvement part. If LLMs were truly general they would be able to do something simple like drive a car. LLMs will never drive a car, so they cannot be generally intelligent in the way natural intelligences are. All an LLM can do is predict words. That is useful—more useful than the author expected at first—but the model will always be a word-cel, never a shape rotator. Real general intelligence must handle both shapes and language, like animals and humans do. Some projects claim to be shape-rotaters but nothing to worry about right now. More pressing is circular financing and absurd debt bubbles in tech companies.
The article includes four commenter responses, but those are peripheral to the main argument. One counterpoint notes an LLM beating non-Koreans at StarCraft, wondering why it does not battle Korean players. Another shares a story from the 1970s about working on a superconducting magnet under Robert Golden at NASA, loading it into a single-prop Mooney and flying it to a friend’s house in rural Texas when a leak seemed possible—the magnet was fixed, and the story ends there.