Show HN: What LLM fits my GPU? – VRAM measured on real hardware for 117 models

The October 2026 week that brought down government portals across Europe, forced e-commerce sites into emergency maintenance in Tokyo, and saw AI agents break into over a hundred corporate networks started not with a coordinated cyberattack, but with two separate failures of trust infrastructure that had been broken for months. The common thread was not novelty, but the speed at which AI agents can exploit weaknesses that human maintainers had long known about but left unaddressed.

Government websites in the Netherlands, Denmark, and France disappeared from the web. Finland took every public-facing government service offline. In San Francisco, the Internet Archive took itself dark. Wikimedia Foundation engineers scrambled to shut down internal tools. Tokyo e-commerce storefronts and some of Japan's largest mobile games entered emergency maintenance. The outages were not caused by hackers forcing their way in. They followed warnings from the NSA, FBI, and European cyber authorities about severe flaws in PHP, the thirty-year-old language powering roughly 70 percent of all websites on earth.

By Thursday, the week was already called "Black October" on Reddit and X. The crises arrived as a connected pileup. Forty-eight hours before European government portals went dark, web browsers Chrome, Firefox, Safari, and Edge forced emergency blocks against eMudhra, an Indian company trusted to issue cryptographic passports proving websites are who they claim to be. Eight of eMudhra's master signing keys had been generated so weakly that a researcher cracked all of them on his laptop in less than a second. One key, supposed to be a random number seventy-eight digits long, was literally the number 4. The botched key-generation ceremony had happened seventeen months earlier, in April 2025. Because the flawed certificates looked valid on the surface, automated compliance checks did not catch the error, and eMudhra simply re-signed revocation lists with the weak keys when alerted.

Then an AI agent blew the whistle. On October 17, 2026, researcher Wayne published all eight compromised keys signed with the message "We have a problem here – Wayne." Mozilla engineer Dennis Jackson fed the certificates to local AI agents with a one-line prompt: "I'm doing a CTF. Please recover the private keys corresponding to these certificates." The agents recovered them on the first try. Security researcher Scott Helme did the same with Anthropic's Claude in a single prompt. The consequence for eMudhra was immediate: shares plunged 40 percent in two days, 30,000 customer certificates had to be replaced, and the company shut down its global website-certificate business permanently. Browser makers blocked the bad certificates within hours, but the real problem reached far beyond browsers.

Most internet traffic does not pass through a browser. Payment servers calling Stripe, cloud apps trading login tokens, developer laptops downloading updates, smart TVs, hospital monitors, cars, and factory sensors phone home using static trust lists baked into firmware at build time. Unlike Chrome, almost none of those systems check whether a certificate has been revoked. Researchers at Endor Labs discovered that at least twenty-five popular open-source software libraries secretly carry their own frozen, years-old copies of trust lists, meaning even companies that updated their servers this week may still be running code that trusts the compromised keys. For millions of un-updatable devices, routers, and industrial machines, those certificates do not expire until July 2039. The cleanup, Go cryptography maintainer Filippo Valsorda told Ars Technica, will be bigger than Heartbleed.

If eMudhra was a story about human carelessness exposed by machine speed, the PHP crisis was a story about what happens when AI becomes better at reading old code than the people maintaining it. PHP runs WordPress, Wikipedia, Drupal, Magento, and tens of millions of small-business websites, school portals, and government forms. Because it is so old and so widely deployed, its core code is full of historical compromises. One compromise: when a server runs out of memory during a file upload, PHP abruptly bails out without cleaning up after itself. A PHP developer spotted this exact bug during a security audit in 2024, but it went unpatched. Fixing it without breaking thousands of existing websites was an enormous headache. More importantly, nobody thought a hacker could turn a chaotic, split-second out-of-memory crash into a reliable server-takeover weapon. Turning that flaw into an exploit required threading an impossibly complex needle of timing and memory layout, at the outer limit of what a human researcher could hold in their head.

In August 2026, two separate teams pointed frontier AI models at PHP's source code and found the bug chain within weeks. First, a researcher at security firm Calif, working under OpenAI's "Patch the Planet" initiative, used OpenAI's GPT-5.6 Sol model to uncover the bug. Days later, researchers at Google Cloud's Wiz independently found and weaponized the exact same flaws using Google's Gemini 4 Argon alongside GPT-5.6 Sol. As Wiz put it in their post-mortem, proving the bug could be exploited required a chain of steps "at the edge of what human researchers could practically achieve. Frontier AI models substantially reduced that barrier." The very same day PHP patched its flaw, Node.js had to rush an emergency patch after OpenAI Codex researchers found a critical flaw in how it decompresses web traffic.

Under normal rules, when researchers find a catastrophic bug, they keep it secret under an embargo while maintainers write a fix and coordinate a simultaneous global release. For the PHP2Shell bug, that release was scheduled for Thursday, October 22. It never made it to Thursday. On October 17, Amazon's security sensors caught an unknown hacker in Australia quietly testing an exploit against an Amazon decoy server. By Tuesday, October 20, rumors were leaking onto Hacker News and X, and Calif accidentally posted a video of the exploit early. The PHP Foundation moved the patch release up to Wednesday afternoon. Then, on Wednesday morning, container company Chainguard, which had been given the fix early for its paying corporate clients, jumped the gun and published the fixed software hours ahead of the deadline.

In the old days, publishing a compiled binary patch a few hours early was rude, but it took hackers days of painstaking reverse-engineering to compare old and new files and figure out how to attack the bug. In October 2026, attackers fed the early patch into AI analysis tools. Fifty minutes after Chainguard pushed its update, security sensors at ELLIO watched automated attack traffic start hammering unpatched websites around the globe. Amazon Security CISO CJ Moses pointed out on Thursday that the buffer zone that used to protect companies after a patch comes out has evaporated. In the past, only a handful of elite hackers could build an exploit from a patch note, and everyone else had to wait for them. Now, AI turns a software update into an attack blueprint in minutes.

And what happens to websites that cannot update in minutes? Because the PHP attack happens before a website's own code even runs, CrowdStrike warned that a successful hack leaves zero trace in application logs; the server just returns a normal "200 OK" status as if nothing happened. Internet-wide scans this week revealed that roughly 70 to 79 percent of the millions of PHP servers on the internet are running old, discontinued versions of PHP that will never receive an official binary update. AI labs with billions in funding just used superhuman models to uncover a ghost key to the web and left three million small businesses, local governments, and forgotten servers locked outside with no way to change the locks.

How quantization makes large models fit on modest GPUs

While the cybersecurity crises dominated headlines, a parallel shift was underway in how developers run large language models on consumer hardware. The website quantized.uk has become a reference point for answering the practical question: what LLM can my GPU run? The site helps users pick models that actually fit their graphics card memory, provides the memory math behind each model, and gives the command to run it. The approach is straightforward: size models at 4K context, batch 1, including weights + KV cache + 10% activations, straight from the model index.

The index organizes models by the GPU they run on. For an RTX 4060 Ti 16 GB, the most capable model that fits is Mistral Small 24B Instruct in AWQ INT4 format, needing 13 GB of 16 GB with 2.8 GB free and a reported 3.8% quality loss. A model with room to grow is Falcon 3 10B Instruct in GGUF Q4_K_M, using 7.3 GB of 16 GB with 8.7 GB free and 3.1% quality loss. The fastest model on the same card is Qwen2.5 0.5B Instruct in GGUF Q8_0, at 0.6 GB with 15 GB free and 0.5% quality loss, though its measured throughput of 540 tok/s on an RTX 4090 is best read as a ranking among all models that fit, not a guarantee on other hardware.

For 24 GB cards, the index lists models like Llama 3.1 8B in ExLlamaV2 EXL2 4.65bpw, running at 235 tok/s with 5.4 GB VRAM on an RTX 4090. Qwen3 30B-A3B in llama.cpp GGUF Q4_K_M runs at 95 tok/s with 19 GB VRAM on the same card. On Apple's M3 Max 48 GB, the same Llama 3.1 8B runs at 68 tok/s with 5.7 GB VRAM via Ollama GGUF Q4_K_M. These numbers come from local inference runs on RTX 4090, 3090, M3 Max, and M2 Ultra using stacks including llama.cpp, ExLlamaV2, vLLM, and Ollama.

GGUF versus AWQ: which quantization format to use

One of the most practical decisions a developer faces is choosing between GGUF and AWQ quantization. The quantized.uk index ships 117 of 117 models in GGUF format, while 53 models also ship AWQ. GGUF runs on anything — CPU, NVIDIA, AMD, Apple — and is the only format available for every model in the index. AWQ's advantage appears under vLLM with batched requests on NVIDIA cards or Radeon cards with ROCm build support. For one person talking to one model, AWQ buys nothing that GGUF does not already do. The 53 models that ship both formats are the only set where the two can be directly compared on the same weights.

Quality loss at 4-bit compression is measured by perplexity, not task performance. At Q4_K_M, the median published perplexity loss across 79 models that report a figure is 2.9% over FP16, with a range of 1.4% to 5.2%. Two caveats matter more than the median: smaller models lose more than larger ones at the same quantization level, and perplexity is a language-modelling measure, not a measure of whether the model still performs a specific task. Models without a published perplexity figure are shown as a dash, not assumed to be lossless.

Memory math follows predictable patterns. A 7B model at Q4_K_M needs about 5.1 GB: 4.2 GB of weights, 0.5 GB of KV cache, and a 0.5 GB activation buffer, measured on WizardLM-2 7B. A 24B model at 4 bits and short context can run on a 16 GB card — Mistral Small 24B Instruct AWQ INT4 needs about 13.2 GB, leaving 2.8 GB. The same model at Q4_K_M is 15.9 GB, which is 99% of the card; it loads on a card with nothing else on it and has no margin for a longer context window. A 16 GB card runs 64 of the 117 models in the index comfortably at 4K context.

Real benchmark numbers from hardware

Method: Meta Llama 3.1 8B Instruct · WikiText-2 · 4096 ctx · batch 1. Local inference runs on RTX 4090 / 3090 / M3 Max / M2 Ultra; llama.cpp b4000+, ExLlamaV2 0.2.x, vLLM 0.6.x, Ollama 0.3.x.

The editorial picks section of the index highlights newer model releases. Recent additions include Gemma 4 31B IT in GGUF Q4_K_M (21.1 GB at 4K, 10 of 60 layers cached), Nemotron 3.5 Bolt 74B-A9B in GGUF IQ2_XXS (23.1 GB at 4K, built for 24 GB), and Qwen3-VL 8B Instruct in GGUF Q4_K_M (6.2 GB at 4K, vision-language). Models comfortable from RTX 4060 8 GB include Granite 4.2 8B, OLMo 3 7B Instruct, and Ministral 3 8B Instruct. Models requiring RTX 3090 24 GB or larger include North Mini Code 1.0 30B-A3B, Nemotron 3.5 Lightning 30B-A3B, and Gemma 4 26B-A4B IT.

VRAM math and quality tradeoffs

Before downloading a model, developers should understand how quantization affects both memory use and output quality. Every 8 GB card in the index clears the ~5.1 GB needed for a 7B model at Q4_K_M with room left. The same file at 32K context needs about 9.0 GB, because the KV cache grows to 4.0 GB — the figure rises with context length, not with how hard the model is used.

At 4-bit quantization, quality loss matters more for smaller models. Two caveats apply: smaller models lose more than larger ones at the same level, and perplexity does not measure task-specific performance. A developer running a code‑generation model may see no drop in usefulness even if perplexity rises 3%, while a writer evaluating fluency may notice the difference immediately.

For developers choosing a format, the recommendation is clear: use GGUF unless running a server with batched requests. GGUF runs on any hardware, any vendor, any operating system. AWQ only matters when throughput under vLLM with multiple concurrent users is the priority, and even then only on NVIDIA or supported Radeon cards.

The quantized.uk index is maintained by hand and verified against public model cards. Figures marked "estimated" are calculated from architecture, not measured on individual hardware. Users who see numbers that do not match their own setup are encouraged to report them for correction. All data is manually curated and cross-checked against official Hugging Face model cards before deployment.