Why 3D generation lags behind text
Frontier large language models can solve complex reasoning problems yet consistently fail at producing usable three.js assets. General Context Labs identified this gap after building instaplay.ai, a platform hosting over 40,000 AI-generated games. While models can describe game mechanics for intricate worlds, the three-dimensional objects they output often miss the mark. The team created 3JSBench, the first open-source benchmark designed to evaluate LLMs on their ability to generate coherent three.js assets, providing a deterministic way to measure what these models get wrong.
3JSBench: a deterministic evaluation framework
Most multimodal evaluations rely on model-as-judge approaches, such as using vLLM to judge screenshot quality. 3JSBench takes a different path: it measures geometric correctness deterministically. The benchmark uses a custom linker that identifies objects within generated scenes, allowing 697 deterministic checks across 100 tasks. This approach sidesteps subjectivity and gives developers a clear, repeatable signal about a model's three-dimensional reasoning ability.
How the linker works: 697 checks across 100 tasks
The custom linker parses generated three.js scenes and isolates individual objects. It then runs a suite of 697 checks across 100 distinct tasks. One common failure mode involves objects that must interact physically: when one piece has to rest on, hang from, or lean against another, almost half of all attempts miss the mark. Intersecting geometry and flickering textures also appear frequently, and these failures render assets unusable in practical three.js workflows.
Connection failures: when pieces don't fit together
Models mostly build the right components—they create the correct shapes, sizes, and surface details. However, the connections between those components remain unreliable. When a piece needs to rest on another, hang from a support, or lean at an angle, the model frequently produces geometry that fails physical constraints. These gaps mean that even visually convincing models cannot be dropped directly into a three.js scene without manual repair.
What players prefer: Opus 5.5 versus Astra
Deterministic checks alone do not capture visual quality. To address this, General Context Labs collected over 9,500 votes from more than 1,000 game creators. These creators compared generated assets side by side. The results showed that users prefer Opus 5.5 slightly above Astra, even though Astra passes more of the geometric correctness checks. This indicates that human judgment of what "looks right" can diverge from strict geometric validity.
Beyond correctness: human preference data
The preference vote data reveals an important insight: a model can score well on deterministic geometric checks yet still lose in side-by-side human comparisons. Conversely, a model that passes fewer checks can still win user preference if its output has fewer visual artifacts or more coherent styling. For developers building or selecting models for three.js content creation, this means evaluation pipelines need both deterministic correctness metrics and human preference data to get a full picture.
Getting hands-on with the benchmark
The 3JSBench platform allows anyone to explore every task, inspect the verifiers, and interact with the generated objects directly. The team invites collaboration and provides a contact email for those interested in learning more or contributing to the benchmark's development. By making the framework open source, they aim to establish a standard way to measure and improve three-dimensional generation across different model families.
Practical implications for developers and teams
For teams integrating generative AI into workflows that require three.js assets, 3JSBench offers a concrete way to identify where models struggle. The benchmark highlights that connection logic—objects supporting other objects, hanging from fixtures, or leaning against surfaces—represents a significant weak point. Developers can use these insights to curate training data, design prompting strategies that emphasize structural constraints, or select models that score higher on connection tasks. The preference data also suggests that visual coherence matters as much as geometric accuracy, so testing with real users remains valuable even when deterministic scores look strong.