Piaget argued that children develop spatial understanding in a fixed sequence: topological relations come first, followed by projective and Euclidean ones. The child first grasps that two points are connected, that one region encloses another, that a sequence preserves its order, before ever computing distance or angle. This ordering is not pedagogical convenience. It reflects a genuine cognitive dependency: you cannot measure how far apart two objects are until you know whether they are separated at all. Yet every major spatial reasoning benchmark for foundation models tests metric properties: distance, direction, size, shape, 3D location. The topological layer is missing entirely.

Ge, Liu, Wang, Garnica, Lyu, Wang, Tan, Gao, Zhang, Hong, Wu, and Li introduce MindTopo to close this gap. It is a benchmark of topological intuition across five properties grounded in both Piaget's developmental taxonomy and formal algebraic topology: continuity, separation, order, enclosure, and knots. The benchmark tests these properties at two cognitive levels: reasoning, where a model identifies topological relations in static scenes, and planning, where a model acts as a closed-loop agent whose actions must preserve or transform topological structure. MindTopo contains 11,030 instances across 13 procedurally generated task types with controllable difficulty, benchmarking 14 multimodal large language models from GPT-5.6-Sol down to 7B-parameter open-weight models, plus three video generative models used as imagined world models for planning.

Why topology is a distinct layer of spatial reasoning

Topology studies spatial relations that remain invariant under continuous deformation. Bending a rope without cutting it does not change whether it is knotted. Stretching a surface does not change how many holes it has. Adding a wall to a maze changes distances but does not change which rooms are connected. These invariants are more primitive than metric properties: the visual system extracts topological features before Euclidean ones, and children master topological relations before they understand distance or parallel lines.

For foundation models, this distinction matters because spatial reasoning benchmarks overwhelmingly test metric properties. A model can score well on spatial QA by computing relative distances or counting objects, without ever understanding whether two regions are topologically distinct. MindTopo isolates the topological layer. Each task requires the model to reason about properties that depend on connectivity, separation, enclosure, or entanglement, not on measurements that change when you stretch or rotate the scene.

Five properties, two cognitive levels

The benchmark organizes topological intuition into five properties. Continuity captures whether a path or surface forms an unbroken whole. Separation captures whether neighboring elements can be distinguished as distinct units. Order captures the sequential arrangement of elements along a path or boundary. Enclosure captures the inside/outside relation induced by a closed boundary. Knots capture entanglement that persists under deformation. Formal topology provides the invariants for each: path-connectedness for continuity, decomposition into connected components for separation, cyclic order types for order, the Jordan Curve Theorem and first Betti number for enclosure, and ambient isotopy class for knots.

Each property is tested at two cognitive levels. Reasoning tasks present one or more rendered scenes and ask the model to identify a topological relation or predict how it changes after a stated edit. Planning tasks place the model in an interactive gym environment where it must select actions that achieve a structural goal: connecting pipe segments into a continuous network, partitioning a colored grid with a single stroke, solving a sliding-block puzzle, enclosing a moving cat on a hexagonal grid, or untangling crossed ropes. The two levels are deliberately paired at the property level: 2D Maze (reasoning) and Pipe (planning) both test continuity, Assembly and One Stroke both test separation, Bead and Swap both test order, Sheep/Hole and Chat Noir both test enclosure, Knots and Untangle both test knots.

This pairing lets the benchmark ask a specific question: does the ability to recognize a topological relation transfer to the ability to act on it? The answer, across all 14 models, is no.

Thirteen tasks with procedural generation

MindTopo contains 11,030 instances: 8,030 reasoning questions across eight task types and 3,000 planning episodes across five task types. All scenes are procedurally generated with controllable difficulty and ground truth computed programmatically from scene state.

The eight reasoning tasks include 2D Maze and 3D Maze (continuity: which points are reachable through a maze), Assembly (separation: which components form a topologically separable sub-assembly), Bead (order: enumerate the sequence of colored beads on a curved string), Origami Point (order: determine whether a folded pattern preserves cyclic order), Sheep (enclosure: which animals lie inside an enclosure boundary), Hole (enclosure: count through-holes in a solid object), and Knots (identify whether a loop is knotted or merely tangled, whether two loops are linked, and what happens when a ring is cut). Each reasoning task includes what-if sub-questions that test whether the model can predict how topological relations change under edits.

The five planning tasks include Pipe (connect rotatable pipe segments into a continuous network), One Stroke (draw a single path that separates same-colored cells), Swap 2D Puzzle (solve a sliding-block puzzle to reach a target permutation), Chat Noir (place blocks to encircle a moving cat before it escapes), and Untangle (move plugs on a board until no ropes cross). Each episode runs for a fixed horizon and succeeds only when the terminal condition is met.

Difficulty varies by maze wall count, grid size, crossing count, and number of steps required. The full generation pipeline is automated: scene generation produces rendered images, topological annotation computes reference answers from scene metadata (graph search, geometric membership tests, simulator predicates), difficulty control varies parameters, and acceptance checks filter invalid or ambiguous instances before export.

Benchmark results: GPT-5.6-Sol leads but stays far below humans

The headline numbers tell a stark story. By task-macro average, GPT-5.6-Sol reaches 61.42%, far below the observed human performance of 97.87%. The gap is not uniform across the benchmark. Every model scores higher on reasoning than on planning, and the gap widens as model capability decreases.

For the top proprietary models, GPT-5.6-Sol averages 66.83% on reasoning tasks but drops to 52.75% on planning. Gemini-3.1-Pro goes from 52.24% to 19.23%, and GPT-5.5 from 53.77% to 24.33%. For the best open-weight model, Qwen3.5-397B-A17B, the numbers are 26.54% on reasoning and 5.27% on planning. Every one of the nine open-weight models scores 0% on both Pipe and One Stroke planning tasks.

The three planning environments where all models struggle most are Pipe (best score: 50.50% by GPT-5.6-Sol), One Stroke (best: 35.00%), and Untangle (best: 29.17%). These tasks require constructing or preserving topological structure through sequences of legal actions, and the current generation of models cannot reliably do this even when they can identify the relevant topological relations in static images.

No single model dominates across all five properties. GPT-5.6-Sol leads on continuity (65.98% average), separation (45.96%), order (76.17%), and enclosure (65.73%). Gemini-3.1-Pro leads on knots (51.38%). Within a single property, the reasoning leader and the planning leader can differ: Gemini-3.1-Pro edges out GPT-5.6-Sol on Sheep reasoning (64.00% vs. 63.94%), but GPT-5.6-Sol dominates Chat Noir planning (69.30% vs. 40.67%). The five Piagetian primitives engage different weaknesses, so strength on one is not predictive of strength on others.

Training on Qwen3-VL-2B: SFT and RL improve reasoning, not planning

The authors test whether the reasoning-planning gap can be closed through training. Starting from Qwen3-VL-2B-Instruct, they compare a frozen base policy with supervised fine-tuning (answer-only LoRA, rank 32, one epoch), reinforcement learning via GRPO (Group Relative Policy Optimization), and SFT followed by GRPO.

The base model averages 8.00% across nine tasks (five reasoning, four planning). SFT raises this to 28.83%, GRPO alone to 18.69%, and SFT plus GRPO to 31.44%. The combined policy achieves 79.60% on Knots reasoning and 58.20% on Sheep reasoning, strong gains from a 2B model. But planning barely improves. Under SFT plus GRPO, average planning success rises from 0.20% to only 6.33%. Pipe stays at 0% under every training condition. The best One Stroke result is 0.80%.

The leave-one-task-out experiment tests whether training on four reasoning tasks transfers to the fifth. It helps: held-out Bead rises from 9.31% to 26.53% and held-out Knots from 11.20% to 15.50%. But it does not help on 2D Maze, Assembly, or Sheep. Transfer is selective rather than systematic, suggesting that topological reasoning does not yet generalize from task-specific supervision to a general policy.

Error analysis: different failure modes for reasoning and planning

The authors label 910 incorrect predictions from Gemini-3.1-Pro and InternVL3.5-241B across 26 model-task pairs. The error taxonomy follows the processing pipeline from instruction compliance through action planning, with failures assigned to the earliest applicable category.

Reasoning errors concentrate at the perception stage. For Gemini-3.1-Pro, 64.2% of reasoning failures are perception-grounding errors: the model misidentifies objects, misreads positions, or misses visual elements. Another 19.7% are feature-state prediction errors: the model correctly identifies the current state but predicts the wrong outcome after a specified change. For InternVL3.5-241B, perception grounding accounts for 54.6%, with instruction following at 20.1% and task misunderstanding at 14.7%. Explicit topological-invariance errors (false invariance or false change) are relatively infrequent at 3.8% and 5.4% respectively, because earlier failures explain the wrong answer before the topological reasoning itself is reached.

Planning errors shift to downstream stages. For Gemini-3.1-Pro, 56.6% of planning failures are action-planning errors (legal but irrelevant actions, short-horizon plans), 29.1% are feature-state prediction errors, and 4.3% are dynamic violations. For InternVL3.5-241B, the profile reverses: 41.0% are dynamic violations (barrier or collision violations, unsupported continuous motion), 32.4% are action-planning errors, and 21.4% are instruction-following errors. The two settings require different diagnostic targets: reasoning is gated by visual extraction, planning by valid state-transition sequences.

Can video generation help? Not reliably

The authors test whether augmenting the planner with imagined visual observations can preserve topology during action sequences. Three video generative models are evaluated: Wan2.2-I2V-A14B (14B parameters, self-hosted), Seedance-2.0-Mini (ByteDance), and Veo-3.1-Lite (Google). Each generates a video of the agent's imagined future observations given the current state and proposed action.

The results are discouraging. Across 945 audited rollouts, no generator achieves process-valid task success on One Stroke, Pipe, or Untangle. Process-valid means the generated video respects all static constraints (color separation, pipe connectivity, rope integrity) and all dynamic constraints (valid rotations, no barrier violations, topology preservation across transitions) at every frame. Wan2.2-I2V-A14B violates environment dynamics in 116 of 119 audited rollouts. LTX-2.5 achieves 35.2% terminal-state acceptance on Pipe (the video's final frame looks like a valid pipe network), but 0% process-valid because intermediate frames are inconsistent.

Image generation (GPT-Image-2 interleaved with the planner) performs better than video on some tasks. GPT-5.6-Luna plus GPT-Image-2 achieves 68.6% on Sheep and 61.9% on 2D Maze, suggesting that single-image prediction of the next state is more reliable than video generation of a full trajectory. But even image augmentation cannot help on One Stroke (0% across all tiers) or Pipe at hard difficulty (2.86%).

The core failure mode is topology flicker in video generation: the generated frames preserve local visual cues (rope color, endpoint positions) but do not maintain global topological properties across transitions. A rope that is unknotted in one frame becomes knotted in the next, or a pipe that is connected in the final frame is disconnected in intermediate frames. This is exactly the property that topology is supposed to enforce, and current video models cannot do it.

Topological biases in foundation models

The authors identify three systematic biases through their probing experiments. First, a connectivity-by-default bias: models tend to assume that connected-looking regions are connected even when the visual evidence is ambiguous. Second, hole undercounting: models consistently underestimate the number of through-holes in solid objects, with the effect becoming more pronounced as difficulty increases. Third, knot-structure collapse: models confuse visually tangled but unknotted loops with genuinely knotted ones, treating visual complexity as a proxy for topological complexity.

These biases explain some of the performance patterns. The Sheep task, which tests enclosure, shows strong answer-prior and text-only performance, meaning the model can often guess the correct answer from the question text alone without looking at the image. The Hole task shows an exact-count cliff: performance drops sharply at specific hole counts rather than degrading gradually, suggesting the model is counting and miscounting rather than failing to understand the task.

What this means for foundation model development

MindTopo establishes that current models have a specific, measurable deficit in topological reasoning that is not captured by existing spatial benchmarks. The reasoning-planning gap is the central finding: models can identify topological relations in static images at moderate accuracy, but cannot use those relations to guide valid action sequences. This is not a general planning failure in the sense that models cannot plan at all, since GPT-5.6-Sol achieves 87.3% on Swap planning. It is specifically about constructing and preserving topological structure through actions.

The training results on Qwen3-VL-2B show that SFT and RL can improve reasoning substantially, but planning gains are negligible. The pipe planning task stays at 0% under every training condition tested. This suggests that the planning deficit is not simply a data or compute problem that more training will fix. It may require architectural changes: the model needs to maintain and update a topological representation of the environment state across multiple action steps, which current MLLMs do not explicitly do.

The video generation results are the most practically relevant for anyone building embodied agents. If a model cannot predict how its actions will change the topological structure of its environment, it will make errors that look like hallucinations but are actually structural: placing a block that appears to close a boundary but actually leaves a gap, or rotating a pipe that appears to connect but actually disconnects. The 0% process-valid rate across all video generators means that no current video model can serve as a reliable world model for topological reasoning.

For researchers building spatial reasoning capabilities into foundation models, MindTopo provides a structured evaluation that separates metric reasoning from topological reasoning, and separates recognition from action. The five properties and thirteen tasks give a diagnostic profile, not just a single score, making it possible to identify which specific topological intuition a model lacks and whether training improvements generalize across properties or remain task-specific.

The benchmark is also a warning about what aggregate scores hide. GPT-5.6-Sol's 61.42% average looks reasonable until you see that it comes from 84.4% on 2D Maze reasoning and 21.67% on Untangle planning, with performance varying wildly across the five Piagetian primitives. A model that scores 77% on Bead (order reasoning) can score 35% on One Stroke (separation planning) and 21.67% on Untangle (knots planning). The properties are not interchangeable, and a single number is not enough.

Read the paper on arXiv