Large language models now solve competition-level mathematics yet it remains unclear whether they grasp the structural logic behind their solutions. This paper introduces the Mathematical Primitive as a probe for structural understanding and releases \hlei{}, a benchmark that splits mathematical reasoning into four measured dimensions: Discovery, Generation, Digestion, and Execution. A broad diagnostic across model families shows that overall accuracy hides divergent capability profiles, that primitives reveal hidden execution strength, and that Discovery acts as the primary bottleneck. Post-training analysis confirms that discovery-limited failures are the most responsive to targeted repair. The authors then build \abs{}, a primitive-privileged self-distillation method that transfers structured primitive-level reasoning into a student model. Experiments across scales from 7B to 70B parameters demonstrate consistent gains over standard distillation on \hlei{} and on out-of-distribution benchmarks including MATH and GSM8K.
Why surface accuracy is insufficient
Models can reach correct answers through pattern reuse without learning the operations that generate those answers. A system that memorizes solution templates for integral calculus problems may fail when the integration variable changes or when a substitution step is required. Without a way to isolate which reasoning step fails, developers cannot tell whether a model needs better algebraic manipulation, better strategy selection, or better symbolic rewriting. The four-dimensional split in \hlei{} addresses this by making each primitive independently measurable.
Defining the four primitives
Discovery asks whether the model can identify a viable approach for an unseen problem — for instance, recognizing that a functional equation problem calls for substituting specific values or exploiting symmetry. Generation asks whether the model can produce a valid solution sequence once the approach is fixed. Digestion asks whether the model can transform an expression into an equivalent but more useful form, such as factoring a polynomial or completing a square. Execution asks whether the model can perform a deterministic computation step correctly, like evaluating a derivative or reducing a fraction. Each primitive is tested with prompts that isolate it from the others, so a low Execution score cannot be blamed on poor Discovery.
Diagnostic findings across model families
The authors evaluate open-weight models including Llama, Qwen, and Mistral variants, plus several closed-weight systems. Two consistent patterns emerge. First, models with similar overall accuracy on standard benchmarks show widely different primitive profiles. One 70B model may score 85 percent on Execution but only 45 percent on Discovery, while another scores 65 percent on both. Second, when primitives are tested in isolation, Execution scores jump significantly — models that struggle to solve full problems often execute individual steps correctly when given the intermediate state. This gap between latent execution capacity and full-problem performance points directly at Discovery as the limiting factor.
Discovery failures are the most repairable
Post-training experiments fine-tune models on data enriched for specific primitives. Adding Discovery-oriented examples — problems where the prompt explicitly requests a strategy or decomposition — raises Discovery accuracy by 15 to 20 points and produces measurable gains on Generation and Execution. Fine-tuning on Generation or Execution alone yields smaller, more localized improvements. The asymmetry occurs because Discovery sits upstream: a model that picks the wrong strategy never reaches the steps where Generation or Execution matter.
Primitive-privileged self-distillation with \abs{}
Standard distillation trains a student to match the teacher's final output distribution. \abs{} instead decomposes the teacher's reasoning trace into primitive-labeled segments. The teacher generates a solution, an automated annotator tags each step with its primitive type, and the student is trained to predict the correct primitive label and content at each position. During inference, the student must emit a primitive label before producing the corresponding reasoning text. This forces the student to replicate the teacher's structural decisions, not just the surface tokens. The framework is model-agnostic and adds minimal overhead — only the primitive tag vocabulary and a lightweight classification head.
Results: consistent gains across scales and benchmarks
On \hlei{} itself, \abs{} lifts Discovery by 12 to 18 percentage points across all model sizes tested, with Execution gains of 8 to 14 points. Generation improves modestly, 3 to 6 points, reflecting its intermediate position in the pipeline. On MATH and GSM8K, \abs{} outperforms baseline distillation by 4 to 7 points absolute accuracy at every scale. The gap widens on problems the teacher originally failed at Discovery, confirming that the framework targets the dominant failure mode. Ablation studies show that removing the primitive tags collapses performance to baseline levels, verifying that the structure — not just the extra data — drives the improvement.
Limitations: scope of primitives and teacher dependence
The four primitives do not cover proof planning, geometric reasoning, or multi-step symbolic derivation that spans many intermediate states. The benchmark focuses on algebraic and analytic problems typical of high-school and undergraduate curricula. \abs{} also requires a teacher that already demonstrates reasonable primitive-level behavior; a teacher that cannot solve the primitive sub-tasks will not provide useful supervision. The authors observe diminishing returns when the teacher's Discovery accuracy falls below 50 percent. Finally, the primitive annotation step adds a preprocessing pass that scales with dataset size, though it is parallelizable and does not require human labeling.
What this means for teams building mathematical LLMs
The \hlei{} benchmark is open and lightweight — it runs in minutes on a single GPU and produces a four-number diagnostic that tells you exactly where your model stands. If Discovery is the weak link, invest in strategy-annotated data or prompt templates that force decomposition before solving. If Execution lags, add deterministic step-checking or tool use for algebraic simplification. The \abs{} codebase provides a drop-in distillation pipeline: tag your teacher's outputs, train the student with primitive heads, and deploy. For organizations without massive compute, the finding that small models gain disproportionately from primitive-privileged distillation means a 7B student can close much of the gap to a 70B teacher on mathematical reasoning without matching its parameter count.