The One-Direction Assumption Breaks Down for Non-Binary Concepts
Interpretability researchers have spent years refining the idea that concepts in language models live as linear directions in activation space. For binary concepts like refusal, this works well enough: find a single direction, add it to the residual stream, and the model refuses. But most concepts are not binary. Animals contain mammals, birds, reptiles, and insects, each with dozens of instances. Countries span continents and cultures. Days of the week form a cycle. The search space over possible representation geometries for these multinomial concepts is far larger than for binary ones, and it is not clear what geometries are most appropriate, nor what methods are most effective at recovering them.
A team from Boston University, Technion, and Harvard's Kempner Institute, led by Divya Appapogu and Aaron Mueller, introduces MAxBench: a geometry-agnostic evaluation framework for multinomial concept representations. The framework evaluates 10 localization methods across 5 geometry types on 6 concepts and 4 language models, using a sampling-based approach that avoids assuming any particular geometry. The results are both practical and sobering for the interpretability community.
Why Binary Steering Methods Do Not Generalize
The core challenge is geometric. A binary concept like refusal can be captured by a single direction in activation space: the difference between mean activations for refusing and complying prompts. But a multinomial concept like Animals requires representing many subcategories, each with multiple instances. The concept might be a set of directions, a Gaussian cloud, a linear or affine subspace, or a non-linear manifold. The dimensionality or rank of a concept representation is generally not known in advance and likely varies significantly across concepts.
Recent evidence supports this. Some concepts form polytopes. Others lie on multidimensional manifolds. Days of the week tend to form a circle with ordered values. But you cannot assume a priori that any given concept will have such structure. This means the interpretability community needs evaluation methods that do not presuppose a particular geometry, and needs to compare methods across a range of possible geometries.
MAxBench addresses this by defining a concept representation as a geometric object in activation space that captures both the boundaries of and variance within a semantic concept. A good representation should be complete (all relevant information is recoverable from it) and exclusive (no unrelated information is recoverable from it). The framework evaluates these properties through sampling and intervention rather than through assumptions about geometry.
The 10 methods evaluated span five geometry types. One-dimensional methods include DiffMean-r1 (the classic difference in means) and ReFT-r1 (rank-1 Representation Finetuning). Affine subspace methods include DiffMean (pairwise differences between class-conditional means), Linear Probe, Schatten Probe (a low-rank extension of linear probes using approximate Schatten-p norm regularization), PCA, and Factor Analysis. Collection of directions is represented by Sparse Autoencoders (SAEs), which learn a dictionary of feature directions and select the top-k features by Matthews Correlation Coefficient. Gaussian mixture is represented by Mixture of Factors Analyzers (MFA), which models activation space as a collection of low-dimensional Gaussian components. Non-linear manifold is represented by spline fitting, which fits a cubic spline through projected centroids.
Sampling as Evaluation
The evaluation pipeline has three stages. First, each localization method recovers a concept geometry from activation vectors extracted at a specific layer of the target model (layer 19 for Llama-3.1-8B, layer 17 for Gemma-3-1B, layer 53 for Gemma-3-27B, layer 27 for Qwen-3.5-4B). Second, points are sampled from the recovered geometry and used to intervene on the model's hidden activations. For affine subspaces, coefficients are drawn from a Gaussian distribution and multiplied by the basis matrix plus offset. For non-linear manifolds, a global parameter is sampled uniformly along the spline. Third, the resulting model outputs are evaluated on four metrics.
The concept score measures exclusivity: whether the generated output contains text relevant to the target concept. It is graded on a 0-2 scale. The diversity score measures completeness: normalized Shannon entropy at both instance level (how many distinct entities appear) and category level (how broadly the generations span higher-level semantic categories). The fluency and instruction-following scores validate that steering does not cause degenerate outputs. The MAxBench score is the harmonic mean of all four, providing a single composite metric that rewards both reliable steering and broad coverage.
The dataset consists of six concepts generated using GPT-5: Animals, Countries, Vehicles, and Plants (hierarchical and unordered), Days (categorical and ordered), and Years (continuous and ordered). Each concept has hundreds of instances organized into categories. For Animals, categories include mammals, birds, reptiles, and insects, with multiple entities per category. Prompts are constructed so the model must predict the concept token as the next token, with the correct entity appearing in the top-5 predictions at least 70% accuracy across all concepts.
Affine Subspaces Win, and the Offset Matters More Than the Basis
The headline finding is that affine subspaces consistently outperform one-dimensional methods. Multidimensional methods achieve higher MAxBench scores across all concepts and models. This is not surprising for multinomial concepts, but the magnitude of the difference is notable. On Gemma-3-1B, the best affine methods (DiffMean Pairs, Factor Analysis, Schatten Probe) achieve MAxBench scores around 1.55-1.59, while one-dimensional methods (DiffMean-r1, ReFT-r1) score around 0.68-1.04.
The more interesting finding is why affine methods work better. By comparing affine variants (with offset set to the mean of concept-related points) against linear variants (with offset at the origin), the authors show that the offset contributes significantly to performance. An isotropic baseline that samples points isotropically around the concept centroid, without constraining them to the span of the recovered basis vectors, outperforms various linear and one-dimensional methods on Gemma-3-1B. This suggests that correctly locating the center of the concept geometry may be at least as important as recovering its specific basis directions, which provides indirect support for a prototype-based view of concepts over a direction-based view.
However, the isotropic baseline performs substantially worse on other models, suggesting that for larger or more capable models, the recovered basis directions also play an important role. The relative importance of offset versus basis varies by model, which means no single recommendation applies universally.
No Method Consistently Beats Prompting
The most sobering finding is that no activation-intervention method consistently outperforms prompting. On larger models like Gemma-3-27B and Llama-3.1-8B, prompting achieves MAxBench scores of 1.86-1.87, comparable to or exceeding the best activation-based methods. On Gemma-3-1B, prompting scores 0.98, which is lower than the best affine methods but competitive with one-dimensional methods.
This is consistent with prior findings on binary concepts, where prompting has been shown to outperform steering in many settings. The implication is that for the kinds of concepts evaluated here, simply instructing the model may be as effective as intervening on its internal representations. Activation-based steering remains valuable for cases where prompting is insufficient, but the gap between the two approaches is smaller than the interpretability community sometimes assumes.
Manifold Steering Is Competitive When Applicable
Spline fitting achieves the highest performance on the Days concept when the complete set of concept values is provided, reaching MAxBench scores of 1.83-1.93 across models. This makes sense: Days have a natural cyclic structure that a spline can capture. When some days are held out before fitting, performance drops but remains competitive with the best affine methods.
The limitation is that spline fitting requires ordered concept values and access to the complete sequence in its standard form. Extending it to unordered categorical concepts like Animals is non-trivial and left to future work. When the geometric assumptions of the method match the concept structure, manifold steering is powerful. When they do not, it is not applicable.
Unsupervised Methods Generalize Better
A surprising finding is that unsupervised methods produce more unseen concept instances during steering than supervised methods. Factor Analysis and MFA generate the largest number of instances not present in the training data. For example, on Gemma-3-1B, Factor Analysis produces 72 unseen instances for Animals, while ReFT-r1 produces only 6. This suggests that unsupervised methods learn more general geometric structures that extend beyond the specific examples they were trained on, while supervised methods may overfit to the training instances.
This has practical implications. If the goal of concept recovery is to enable steering that generalizes to novel instances of a concept, unsupervised methods may be preferable despite their lower scores on some metrics. The trade-off between specificity and generalization is real and concept-dependent.
The MAxBench Score and Metric Independence
The composite MAxBench score combines concept score, diversity score, fluency, and instruction following into a single number via harmonic mean. A meta-evaluation of metric correlations reveals that concept score is somewhat correlated with instance and category diversity (expected, since diversity can only be maximized when outputs contain concept-related values), and that concept score and instruction score are negatively correlated. When the target concept is injected more strongly, it becomes harder for the model to stay on topic with respect to the original instruction. Fluency scores have relatively weak correlations with other metrics.
This negative correlation between concept strength and instruction following is a fundamental tension in activation-based steering. Stronger interventions produce more concept-relevant outputs but at the cost of coherence. The framework captures this trade-off explicitly, which is useful for practitioners deciding how to deploy steering in real applications.
Limitations and What Comes Next
The framework evaluates concepts at fixed layers, chosen based on prior work showing that semantic information is well-formed at middle-to-late layers. Different layers might yield different results, and the framework does not currently evaluate layer selection as a variable. The concepts studied are relatively well-structured (animals, countries, days), and it is unclear how the methods would perform on messier, less categorical concepts.
The evaluation uses LLM judges, which introduces their own biases. The diversity score relies on an LLM to extract entities and assign category labels, which may miss instances or miscategorize them. Human evaluation would provide a useful complement, though at much greater cost.
The spline-fitting manifold method is restricted to ordered concepts. Many real-world concepts are unordered but structured (e.g., programming languages, political ideologies), and extending non-linear manifold methods to these settings remains an open problem.
What This Means for Interpretability Research
MAxBench demonstrates that the interpretability community's focus on one-dimensional concept representations is insufficient for the majority of real-world concepts. Multinomial concepts require richer geometries, and affine subspaces are a practical middle ground between the simplicity of linear methods and the complexity of non-linear manifolds. The finding that the offset matters more than the basis challenges the direction-centric view that has dominated the field.
The prompting result is perhaps the most important practical takeaway. If prompting achieves comparable or better performance than activation-based steering for many concepts, then the case for steering as a practical tool is weaker than often presented. Steering remains valuable for concepts that cannot be elicited through prompting, or for applications where you want to modify model behavior without changing the prompt. But for general concept elicitation, the overhead of activation intervention may not be justified.
The generalization finding for unsupervised methods suggests a productive research direction. If Factor Analysis and MFA learn geometric structures that extend beyond their training instances, they may be better suited for real-world applications where you cannot enumerate all instances of a concept. Combining unsupervised geometric recovery with supervised concept selection could yield methods that are both general and specific.