Nathan Egbuna published a preprint describing LaTense, a method that fixes a fundamental problem with how language models are steered at inference time. The approach eliminates the text-looping and reasoning degradation that plague standard activation steering, while cutting compute requirements by 4.6x compared to multi-path sampling methods.
The Problem With Static Steering
Activation addition works by adding a fixed steering vector to a model's hidden states during inference. The idea is to push model behavior in a desired direction without retraining. The problem is that a constant steering magnitude applies the same pressure regardless of what the model is actually doing at a given token. In heavily optimized architectures like Gemma 2 and Qwen 2.5, this rigid pressure overrides delicate factual and syntax circuits, causing what Egbuna calls "latent collapse" where the model either loses its reasoning ability or gets trapped in degenerate text loops. In benchmarks, static steering produced repetition rates as high as 38.07% on Qwen-2.5-7B-Instruct running MATH-500.
The root cause is a mismatch between a static coefficient and a dynamic model state. Tokens that are already well-aligned with the steering direction get over-corrected, while tokens that have drifted far from the intended trajectory get insufficient correction.
How LaTense Adjusts Per Token
LaTense solves this with two geometric gates applied at each target layer. The first is alignment attenuation: a cosine penalty that reduces steering pressure when the hidden state is already aligned with the reasoning vector. When the angle between the two is small, the intervention attenuates toward zero to preserve the factual circuits the model is already using correctly. The second is manifold-consistency scaling: a norm ratio that prevents the hidden state from being pushed out of its natural distribution. The combined intervention at each layer is a single closed-form expression that scales the steering vector proportionally to both the misalignment angle and the relative magnitude of the hidden state.
The practical effect is that LaTense applies restorative pressure only when the model actually needs it, and backs off when the model is already on the right trajectory.
Eliminating Text Loops Across Benchmarks
Egbuna evaluated LaTense on three tasks: strategic commonsense reasoning (StrategyQA), formal multi-step mathematics (MATH-500), and open-domain factual retrieval (TriviaQA). The results show LaTense eliminates text-looping entirely across all tested model families. On Gemma 2 9B IT running StrategyQA, static steering produced a 3.35% repetition rate; LaTense reduced that to 0.00% while matching or beating the base model's accuracy. On Qwen-2.5-7B-Instruct running MATH-500, where static steering produced a 38.07% repetition rate, LaTense again hit 0.00%.
Accuracy improvements were modest but consistent where static steering had degraded performance. On Gemma 2 9B IT running StrategyQA, LaTense matched static steering's 65.60% accuracy but eliminated the looping. On Llama 3.1 8B Instruct running the same task, LaTense improved accuracy from 71.75% (static) to 73.65%, approaching the 74.20% achieved by Self-Consistency with k=5 sampling.
The Compute Advantage
The most significant practical benefit is efficiency. Self-Consistency and Best-of-N methods generate multiple full candidate chains and aggregate majority votes, which multiplies compute cost. LaTense runs on a single forward pass with under 1.2% overhead from the geometric gating calculations. This reduces total generated tokens by 4.6x while remaining competitive with multi-path sampling on accuracy. For teams deploying steering at scale, the difference between one forward pass and five is meaningful in both latency and cost.
The approach is training-free and applicable to any dense architecture where activation addition already works. The geometric gating requires no additional parameters or fine-tuning, which makes it a straightforward drop-in replacement for static steering in existing inference pipelines.