An Independent Researcher Flips a Language Model's Decision on Command
Language models make decisions at the token level. When a model decides that two numbers are equal, it commits to a specific sequence of next tokens. What happens if you reach into the model's internal state and nudge it to change its mind? Rayan Pal, an independent researcher, demonstrated exactly that on a 4-billion parameter model called PCCG-Qwen3-4B. The result: 40 out of 40 cases where the model was flipped from producing a "GO" continuation to producing an "EOS" termination, and 40 out of 40 cases in the opposite direction, with 640 control cases left untouched.
The work is published as an open repository on GitHub with full reproduction instructions, fitted lens data, and SHA-256 verification hashes. It uses Anthropic's Jacobian Lens, a tool that maps a model's internal activations into its vocabulary space, and applies a single fixed activation direction to alter the model's output at a precise moment in its computation.
How the Model Decides
PCCG-Qwen3-4B is a thinking-enabled model that performs digit comparison. Given two operands, it generates four comparison bits, closes its thinking block with, and then produces either GO (followed by EOS) if the operands are equal, or immediate EOS if they are unequal. The decision point is the first forward pass after the model closes its reasoning, where it selects the final-answer token.
Pal fitted a direction vector by computing the mean difference between activations at the reasoning-closure position for equal pairs versus unequal pairs. The fitting used 32 matched equality pairs, testing layers 24 through 29 and intervention strengths ranging from 0.5 to 2.0. The selected direction is a 2,560-dimensional vector applied at the output of block 29, added or subtracted at the exact token where the model closes its thinking block.
The intervention strength was set to 2.0, which means the direction is added at twice its fitted magnitude. The prompt, model weights, and generated reasoning prefix are all held fixed during intervention. Only the activation at that one point in the network changes.
The Jacobian Lens
The Jacobian Lens is an implementation from Anthropic that fits a set of matrices mapping activations from intermediate layers to the model's final layer. For this study, six 2,560 by 2,560 Jacobian matrices were fitted using 16 generated reasoning prefixes, covering source layers 24 through 29. The fitter uses a 64-token cap, excludes the first 16 positions in each prefix, and processes 16 output dimensions per backward pass.
Once fitted, the lens transports captured activations into the final-layer basis for vocabulary readouts. The fixed activation direction is projected into vocabulary space as well, which makes it possible to read off which tokens the intervention pushes toward or away from. The fitted lens, matrices, readouts, projections, and calibration records are all stored in the results/ directory.
What the Numbers Show
The confirmation measurement is the core result. Across 80 cases (40 equal pairs, 40 unequal pairs), the model generated the correct comparison trace and answer in every case. Unmodified replay also succeeded in all 80 cases, confirming the base model's competence. The intervention results were clean: 40 out of 40 GO-to-EOS flips, 40 out of 40 EOS-to-GO flips, and 640 out of 640 controls unchanged.
The controls are important. Pal tested five types: zero-vector sham, same-truth activation replacement, three norm-matched random vectors, and three norm-matched orthogonal vectors. Each control set contains all 80 confirmation cases. The fact that none of the controls changed anything while the fitted direction changed everything demonstrates that the effect is specific to the direction, not a side effect of adding a vector to the activation.
Why This Matters
The demonstration is narrow by design. It flips a single binary decision in a small model under controlled conditions. But it establishes something important about the mechanics of activation intervention: you can take a model that has already decided, reached the point where it selects its answer token, and change its decision by modifying one activation vector at one layer.
This has implications for model interpretability. If a model's decision can be flipped by a single direction at a specific layer, that direction encodes something about how the model represents the concept being tested. The Jacobian Lens makes it possible to read that representation in vocabulary space, connecting the internal activation to the tokens the model would produce.
It also has implications for model safety. If an external actor can identify the activation direction that controls a specific decision, they can potentially alter that decision without changing the model weights or the prompt. The intervention is applied at inference time, which makes it a runtime attack vector, not a weight-space modification. The practical difficulty of finding and applying the right direction in a larger model with more complex decisions is real, but the proof of concept is here.
Reproducibility
The repository includes verify.py, which checks package hashes and recounts confirmation records using only the Python standard library. reproduce.py provides full reproduction with model download requirements. All data is stored with SHA-256 hashes: the model identity, the fitted direction, and the Jacobian Lens all have published fingerprints. The project is Apache 2.0 licensed, and the vendored Jacobian Lens implementation retains its own license.
For developers working on model interpretability or mechanistic steering, the repository is a working example of how to apply and validate activation interventions end to end, from fitting a direction through Jacobian readouts to confirmation measurements with full control sets. The code is small, the model is small, and the results are unambiguous.