Prompt injection remains a critical security challenge for LLM deployments that combine trusted system instructions with untrusted user data. As models gain tool-use capabilities and agentic behavior, an injected instruction hidden in visible text can override developer-intended guardrails. Prior defenses have leaned on finetuning or coarse architectural modifications, but these often come at the cost of reduced model utility or limited adaptability to new attack forms. This paper introduces a different approach: instead of retraining the full model or redesigning its architecture, it intervenes within the attention mechanism itself.
How learned per-head attention gates enforce instruction-data separation
The core idea of HoS is to attach a small learned gate to each attention head. During inference, each gate evaluates whether a given token belongs to the trusted instruction stream or the untrusted user data stream. Heads that are prone to following injected directions receive a bias toward attending to trusted tokens, while heads that benefit from surveying untrusted content keep broader access. The gates are trained offline via a meta-learning process that anticipates prompt injection patterns, then applied as a lightweight inference‑time modification. This means the base model weights stay unchanged; only the attention redistribution happens at runtime. The result is that the model can still reason over user‑provided data—summarizing, extracting facts, or calling tools—but it no longer treats injected text as authoritative directive.
Resilience to black‑box and white‑box adaptive attacks
The evaluation pairs HoS against two strong baselines: SecAlign++, which enforces instruction-data separation through finetuning, and ASIDE, which embeds the same separation principle directly into the model architecture. Attackers use a black‑box RL‑Hammer method and a white‑box variant called Claudini, both strengthened through automated search. Against the RL‑Hammer attack, the black‑box ASR reaches 97 % on the unmodified GPT‑OSS‑20B and 85 % on SecAlign, but drops to just 10 % when HoS is active. Claudini, the state‑of‑the‑art white‑box attack, finds that reaching an equivalent attack success rate on HoS requires roughly three times the FLOP count compared with GPT‑OSS‑20B or SecAlign. This FLOP penalty reflects the gate‑induced redistribution: the attacker must overcome per‑head gating that constantly re‑weights attention toward the trusted stream.
Utility preservation on standard benchmarks
A defense that hardens security at the expense of everyday usefulness is of limited practical value. The authors report that HoS preserves base‑model performance across a suite of multiple‑choice and prompting benchmarks. SecAlign and ASIDE, by contrast, show measurable drops in few‑shot settings, particularly on tasks that require reasoning over mixed trusted‑untrusted inputs. HoS’s gate mechanism is selective enough to keep the model’s latent knowledge accessible while still blocking instruction-following from untrusted tokens. The authors also note that HoS matches the top‑tier performance on standard agentic prompt injection benchmarks, where the model must decide whether to execute a tool call based on a prompt that may contain hidden directives.
Agentic benchmark security‑utility trade‑off
On agentic benchmarks that feature static (non‑adaptive) prompt injection, HoS and SecAlign achieve comparable robustness, and both substantially improve the security‑utility trade‑off relative to the unprotected base model. ASIDE lags behind, scoring only 10 % utility on tool‑use evaluations. The gap stems from ASIDE’s architectural constraints, which restrict the model’s ability to flexibly invoke functions when the separation between instruction and data is baked into the model’s connectivity rather than applied dynamically at inference time. HoS’s runtime‑only gating avoids these structural bottlenecks, making it particularly suited for deployment environments that rely heavily on tool use.
Limitations and trade‑offs
The paper acknowledges that HoS introduces a new inference dependency: the gates must be present and operational for every token‑processing step. If a deployment strips the gates for compatibility reasons, the security benefit vanishes instantly. Additionally, the gate training procedure requires access to a diverse set of prompt injection examples; in settings where such data is scarce, the gates may not generalize to novel attack forms. The authors also note that HoS’s strength lies in redirecting attention; it does not address other attack vectors such as parameter‑level extraction or physical‑world prompt delivery, and practitioners should combine it with other hardening layers when those risks are present.
What it means for working developers
For teams building LLM‑backed products that accept user‑generated text alongside system prompts, HoS offers a path to harden the model without a full retraining cycle or a switch to a different model family. The gate can be packaged as an inference‑time middleware layer, meaning existing serving pipelines need only a small code change to plug in the learned weights. Because the gates are head‑specific, they also expose a degree of interpretability: engineers can inspect which heads received the strongest injection‑blocking bias and monitor those heads for unexpected shifts during deployment. In practice, this means a deployed chatbot or agent can remain responsive to legitimate user commands while discarding or down‑weighting concealed instructions that attempt to override developer‑set behavior.