Large language models are routinely fine-tuned, shared, and downloaded from third parties. The model you deploy may carry a hidden backdoor that behaves normally on every benign query but switches to attacker-controlled behavior when a secret trigger appears. Pre-deployment auditing catches some of these, but a service that receives new checkpoints, adapters, or updates regularly cannot re-run a full audit after every change. Runtime monitoring fills this gap, but existing detectors either assume knowledge of the trigger form, which fails on stealthy attacks, or require extra model computation such as input perturbations or an additional generation pass. In a latency-sensitive serving stack, that cost is a dealbreaker.
Speculative decoding already computes the signal you need
SpecGuard starts from an observation about how modern LLM serving already works. Speculative decoding speeds up inference by having a small draft model propose K candidate tokens, then a larger target model verifies them in parallel. Each draft token is accepted when the target agrees and rejected otherwise. This accept/reject stream is normally used only for speed. But it also measures draft-target agreement, and that measurement is exactly the signal you need to detect backdoor activation.
When a backdoor is triggered, the target model shifts toward the attacker's behavior. The clean draft model does not predict this shift, so the target starts rejecting draft tokens at a much higher rate. The query's acceptance rate drops sharply. The paper formalizes this with a clean mathematical connection: the local acceptance rate equals one minus the total variation distance between the target and draft distributions. A backdoor that moves the target's distribution away from the draft's distribution directly reduces the acceptance rate.
The detection gap, the difference between clean and triggered acceptance rates, satisfies a lower bound: it is at least the total variation distance induced by the backdoor minus twice the normal mismatch between the clean target and the draft. In other words, the signal appears whenever the behavioral shift from the backdoor dominates the normal draft-target disagreement. And here is the key constraint on adaptive attackers: Theorem 2 in the paper proves a potency-stealth tradeoff. A more potent backdoor (higher attack success rate) creates a larger behavioral shift, making it more detectable. An attacker who suppresses the acceptance-rate signal must reduce probability on tokens the clean draft does not expect, which weakens the attack itself.
Four backdoor families, three model families, up to 32B parameters
The evaluation covers four distinct backdoor types. BadNet uses a lexical trigger (the token "cf" inserted at a random position) with a fixed malicious response. Syntactic backdoors use a structural trigger (a subordinate-clause paraphrase) that is invisible to token-level filters. Sleeper agents activate on contextual triggers (a date in the system prompt). Instruction backdoors embed the trigger in the system prompt itself, mimicking jailbreak-style attacks.
On LLaMA-3B with a 1B draft model and K=4 draft tokens, SpecGuard achieves per-query AUROC above 0.92 on all four backdoor types from a single query. BadNet: 0.946. Syntactic: 0.956. Sleeper Agent: 0.929. Instruction: 0.974. At TPR=0.95, false positive rates range from 3.3% to 9.7%. These are not aggregate statistics over a dataset. They are per-query detection scores, meaning each individual triggered query is classified correctly with high probability.
The method generalizes across model families and scales. On Qwen3, pairing a 1.7B draft with targets up to 32B, per-query AUROC stays above 0.96 in every tested configuration, reaching 0.996 with a 4B target. On Gemma 3, pairing a 1B draft with targets up to 27B, AUROC ranges from 0.950 to 0.989. The signal does not degrade with model scale. If anything, larger targets produce larger behavioral shifts when triggered, strengthening the detection gap.
Stealthy payloads that evade input-level filters
The real test is whether SpecGuard catches attacks that look benign on the surface. The authors tested four payload variants on the BadNet trigger. A phishing paragraph (95 tokens with a malicious URL) achieves AUROC 0.980. Unsafe security advice (claiming certificate verification is unnecessary under HTTPS) achieves 0.958. Insecure code (Python with verify=False that looks normal to a human reviewer) achieves 0.927. All three payloads are semantically different from a fixed canary string, yet all three produce strong acceptance-rate drops because the target model's distribution shifts away from what the clean draft predicts.
The hardest case is topic steering, where the backdoor promotes a chosen brand in technology conversations with no fixed response string. The attack success rate drops to 0.44, and the acceptance-rate signal becomes weak (AUROC 0.572). This is consistent with the potency-stealth tradeoff: a subtle backdoor that barely changes the target's distribution is inherently harder to detect. But even here, SpecGuard outperforms input-level baselines. ONION and perplexity-based detection both score near chance (AUROC 0.51), and CleanGen, which requires a full extra generation, reaches only 0.66.
On hard and diverse clean traffic pooled from ShareGPT, GSM8K, HumanEval, and XSum, SpecGuard's per-query AUROC averages 0.969. XSum is the hardest domain at 0.874 because its clean acceptance distribution is wider and closer to the triggered range, but aggregation recovers near-perfect separation.
Why existing detectors cost more and detect less
The comparison with baselines is where SpecGuard's practical value becomes clear. CleanGen requires running a reference model to generate clean completions and comparing them against the target's output, adding one full extra model forward pass per query. On the four backdoor types, CleanGen achieves AUROC 0.935, 0.976, 0.935, and 0.989. SpecGuard achieves 0.946, 0.956, 0.929, and 0.974, competitive across the board, while adding zero extra model computation.
ONION and perplexity-based detection require running the target model on perturbed or truncated inputs. ONION is particularly weak on syntactic backdoors (AUROC 0.226, worse than random) because the trigger is not a removable token. Perplexity detection fails on sleeper agents (AUROC 0.803) because the triggered response is not unusually perplexing. SpecGuard avoids these blind spots because it watches behavioral divergence in the output distribution, not statistical anomalies in the input.
Adaptive attackers hit a fundamental tradeoff
The paper tests two adaptive attack strategies. The first adds KL divergence to the base model as a regularization term during fine-tuning, constraining the backdoored model to stay close to the original distribution on triggered inputs. At lambda=0.02, attack success remains 1.00 but per-query AUROC drops from 0.953 to 0.819. At lambda=0.1, attack success drops to 0.86. At lambda=0.5, attack success collapses to 0.39. The attacker cannot maintain both high attack success and low detectability.
The second strategy is draft-aware: the attacker directly optimizes acceptance rate during fine-tuning by penalizing KL divergence between the backdoored target and the clean draft on triggered inputs. At lambda=0.2, attack success is 1.00 but AUROC is still 0.920. At lambda=0.5, attack success drops to 0.07. At lambda=1.0, the triggered acceptance rate reaches 0.618, close to the clean mean of 0.657, but the attack succeeds on only 1% of triggered inputs. The attacker has effectively disabled their own backdoor to avoid detection.
Deployment hardening for real-world conditions
The paper studies several deployment challenges and provides concrete mitigations. If the draft model is also poisoned but with a different trigger, the signal is preserved (AUROC 0.980). If the draft has the same trigger but a different response, the signal is amplified (AUROC 0.997). If the draft has the same trigger and same response (matched backdoor), the acceptance rate reverses direction, increasing instead of dropping. The mitigation is two-sided scoring: alarm on any large deviation from the clean baseline, not just drops.
Windowed aggregation dramatically improves separation. Per-query AUROC of 0.929-0.974 on LLaMA-3B backdoors improves to 0.991-0.996 at window size 2 and reaches 1.000 at window size 5. Even an adaptive attack at lambda=0.02 (per-query AUROC 0.819) recovers to 0.986 at W=10 and 1.000 at W=20. For sparse attacks where only a fraction of queries carry the trigger, larger windows are needed: 25% triggered queries saturate by W=20, 5% need W=100.
Response padding, where the attacker appends benign text after the malicious response to dilute the per-query mean, is handled by using early-token statistics. The acceptance-rate drop is front-loaded: triggered acceptance collapses in the first few tokens and recovers once the draft is conditioned on the attacker's response. Using the first 8 evaluated positions (about 2 verification rounds at K=4) preserves AUROC 0.89 even when 200 tokens of benign padding follow, compared to 0.54 for the whole-generation mean.
Self-drafting deployments, where the target model's own multi-token-prediction head serves as the draft, also work. On Gemma-4-31B, BadNet achieves AUROC 0.916 per-query and 0.997 at W=5. The instruction backdoor is the exception: the drafting head partially follows the triggered continuation, causing triggered acceptance to increase. Two-sided scoring recovers AUROC 0.763 at W=5.
What this means for LLM deployment
SpecGuard turns a latency-optimization technique into a security tool at no additional cost. If your serving stack already uses speculative decoding, you are already computing the signal. You just need to log the accept/reject decisions and compare the acceptance rate against a clean baseline. The detector runs during generation, so you can buffer responses briefly, score the early trace, and release or quarantine without a second model call.
The practical constraints are straightforward. The draft model must be clean, which the operator can enforce by pinning a known-good checkpoint. The clean baseline must be calibrated to the input distribution, which rolling aggregation handles. Short generations produce noisier signals, but the false positives are concentrated in outputs of 2 tokens or fewer, which are rarely the queries you care about protecting. And the potency-stealth tradeoff means that any backdoor strong enough to be useful to an attacker is strong enough to be detected. Read the paper on arXiv