Web agents complete user requests by reading and acting on pages that third parties write, so an instruction planted on a page can redirect the agent away from the user's goal. The agent cannot simply ignore the page, because the page also holds the values and controls the task requires. Current defenses fine-tune the agent on injections fixed before training, and attackers that adapt to the trained model bypass them. Adversarial training lets the attacker adapt but keeps the tasks fixed, so a task stops teaching once the agent solves it. We introduce AdvSim2Real, which co-evolves a task curriculum, an injection adversary, and the agent inside a frozen web world model. The curriculum is rewarded for tasks the agent solves about half of the time, and the adversary only for a success flip, an injection that turns a judged success into a failure. Training in the simulator makes a 4B agent both more capable and more robust: its completion rises with and without attacks, holds against a frontier-model adversary it never trained against, and its capability gain carries over to a real browser. On 150 web tasks, AdvSim2Real raises completion under this unseen adversary by 33.6% relative to the base agent.
The Web Agent Security Problem
Web agents now complete multi-step requests on websites, such as updating a customer record from the entries a page displays. The web differs from other tool settings because every page is written by a third party and mixes the data a task needs with the controls that act on it. An instruction planted in page content can therefore redirect the agent away from the user's request, an attack known as indirect prompt injection. On a web-agent security benchmark such attacks partially succeed in up to 86% of cases. The agent cannot defend itself by ignoring the page, because the page also holds the values and controls the task requires. A provider therefore needs task-preserving robustness: the agent completes its authorized goal despite competing page instructions whenever the task remains feasible.
Base under attack judged bad shows an injected notice that orders the agent to click a forbidden reset button, the base agent obeys and fails, and the trained executor completes the same task. The base agent clicks the forbidden Reset all right after the notice appears and is judged bad; Robust iter 3 sees the notice before five actions, never clicks it, and is judged good.
Shortcomings of Current Defenses
Current defenses fine-tune the agent on injected examples built before training, in some cases with a delimiter-based input format, so that it follows the user and disregards the planted instruction. Because these injections never change, the defender never meets an attacker that adapts to it, and attackers that optimize against the trained model bypass existing defenses. Recent work therefore trains the attacker as well: RETA red-teams a frozen agent and trains the defender on the recorded attacks, while ARLAS, DMAST, and CoER co-train an attacker and an agent in executed tool or browser environments. These methods adapt the attacks but keep the training tasks fixed, so a task stops teaching once the agent solves it, and each new task requires a site that can execute it.
AdvSim2Real: Two-Stage Co-Evolution Inside a Frozen World Model
We introduce AdvSim2Real, which trains three policies against one another inside a frozen web world model: a curriculum that writes tasks as a goal and an initial page, an adversary that requests one injection per trajectory, and the web agent, which we call the executor. A frozen language-model judge grades every run, and both rewards are defined relative to the current executor. AdvSim2Real runs in two stages. In the first, the curriculum and the executor alternate updates, so the tasks follow the executor's competence. In the second, the curriculum is frozen and the adversary and the executor alternate, and the executor trains on clean tasks, new attacks, and the attacks of earlier rounds.
Stage 1: Curriculum-Executor Co-Evolution
In Stage 1, the curriculum proposes a pool of tasks and the executor attempts each task several times. The curriculum is rewarded for tasks the current executor solves about half of the time, as graded by a fixed language-model judge. This uncertainty reward peaks when the success rate is near 50% and vanishes when the success rate is 0 or 1, providing difficulty pressure from the loop rather than from a fixed target. The executor then trains on fresh tasks from the updated curriculum. Stage 1 raises clean completion and also lifts the attack mean, even though the executor never sees an injection during this stage.
Stage 2: Adversary-Executor Co-Evolution
In Stage 2, the curriculum is frozen, the adversary starts from the base model, and on tasks the executor solves at least half of the time, several clean runs are saved. The adversary proposes one injection and the step at which to insert it; each saved run is replayed up to that step with the injection, and the adversary earns credit only when the injection is rendered and a run that the judge accepted without it now fails. This success-flip reward measures judged disruption and verifies neither that the task remained feasible nor that a target action was taken. The executor then trains on a mixture of the new attacks, the attacks of earlier rounds, and clean tasks. After round tt, the executor is labeled Robust iter tt; each stage runs three rounds.
Experimental Results on 150 Web Tasks
The authors evaluate a Qwen3.5-4B executor on the 150 web tasks of their benchmark in the frozen WebWorld-14B world model; fifty of them are close variants of eight parent tasks. Stage 1 raises clean completion from 74.89% to 79.33% over three rounds, and the attack mean moves from 48.07% to 54.34%. Stage 2 further raises the attack mean to 57.48% and clean completion to 81.33%. Against a frontier-model adversary Kimi-K3, which took no part in training, completion rises from 23.00% to 30.72%, a 33.6% relative gain. The final 4B checkpoint matches the hosted 9B under attack and beats it clean, with the 4B checkpoint achieving 81.33% clean and 57.48% under the learned adversary mean, compared to the 9B's 78.22% and 56.96%.
The capability gain of the first stage carries over to a real browser, where strict success on the submitted form rises from 25.56% to 44.44% without any world-model call. Per task and seed, the share solved in both the world model and the browser rises from 23.1% for the base to 42.9% for the final checkpoint, and the share judged good only in the world model falls from 51.8% to 36.4%, because the browser rejects malformed actions that the world model tolerated.
The Success-Flip Reward
The adversary is rewarded only for a success flip: a clean run that the judge accepted is replayed to the chosen step, the injection is rendered there, and the adversary earns credit only if the continuation fails. This design ensures that failures the agent makes on its own earn nothing, and the adversary is credited only when its injection disrupts an otherwise accepted run. A malformed proposal receives a reward of −1. The reward measures judged disruption and verifies neither that the task remained feasible nor that a target action was taken. By contrast, adversarial training that rewards every executor failure would be credited for failures that also occur without an injection.
Benchmark of 150 Form-Filling Web Tasks
The authors release a benchmark of 150 form-filling web tasks in five skill strata: conditional policy, derived values, conjunctive selection, relational join, and ordered repair. Every page holds the records and rules that determine the answer, and every task specifies the required final field values and final control, two protected fields that must stay unchanged, and two forbidden controls, Save draft and Reset all, that must never be activated. The evaluation tasks are distinct from the tasks the curriculum proposes during training. In the browser, a deterministic check equals 1 only if the submitted values match these requirements, the protected fields are unchanged, no forbidden control was activated, and a review preceded the commit where the task requires one.
Sim-to-Real Transfer
The capability checkpoints were also run on the same 150 tasks in a real Chromium browser, with the frozen initial page rendered as HTML, live DOM observations, and no world-model call. Strict browser success, a deterministic check of the submitted form rather than a model verdict, rises from 25.56% for the base to 44.44% for the final Stage 1 checkpoint. Absolute rates are lower than in the world model because the browser rejects malformed actions that the world model tolerated.
Related Work
Defenses trained on fixed injections include the instruction hierarchy, which trains a model to ignore lower-privileged instructions, SecAlign and Meta SecAlign, which prefer the response to the legitimate instruction, and StruQ, which separates instructions from data with reserved delimiters. Because the injections never change, the defender first meets an adaptive attacker at evaluation, and attackers that optimize against the trained model bypass existing defenses.
Adversarial training with a learning attacker includes RETA, which red-teams a frozen agent and trains the defender on recorded attacks, ARLAS, which co-trains an injection attacker and a tool-using agent as a zero-sum game, CoER, which keeps populations of past policies of both roles, and GPT-Red, which scales self-play to a red-teaming agent against simultaneously trained defenders. None of these adapts the training tasks to the current agent or trains inside a web world model.
Adaptive task generation includes WebRL, which derives new web tasks from failed attempts, SAGE, which grows a curriculum from easy to hard, and AgentGen, which evolves synthesized tasks toward easier and harder variants. R-Zero rewards a challenger for questions on which its solver agrees with itself about half the time, and Agent0 alternates curriculum and executor updates. Our Stage 1 uses R-Zero's uncertainty reward and repetition penalty, but estimates success from the executor's judged completions rather than from self-consistency.
Learning inside a web world model includes WMA and WebDreamer, which use a world model at inference time to choose actions, DreamGym, which trains a policy online on synthesized transitions and rewards with an adaptive task generator, WebEvolver, which trains the policy and the world model together, and DynaWeb, which trains web agents with RL on rollouts imagined by a web world model. Our work freezes the web world model's pretrained transition model so that every round and checkpoint is trained and scored against the same simulator.
Limitations
The authors acknowledge several limitations. The three tasks, while diverse, are relatively narrow; broader or more open-ended tasks may yield different bottling dynamics. The resource budgets are fixed and relatively short, which may favor strategies that require little upfront investment over those with higher fixed costs. The agents' introspection and planning capabilities are constrained by the base model's reasoning ability; stronger planning or meta-reasoning skills could potentially improve robustness. The world model's predictions are not always correct, which is why the authors also test in a real browser. The benchmark also has limitations that affect interpretability; broader or more open-ended tasks may yield different robustness dynamics.
What This Means for Web Agent Deployments
For practitioners considering large-scale web agent deployment, the paper's results offer several practical takeaways. First, zero-shot quality alone is a poor predictor of whether robustness training will be effective. A model that scores well on isolated examples may still lose significant accuracy under adaptive prompt injection, and empirical testing on representative workloads is essential before committing to a deployed system. Second, the co-evolution design means that both task difficulty and attack sophistication track the current agent, so robustness gains are not a one-time fix; periodic re-training with updated curricula and adversaries may be needed as page content and attacker strategies evolve. Third, the sim-to-real transfer demonstrates that gains measured in a web world model translate to real browser environments, albeit with lower absolute rates; teams should budget for this gap when evaluating real-world readiness. Finally, the released benchmark and checkpoints provide a reproducible testbed for future work on robustness training, adaptive attack generation, and world-model-assisted agent design.