Why Language Agents Fail at Execution, Not Planning
Large language models can generate fluent text and solve many short-horizon reasoning problems, but these abilities do not reliably produce agentic behavior in interactive environments. The central problem is not what an agent knows, but how it coordinates fast action proposal, slower deliberation, memory, and execution-time monitoring. A common failure pattern in environments like ScienceWorld is that a reasonable high-level plan degrades at execution time: the agent repeats a stale action, proposes an action unavailable in the current state, or continues navigating despite no measurable progress. These failures occur at the interface between language generation and environment transition, not at the level of knowledge or reasoning.
João Meneses dos Santos and Arlindo L. Oliveira from Instituto Superior Técnico, Universidade de Lisboa, study this problem through a dual-process language-agent architecture. They extend SwiftSage, a system that separates Swift (a fast System 1-style action proposer) from Sage (a slower System 2-style planner), with two modular cognitive extensions. The Adaptive Memory Module (AMM) adds salience-gated episodic storage and trigger-driven retrieval. The Self-Reflection Module (SRM) adds bounded execution-time validation and corrective intervention. The key finding is that execution-time control is the dominant bottleneck in this setting, while episodic memory becomes most useful once the runtime loop is stabilized.
The Dual-Process Baseline and Its Limitations
SwiftSage already instantiates fast and slow thinking in ScienceWorld. Swift proposes local actions using an efficient model (Flan-T5-large), while Sage performs higher-level planning and grounding through an action buffer using a larger model (Qwen2.5-7B-Instruct-1M). This design improves efficiency over methods that query a large model at every timestep, but it leaves two important limitations in long-horizon tasks.
First, the agent lacks persistent episodic memory. It cannot selectively reuse salient prior experience across episodes. When the agent encounters a similar situation in a later trial, it starts from scratch. Second, it does not systematically validate actions immediately before execution or intervene when behavior stagnates. Actions that are plausible in language but invalid in the current environment consume steps without making progress.
The paper frames these as two distinct causal interfaces. AMM is informational: it changes what the controller can remember and reuse, but never chooses the final action. SRM is control-oriented: it changes what is allowed to reach the environment and when corrective actions are inserted. This separation makes the resulting ablation study interpretable as cooperation between evidence and control rather than as an opaque redesign of the baseline agent.
Adaptive Memory Module: Salience-Gated Episodic Storage
AMM has two hook families: a post-step write hook and pre-decision retrieval hooks. The write hook runs after the environment returns an observation and score transition. It receives the task, previous state, executed action, resulting observation, score change, and recent history, then builds a candidate episodic record. This record is stored only if a salience gate detects an informative transition: terminal success, positive score change, near-miss progress, explicit invalid-action feedback, or an avoidance-worthy failure.
Stored memories are compact semi-structured records, not raw transcripts. Each memory includes fields such as task, local state, recent context, executed action, resulting observation, score transition, and a type tag. The tag distinguishes success, near-miss, and avoidance-oriented memories. This representation improves retrieval targeting and makes prompt injection safer, because retrieved evidence is already concise, typed, and easy to filter or truncate.
Retrieval is trigger-driven. T1 corresponds to Swift failure: when Swift fails to produce a valid action, AMM retrieves related episodes and retries Swift with memory-conditioned context. T4 corresponds to System 2 planning: when the baseline invokes Sage and memory planning is enabled, AMM retrieves success and near-miss episodes for deliberative planning. T2 and T3 are stagnation and repeated-invalid-action hooks, but in the evaluated AMM-only configuration they do not directly modify model inputs. This conservative design avoids injecting weakly grounded negative evidence into the fast pathway.
Prompt augmentation is bounded and explicitly delimited. Swift receives only a small memory block (three retrieved episodes) in recovery mode, while Sage receives a slightly larger block (five episodes) because it is already the deliberative pathway. Prompts instruct the model to treat memories as hints rather than authority: current observations, inventory, admissible actions, and runtime constraints always dominate. Retrieved memories are deduplicated, filtered by operational type, and truncated before injection. If retrieval, formatting, or prompt-structure checks fail, execution falls back to the unmodified baseline prompt.
Self-Reflection Module: Bounded Execution-Time Control
SRM consists of three components: Gate-1 validation, post-step stagnation detection, and bounded Critic intervention.
Gate-1 is applied immediately before an action reaches the environment. It receives the proposed action, current valid-action set, recent state descriptors, and runtime constraints. It normalizes the action, checks admissibility, applies deterministic repair when the mismatch is minor and safe, and drops the action if it remains invalid or violates constraints. Gate-1 is source-agnostic: Swift actions, Sage-buffered actions, and Critic-generated actions all pass through the same pre-execution control surface.
After each executed step, SRM updates deterministic diagnostics over the recent trajectory. These diagnostics track repeated observations, repeated actions with no effect, invalid-action loops, excessive navigation without state diversity, and verb-level loops that do not change the effective state. When these signals cross a threshold, SRM emits a structured stagnation report summarizing the failure pattern, actions to avoid, and relevant runtime constraints.
The Critic is invoked only when stagnation is detected and safeguards permit a call. Its prompt contains the task, current state, recent trajectory, stagnation report, runtime constraints, and, in the full system, optional AMM evidence. The Critic must output a short executable action list (up to five actions). Its output is parsed, filtered, and inserted into the same FIFO buffer used by Sage; it is never executed directly. Buffered Critic actions still pass through Gate-1. Calls are bounded by a budget (three per episode in the main configuration) and cooldowns, making SRM a controlled repair mechanism rather than an always-on second planner.
Experimental Setup: Four Configurations on ScienceWorld
The evaluation uses ScienceWorld, an interactive text benchmark requiring agents to perform elementary science tasks through grounded sequential actions. Tasks 0-29 are used with up to ten natural-language variations per task, yielding up to 271 evaluation episodes per configuration. All configurations share the same environment interface, valid-action grounding, one env.step per timestep, Swift/Sage arbitration, and action-buffer execution.
Four configurations are compared under a shared runtime substrate: baseline (SwiftSage without extensions), baseline+AMM, baseline+SRM, and the full system (AMM+SRM). AMM-enabled configurations use a populated memory agent, initialized with a warm-up stage of two variations per task (60 episodes) that are excluded from all reported metrics. The protocol is a controlled ablation, not a claim of new ScienceWorld state-of-the-art.
Three primary metrics are reported: final score at termination, task success rate, and Steps@Succ (the mean number of steps conditioned on successful completion). Scores are first aggregated at the task level and then macro-averaged to avoid overweighting tasks with more variations. Tasks are also grouped by oracle trajectory length: Short (0 < Len <= 20), Medium (20 < Len <= 50), and Long (Len > 50).
Results: SRM Dominates, Memory Helps After Stabilization
The full system achieves the best aggregate performance with a mean final score of 64.62, success rate of 43.17%, and Steps@Succ of 19.33. Baseline+SRM is close behind at 64.33. Baseline+AMM reaches 53.83 compared with 51.67 for the baseline. Relative to the baseline, the full system improves mean final score by 25.1%, SRM alone by 24.5%, and AMM alone by 4.1%.
The grouped results refine this picture. On Short tasks, baseline+SRM performs best (81.50 versus 57.74 for baseline, a 41.2% improvement), suggesting that just-in-time execution filtering is especially valuable when trajectories are short and mistakes consume a large fraction of the budget. On Medium and Long tasks, the full system obtains the highest score, although on Long tasks it is nearly tied with SRM alone. This indicates that memory is most useful in the harder regimes where prior episodes can support recovery or planning, but only after execution has been stabilized.
Success rates show the same trend. Success increases from 23.99% in the baseline to 26.94% with AMM, 41.33% with SRM, and 43.17% in the full system. Successful trajectories also become shorter in SRM-enabled settings: Steps@Succ falls from 24.48 in the baseline to 19.76 with SRM and 19.33 in the full system. The strongest configurations do not merely collect more partial reward; they complete more episodes and require fewer steps when they succeed.
Mechanism-Level Logs: What Actually Changed
Mechanism-level logs explain why the improvements happen. AMM writes 4.48 memories per episode in the AMM-only configuration and 4.89 in the full system. Most writes are near-miss records, meaning the memory store primarily captures partial progress rather than only terminal successes. However, memory is used less frequently once SRM is enabled: T1 triggers fall from 17.13 to 12.15 per episode, T4 triggers from 19.13 to 11.77, Swift injections from 30.81 to 23.03, and Sage injections from 19.13 to 11.77. The full system writes slightly more experience but needs fewer memory-conditioned recovery attempts, consistent with SRM reducing the number of unstable trajectories.
SRM shows the complementary profile. Gate-1 drops are much more frequent than Critic calls: 20.03 per episode in SRM-only and 17.48 in the full system, mostly because proposed actions are not in the current valid-action set. Effective Critic calls remain close to one per episode (1.35 in SRM-only, 1.29 in the full system). This implies that SRM's main contribution is not frequent reflective replanning, but continuous filtering of invalid or stale actions before they consume environment steps.
Controller usage further supports this interpretation. Direct Swift-executed steps increase from 44.08% in the baseline to 49.16% with AMM, 57.13% with SRM, and 62.01% in the full system. Buffer-executed steps fall from 55.92% to 37.99%, and System 2 call rates fall from 24.6 per episode in the baseline to 14.1 in the full system. Better performance comes from making execution more reliable, not from more deliberation. Fewer invalid actions reach the environment and fewer costly System 2 interventions are required.
Sensitivity Analyses: Reflection Must Be Bounded
Several sensitivity analyses reinforce the main conclusions. Evaluating AMM with populated memory improves score from 51.47 in a weaker-memory regime to 53.83, confirming that memory benefits from a mature episodic store. Increasing the SRM Critic budget from three to six calls worsens performance, reducing score from 64.33 to 52.95. This is the strongest warning against unbounded reflection: expanding reflective intervention degrades performance. Disabling Swift-level T1 memory injection in the full system lowers score from 64.62 to 63.33, with the strongest negative effect on Long tasks.
The Critic-budget result is particularly important. It shows that more reflection is not automatically better. The Critic is powerful but risky: if used too often, it can interrupt otherwise valid trajectories or replace local action selection with unnecessary deliberation. Gate-1 should remain the cheap always-on safeguard, while the Critic should be reserved for trajectories that exhibit explicit stagnation.
Limitations and What This Means for Agent Design
The paper acknowledges several limitations. AMM currently relies on compact semi-structured episodic records stored in an external memory agent (Letta archival memory). This design is appropriate for controlled integration but likely constrains the standalone value of memory: stored traces are useful as situated evidence, yet their representation, retrieval ranking, and conversion into concrete action improvements remain simple. The system does not implement a mature semantic consolidation layer that abstracts across episodes into reusable skills or task-general regularities. From a Complementary Learning Systems perspective, the current implementation covers the rapid episodic side more than the slower semantic side.
SRM's control layer is intentionally interpretable and rule-bounded, but several choices are hand-crafted: stagnation thresholds, action-filtering rules, fixed Critic budgets, and deterministic repair policies. Gate-1 is effective as a conservative safeguard, but its repairs are shallow when an invalid action cannot be cleanly normalized or mapped to an admissible alternative. Learned verifiers, confidence-aware escalation, richer affordance models, and adaptive reflective budgets may improve this component while preserving the boundedness that proved important.
The evaluation is restricted to ScienceWorld, a text-based simulated science environment. The results may not transfer directly to other interactive environments, multimodal settings, robotic action spaces, or different model backbones. The analysis also remains partly correlational at the prompt level: mechanism-level logs show when memories are written and retrieved, and when SRM drops actions or invokes the Critic, but do not fully isolate every causal prompt-level factor behind individual successes or failures.
The broader implication for agent design is precise. Cognitively inspired extensions are most useful when inserted at the right causal interface. Adding memory is not sufficient if the agent still executes invalid or stale actions. Adding reflection is harmful when it is unbounded. In this setting, robust interactive behavior emerges from a disciplined composition: memory provides evidence, reflection controls execution, and the baseline fast/slow controller remains the shared substrate. The contribution is not that memory and reflection should be added everywhere, but that each mechanism should be placed at a causal interface that matches its role.
Read the paper on arXiv