Robot foundation models provide strong visuomotor control, yet their performance can degrade when object positions or task instructions change. Further improvements often require post-training on substantial robot data, which can be costly to collect through methods such as teleoperation. This practical constraint creates a barrier for deploying adaptable robot systems in settings where task specifications shift frequently. Standard fine-tuning on new data is expensive and risks interfering with previously learned behaviors. The research community has explored agentic frameworks that operate around the model, but existing self-evolving systems use robot trials inefficiently when deciding which code and skill modifications to pursue. This paper introduces EmbodiedRSI, a self-evolving agentic system that autonomously decides where to explore next and turns the resulting physical interaction into improved code and skills.
Why Robot Systems Need Continual Adaptation
Modern robot foundation models achieve impressive visuomotor control when deployed in simulation or structured environments. These models are typically pre-trained on large-scale datasets that cover a broad distribution of scenes, objects, and task structures. However, performance degrades when the deployed system encounters object configurations or instruction phrasings that differ from the training distribution. The common remedy is continued training on task-specific robot data, but collecting such data through teleoperation or real robot interaction is time-consuming and expensive. In many practical settings—home assistants, laboratory equipment, collaborative cells—the cost of data collection limits how frequently a system can be updated.
Agentic frameworks have been proposed as a way to adapt around fixed model parameters. These systems select tools, invoke skills, or revise code on each decision step, effectively extending the model's capabilities without changing its weights. However, many current self-evolving harnesses decide which code path or skill to try next using heuristics or simple scoring rules. They lack a principled method for choosing experiments that will most improve future performance, leading to inefficient use of limited robot trial budget. This paper addresses that gap by introducing a framework that explicitly reasons about the value of information before acting.
Fast-Slow Dual-System Architecture
EmbodiedRSI realizes active continual learning through a Fast-Slow Dual-System Architecture. The Fast System operates on a short timescale, selecting and executing code and skill hypotheses for the current trial. The Slow System operates on a longer timescale, building hierarchical memory of past trials and evaluating which memories are most valuable for improving the Fast System. This division mirrors computational patterns observed in biological cognition, where rapid, model-free actions are guided by slower, model-based learning systems.
In EmbodiedRSI, the Fast System maintains a Hypothesis Graph—a compact data structure that stores competing code and skill hypotheses generated from recent physical interactions. Each node in the graph represents a distinct program or motor primitive, and edges encode contextual relationships (e.g., "if object A is left of object B, try skill C"). The graph is incrementally updated as new trials produce new hypotheses; old hypotheses that have not been selected for several rounds are pruned. This structure allows the system to maintain diversity of behavior while keeping the representation tractable.
Value-of-Information Experiment Selection
A central problem for any self-evolving system is deciding what to try next. EmbodiedRSI addresses this with a Value-of-Information (VoI) approach: before each physical experiment, the system evaluates which candidate hypothesis resolution would most improve expected future performance. The VoI computation considers two factors: (1) the distinctness of the candidate hypothesis from currently active ones, and (2) the projected impact on the Fast System's success probability if the hypothesis were confirmed or rejected. Candidate experiments that maximally distinguish between competing hypotheses—and that are expected to shift the Fast System's behavior significantly—are prioritized.
After a physical trial executes, the outcome (success/failure along with intermediate state measurements) is fed back into the Hypothesis Graph. If a hypothesis accurately predicted the outcome, its edge weights are strengthened; if it failed, its weight is weakened or it is temporarily suppressed. This update rule ensures that the graph reflects the accumulating evidence from real robot experience.
Code-Skill Co-Evolution
The outcomes from VoI-selected experiments guide a Code-Skill Co-Evolution process. When a hypothesis is confirmed through successful trial execution, its associated code fragment or skill parameterization is extracted and added to the system's reusable library. When a hypothesis is disconfirmed, the system discards the ineffective code path and generates alternative variants via a small language model operator that mutates function arguments, loop bounds, or sensor thresholds. This co-evolutionary dynamic means that the library of available skills grows toward those that are demonstrably useful for the task distribution encountered so far, while dead-code paths are regularly purged.
Hierarchical Memory and Reward-Grounded Learning
The Slow System builds Hierarchical Memory across rounds of robot operation. Memory is organized into layers: lower layers store raw sensorimotor tuples (e.g., "grasp succeeded when gripper width was 5cm"), intermediate layers store abstract task progress markers (e.g., "object moved from zone X to zone Y"), and higher layers store relational knowledge about object affordances ("red cylinders are easier to grasp than blue cubes"). This hierarchy allows the system to retrieve relevant knowledge at the appropriate granularity depending on the current task demand.
Reward-Grounded Memory Learning selects effective memory according to their value for later Fast-System improvement. Each memory entry is tagged with a reward signal reflecting the trial outcome. The Slow System maintains a running estimate of each memory's expected future contribution, computed as the product of its immediate reward and its projected relevance to the Fast System's next hypothesis selection. Memories with high estimated contribution are promoted to more accessible positions in the hierarchy; those with low contribution are archived. This gating mechanism ensures that the Fast System is most often guided by memories that have demonstrably improved past performance.
Benchmark Results: RoboCasa365 and LIBERO-Pro
EmbodiedRSI was evaluated on two major robot benchmark suites. On RoboCasa365, a suite of 365 diverse robot tasks built on the IsaacGym physics engine, EmbodiedRSI achieves an overall success rate of 77.0% and a Composite-Unseen success rate of 71.3%. The best prior baseline achieves 40.1% overall success and a significantly lower Composite-Unseen rate, demonstrating that EmbodiedRSI's information-directed exploration generalizes to task variants it has not seen during learning.
On LIBERO-Pro, a benchmark of 1000 dexterous manipulation tasks, EmbodiedRSI reaches an overall success rate of 86.8%. This result indicates that the framework is effective even in settings requiring fine-grained motor control and long-horizon planning. Both benchmark results were obtained in simulation-first fashion, where the system interacts with high-fidelity physics engines before any real-world deployment.
Real-World Zero-Shot Transfer
Beyond simulation benchmarks, EmbodiedRSI was deployed on a physical robotic platform for zero-shot transfer. The system was taken directly from the simulation-training regime without fine-tuning on real robot data, and tested on a set of physically grounded tasks including block stacking, container pouring, and tool use. Across these tasks, EmbodiedRSI achieved an overall success rate of 71.3%, with notable behavior such as adapting to novel object arrangements and recovering from unexpected perturbations. This zero-shot transfer result is particularly striking because it demonstrates that the knowledge acquired through simulation-directed continual learning generalizes to the unstructured, noisy dynamics of the physical world.
What This Means for Robotics Practitioners
For robotics engineers building systems that must operate over extended timescales in changing environments, EmbodiedRSI offers a practical architecture for active continual learning. The key design principles are: (1) maintain a structured hypothesis pool rather than relying on a single fixed policy, (2) direct physical experiments toward resolving the most uncertain or most impactful hypotheses, and (3) codify successful behaviors into a reusable library while discarding ineffective ones. The Fast-Slow dual-system organization provides a modular decomposition: researchers can focus on improving the Fast System's hypothesis selection without redesigning the entire Slow System's memory architecture, and vice versa.
From an implementation standpoint, the Hypothesis Graph can be maintained using lightweight data structures (e.g., directed acyclic graphs with edge weights). The VoI computation, while requiring a model of the Fast System's future behavior, can be approximated using sampling-based methods that avoid expensive planning loops. The Code-Skill Co-Evolution process depends on having a program synthesis operator; the paper uses a simple mutation-based operator that perturbs function arguments and loop bounds, which can be replaced with more sophisticated program repair or gradient-based synthesis methods as they become available.
Practitioners should note that EmbodiedRSI's performance depends on the diversity of the initial hypothesis pool. If the system starts with a very narrow set of code and skill primitives, the VoI-directed exploration may be slow to discover effective behaviors. The paper observes that initializing the Hypothesis Graph with a broad set of parameterized primitives—covering common motor patterns such as "push," "lift," "place," and "transport"—accelerates early learning significantly. Additionally, the Hierarchical Memory organization benefits from having multiple abstraction levels pre-populated; engineers should invest in designing meaningful sensorimotor abstractions that correspond to the semantic structure of their task space.
Computational cost is another consideration. The Slow System's memory updates and VoI calculations add overhead per trial, but the paper reports that this overhead is modest relative to the physics simulation cost and does not become a bottleneck up to hundreds of trials per day. For real-robot deployment, the dominant cost remains physical trial time, and EmbodiedRSI's experiment-efficiency gains translate directly into reduced data-collection time.
A Natural Ending
EmbodiedRSI demonstrates that active continual robot learning can be both efficient and effective when the system explicitly reasons about the value of information before acting. By combining a Fast-Slow dual-system architecture, a Hypothesis Graph for maintaining competing code and skill hypotheses, VoI-directed experiment selection, and reward-grounded hierarchical memory, the framework achieves substantial improvements over prior baselines on simulation benchmarks and enables zero-shot transfer to real-world robotic tasks. The approach shifts the paradigm from passive fine-tuning to active, information-directed improvement, and it provides a concrete architecture that roboticists can adopt or extend for their own systems. As robot deployment becomes increasingly common in unstructured environments, methods that make efficient use of every physical interaction will be essential for keeping systems adaptable over their operational lifetime.
Read the paper on arXiv