Catastrophic forgetting in long-horizon agents
Agents operating over extended trajectories must acquire a diverse set of capabilities, often from interleaved data streams. A persistent problem in this setting is catastrophic forgetting: updating shared parameters to learn a new skill can erode previously acquired ones. Conventional joint training and static adapter merging maintain a fixed parameter configuration throughout execution, meaning different stages of a trajectory that require different capabilities cannot dynamically reallocate resources. This rigidity limits the agent's ability to compose skills on the fly, particularly when tasks vary in their perceptual, reasoning, or computational demands.
WeAct: weight-level internal actions
WeAct (Weight-level Internal Actions) addresses this by training independent capability-specific LoRA adapters over a frozen backbone and inserting adapter selection into the agent's action space. Each adapter specializes in a distinct capability—such as code generation, textual reporting, or visual reasoning—while the backbone weights remain unchanged. During execution, the agent invokes the appropriate adapter as an "internal action," effectively routing to the right skill without modifying shared parameters. Because adapter selection happens through the model's native tool-calling interface, composition occurs naturally in a shared context: the model can switch between adapters mid-trajectory, combining capabilities as needed.
Empirical results across deep-research benchmarks
Evaluated on eight deep-research benchmarks, Qwen3-32B with WeAct achieves a 6.2% average accuracy gain over full fine-tuning counterparts while consuming less compute. Relative to the strongest static LoRA-merging method, the improvement widens to 27.5%. These gains demonstrate that dynamic adapter selection outperforms both full parameter updates and fixed merging strategies, particularly when the agent must toggle between diverse skills within a single horizon.
Modular continual adaptation
Beyond improving performance on existing tasks, WeAct supports modular continual adaptation. A new code computation adapter can be added to expand the agent's capabilities, while an existing writing adapter can be updated to improve report quality. Both operations leave the backbone and all other adapters untouched, meaning the agent accumulates new skills without retroactively degrading previously learned ones. This property makes WeAct well-suited for agents that must grow their repertoire over time, such as research assistants or multimodal coordinators.
What this means for agent design
For practitioners building long-horizon agents, the key insight is that capability composition need not rely on parameter updates. By treating adapter selection as an action the agent can call, the system gains on-demand flexibility while preserving all previously acquired skills. The compute efficiency—6.2% better accuracy with less training compute—also suggests that WeAct can be a more sustainable alternative to full fine-tuning in resource-constrained settings.
Limitations and future directions
The current work focuses on a fixed set of capability-specific adapters; the discovery or pruning of adapters at runtime remains an open question. Additionally, the benchmarks evaluate relatively short horizons; extending WeAct to truly multi-day or multi-agent coordination tasks would require further investigation. The interaction between adapter selection and the model's attention mechanism also warrants study, as dynamically routing to different LoRA subspaces could influence representation overlap and interference.
Nevertheless, the central idea—decoupling capability acquisition from backbone updates and exposing adapter choice as an agent action—provides a reusable pattern for reducing forgetting and enabling modular growth in agent systems.