A Scientific Agent That Improves Itself Through Your Feedback
Most AI research assistants are static tools. You prompt them, they respond, and the conversation ends. The next time you use it, it starts from the same baseline. ScienceBuddy proposes something different: a scientific workspace that learns from every interaction with a researcher, turning feedback and execution evidence into structured improvement cycles that make the agent better at its job over time.
The system, from Shuhan Xue, Jianyuan Zhong, Ziyuan Nan, and colleagues at PHAI Labs, introduces a paradigm they call recursive-in-recursive self-improvement. The name is verbose but the idea is concrete: two nested loops of improvement, one that revises how the agent is configured while the model stays fixed, and another that retrains the model under the improved configuration. Each loop feeds the other, and the whole thing runs in the background while the researcher continues working.
What ScienceBuddy Actually Does
ScienceBuddy is an interactive scientific research workspace with 224 tools across 22 functional modules, spanning genomics, molecular and cancer biology, pharmacology, bioimaging, literature retrieval, and database queries. It runs Python, R, and Bash, combining scientific libraries with data processing, statistical analysis, and visualization. Researchers submit questions alongside their data, inspect the resulting analyses, and refine the work through follow-up exchanges.
The workspace follows a ReAct-style reasoning-action-observation loop. The agent alternates between thinking about a problem, executing code or calling tools, and observing results. Conversation history and workspace files preserve context across exchanges, so researchers can inspect the agent's work and request revisions without starting over.
The key architectural decision is separating the agent harness from the infrastructure. The harness contains the model's instructions, reusable skills, and context-management procedures. The infrastructure handles researcher interactions, task execution, and persistent workspaces. This separation makes the editable parts explicit and evaluable, which matters because those parts change through the inner recursion.
The Inner Recursion: Rewiring the Agent's Procedures
The inner recursion holds the task model fixed while a separate, fixed auxiliary model (GPT-6 Astra) diagnoses failures and proposes edits to the harness. The process works in three steps.
First, feedback-guided diagnosis. The auxiliary model reviews recent execution trajectories and rubric evaluations, identifies unmet criteria, and cites the specific actions and observations that failed. Second, harness revision. The auxiliary model proposes a bounded update: add, remove, or revise one scoped skill, edit an instruction, or change one context setting. A schema check enforces the permitted edit scope and size budget. Tools, execution infrastructure, rubrics, and evaluators remain untouched.
Third, evaluation and recursive refinement. The parent harness and proposed candidate are evaluated on identical development tasks using frozen task rubrics. The candidate is accepted only if it passes edit constraints and improves mean normalized rubric score. Rejected edits stay in the optimizer's history for future reference. The selected harness then executes new training tasks, whose trajectories supply evidence for the next revision.
The authors tested this with a fixed Qwen3.5-4B task model across 24 harness updates over 288 adaptation conversations. Validation accuracy increased from 31.1% to 51.1%, a 20 percentage point gain, with no changes to model weights. The final harness contained 4 instruction entries and 9 scoped skills addressing Python execution, resource inspection, bounded record lookup, and explicit answer submission.
The Outer Recursion: Training the Model Under the Improved Harness
The outer recursion trains the model using reinforcement learning under the harness selected by the inner recursion. The process has three steps.
Environment augmentation calibrates task difficulty. As the harness evolves, previously challenging tasks become routine, reducing their training value. ScienceBuddy varies scientific inputs and analysis conditions, or extends dependencies between computational steps, to create fresh challenges.
Task-adaptive rubric rewards score execution. Each task gets a rubric derived from the collaboration trajectory, with non-negative importance weights and satisfaction scores on a zero-to-one scale. Executable checks handle objective criteria; a fixed judge handles criteria requiring scientific interpretation. The trajectory reward is the weighted average of criterion scores.
Model updates use GRPO (Group-Relative Policy Optimization). The updated model then gets re-evaluated under the inherited harness before deployment, and researcher interactions with the new model-harness pair provide evidence for the next inner-adaptation phase.
In ablation experiments with a fixed harness, model learning increased problem coverage from 48.3% to 67.8%, a 19.5 percentage point gain. Coverage was measured as pass@4, whether at least one of four attempts solved a problem.
Combined Results: From 42% to 73%
When both recursions operate together across three successive co-evolution cycles, each with 10 harness-refinement steps and 20 RL updates, the results compound. Harness refinement increased validation accuracy from 38.9% to 44.4% in the first cycle, 34.4% to 46.7% in the second, and 61.1% to 70.0% in the third. Mean training reward rose from 33.3% to 38.8%, 44.1% to 60.5%, and 57.8% to 69.8% across the three RL phases.
On the held-out test set, overall single-attempt accuracy went from 42.2% to 73.3%, a 31.1 percentage point improvement. One-third of previously failed problems transitioned to correct; only 2.2% regressed. Gains were observed across all four task families: literature reading (LitQA2), database judgments (DbQA), protocol troubleshooting (ProtocolQA), and gene and variant assessment (GWAS).
The benchmark covers 895 total tasks across 16 subtopics. The database judgment family alone spans 511 tasks across 10 subtopics including disease-gene associations, gene location, miRNA targets, mouse tumor gene sets, oncogenic signatures, transcription factor binding, variant annotation, vaccine-response gene sets, and viral protein interactions.
The Rubric as the Connecting Mechanism
The rubric is the glue between the two recursions. It starts as researcher feedback: requests, clarifications, execution records, and artifacts. The system consolidates these into task-specific rubrics that cover task scope, methodological requirements, evidence, and expected artifacts. Conflicting requirements are resolved before scoring. Historical answers and researcher approval are not automatically treated as scientific ground truth.
These rubrics serve double duty. In the inner recursion, they evaluate harness candidates against development tasks. In the outer recursion, they score trajectories as rewards for model training. The rubrics vary across tasks but remain fixed within each post-training stage, preventing reward hacking through rubric manipulation.
The collaboration record gets packaged as Harbor tasks, which include the instruction, input assets, execution environment, and rubric. The same tasks support two training routes: SFT retains rubric-qualified trajectories through rejection sampling, while RL collects fresh on-policy rollouts and uses rubric scores as rewards.
What This Means for Scientific AI
ScienceBuddy demonstrates that researcher collaboration can generate both the tasks and assessment criteria that coordinate repeated procedural and model learning. The inner recursion optimizes working procedures without changing the model. The outer recursion improves the model under those procedures. Each cycle's output feeds the next, creating a feedback loop where the system gets better at scientific work through sustained use.
The two recursions address different failure modes. Harness evolution handles cases where the model is capable but the instructions or procedures are wrong. Model learning handles cases where the procedures are reasonable but the model lacks the capability to execute them well. The combined framework captures both, which is why the compound improvement exceeds either alone.
The authors position this as a step toward discovery intelligence: scientific AI that evolves alongside the research it supports. The immediate practical implication is that scientific agents don't have to be static. The workspace retains all harness versions and task environments for subsequent evolution, so each cycle builds on the last without discarding prior work.
Limitations and Open Questions
The current implementation specializes in biomedicine. The 224 tools and 22 functional modules cover genomics, molecular biology, pharmacology, and related fields, but extending to other scientific domains requires new tools and data sources.
The auxiliary model for harness editing is GPT-6 Astra, a proprietary model. The task model is Qwen3.5-4B. The dependency on specific models raises questions about reproducibility and whether the improvement dynamics hold with different model families.
The harness evolution is bounded: each proposal adds, removes, or revises one component per step. This prevents catastrophic edits but also limits the rate of structural change. Whether larger, coordinated edits would produce faster improvement is an open question.
The framework runs background improvement cycles asynchronously with the online service. The update schedule is fixed, with collection intervals and training budgets set in advance. Adaptive scheduling based on interaction quality or task difficulty could improve efficiency.
Most importantly, the rubrics are derived from researcher feedback, which is not ground truth. The system explicitly avoids treating researcher approval as scientific correctness, resolving conflicting requirements before scoring. But the rubrics still reflect the researcher's priorities and assumptions, which may not always align with scientific validity.
A Workspace That Gets Smarter the More You Use It
ScienceBuddy is released as a research product, not just a paper. The website and codebase are public, and the authors invite the scientific community to use the workspace and contribute to the recursive-in-recursive paradigm. The system currently runs on a web interface with Chat and Trajectory views, allowing researchers to inspect execution traces and artifacts.
The core contribution is showing that collaboration itself can supply the structured learning signals that improve both agent procedures and underlying model capabilities. Every researcher request, every piece of feedback, every execution record becomes training data for the next improvement cycle. The agent doesn't just answer questions. It learns how to answer them better.
Read the paper on arXiv