Sakana AI has released AI Scientist v3, a system that turns a task description into a full research campaign — experiments, ablations, and a compiled paper — driven by coding agents. The project treats agents as AI scientists. Starting from a task.md file, they propose ideas, run experiments, evaluate outcomes against a rubric, and iterate toward stronger programs. The pipeline then runs baselines and ablations, a sealed final evaluation, post-hoc analyses, and writes the paper, with every stage kept under one run directory.
How the Pipeline Executes
The recommended setup runs every agent phase inside Docker. The default command agsci run points at a task directory and executes six phases in sequence: task drafting, investigation, ablations, final evaluation, post-hoc analysis, and paper writing. Optional review and revision phases follow. Each phase runs in its own container and only ever reads the run-local snapshot, so results stay reproducible and the examples directory is never modified.
The default configuration uses the Claude agent, every phase in Docker, and the CPU-only circle-packing example. It needs a Claude login, no GPUs, and exercises every phase through to a compiled paper. Adding --dry-run validates the configuration and prints every phase command without running anything. Every option is a Hydra key-value override; agsci run --help lists them all.
Phase Breakdown
Task drafting writes a scaffold only when the task directory has none. Every shipped example already includes its evaluator, so the drafting phase snapshots it and the investigation phase builds and validates the task runtime. Drafting from a bare research brief remains experimental.
Investigation is the agentic idea → experiment → evaluate loop. Agents propose ideas, run experiments, evaluate outcomes against the task rubric, and iterate toward stronger programs. Ablations compare the winning program against baselines and ablated variants. The final evaluation re-scores the winner in a sealed environment. Post-hoc analysis mines the artifacts for extra evidence. The paper phase writes, compiles, and audits the manuscript against that evidence.
Each run writes one predictable envelope under results/: results/. This directory contains task_drafting logs and validation audit, an immutable run-local task snapshot, investigation ideas and attempts with winner and optimizer memory, ablations baselines and variants, final_evaluation sealed results, posthoc analyses, and paper manuscript PDF with manifest and optional AI reviews. Investigation plus the task snapshot is enough to write a paper; the other phases add evidence when available and are reported as degraded when they fail.
Agent and Model Flexibility
The system drives Claude, Codex, Cursor, Opencode, Gemini, and Pi through Headless CLI, so the same experiment can be repeated across agents and models. Pick one with agent.name=. Agent CLIs are already inside the Docker image. For native runs, install the CLI of the agent you use plus Git and Node.js.
Only the selected agent's API keys are forwarded into its containers. The .env file at the repo root hosts keys like CLAUDE_CODE_OAUTH_TOKEN for Claude, OPENAI_API_KEY for other agents, GEMINI_API_KEY for Gemini, and CURSOR_API_KEY for Cursor. Concept figures use OPENAI_API_KEY, fall back to GEMINI_API_KEY, and otherwise ship a plain export. Never commit the .env file.
GPU Requirements and Presets
GPUs are required only by the GPU task presets. The pipeline discovers them through nvidia-smi and refuses to start compute phases unless the visible GPUs match the preset. Task presets under configs/preset/ hold the campaign budgets, timeouts, paper format, and required GPUs. configs/preset/paper.yaml is the shared campaign preset and configs/preset/grpo.yaml composes it for the GRPO task (two GPUs, harness-side GPU evaluation).
Explicit overrides win over the preset. For example: agsci run examples/grpo preset=grpo agent.name=opencode agent.model=amazon-bedrock/global.anthropic.claude-opus-5-5 investigation.execution.max_idea_cycles=20. Shrinking the ablation and post-hoc budgets makes the first end-to-end run finish faster without skipping a phase: agsci run examples/circpack investigation.execution.max_idea_cycles=2 ablations.execution.num_replications=1 ablations.execution.max_evals_per_variant=2 posthoc.execution.max_selected_analyses=1.
Resuming and Review
Every phase resumes against the run directory. The pipeline recovers the original idea-cycle budget; pass the original agent and model again to continue after investigation with ablations, post-hoc analysis, and the paper: agsci run examples/circpack execution.phases=ablations,posthoc,paper paths.run_dir=results/circpack/claude/20260331_094917. Rewrite only the paper with agsci run examples/circpack execution.phases=paper paths.run_dir=results/circpack/claude/20260331_094917.
Adding the review phase sends the complete manuscript to an OpenAI-backed reviewer. Review failures are recorded but never discard the paper. The same reviewer runs standalone on any PDF with agsci review. It reads OPENAI_API_KEY from the environment or the repo .env and prints the review JSON: agsci review results/circpack/claude/.
Autoillustrate and Standalone Investigation
Autoillustrate is optional and built into the image for concept figures. To use it directly on the host, follow the single-command install in the Autoillustrate README. The standalone investigation loop runs natively by default, optionally with parallel workers: agsci investigate agent.name=claude paths.task_dir=examples/circpack execution.workers=2 execution.max_idea_cycles=4.
Security and Installation
The system launches coding agents with full tool permissions (Headless --allow yolo). They can run arbitrary commands and inherit the API keys you give them, and the task files and papers they read steer what they do. Keep runs inside the Docker image rather than on your workstation, only run tasks and sources you trust, and review run directories before sharing them: agent logs can echo document contents and paths.
Install Docker, then build the agent image: docker build -t agentic-scientist/agent-cli:latest -f src/docker/agent-cli/Dockerfile .. If the build cannot reach package mirrors, add --network=host. Python ≥ 3.11 with uv. uv sync creates .venv and installs the agsci command. Create .env at the repo root with the selected agent's keys only.
Run agsci doctor to check the host, or agsci doctor examples/ to check everything one specific run needs. Browse any run in the browser with agsci inspect start (http://127.0.0.1:8765). The paper lands in results/circpack/claude/.
If you use Agentic Scientist in your research, cite: The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery, Lange et al., arXiv preprint arXiv:2610.18933, 2026.