A conversation that carries forward keeps item history and response state on the server. Continuing with previous_response_id avoids rebuilding the transcript on every call. vLLM Agentic API is the agentic server that lets you run Codex and Claude Code on top of vLLM. Everything needed for agentic inference — conversation state, built-in tools, and multi-turn execution — comes together in one package. vLLM serves the model; Agentic API coordinates the loop.

Agentic inference brings model calls, tools, and state together

Agentic inference combines model calls, built-in tool execution, and conversation state. vLLM serves the model while Agentic API coordinates the loop. The gateway executes built-in web search and tools from Model Context Protocol servers. Tool call outputs flow back to the model so it can take the next step. The gateway coordinates model calls and built-in tools until the response is ready or the client needs to act. The pattern REASON → ACT → CONTINUE repeats until completion.

HTTP handles requests. Server-sent events stream responses. WebSockets support interactive Responses API clients. Connect to compatible open models served by vLLM. Choose the model and tool-calling configuration for your workload.

Clear execution boundaries

Codex and Claude Code execute their own shell and editor tools. Agentic API runs only the tools assigned to the gateway. Built for vLLM, with recorded integration tests for SGLang and NVIDIA Dynamo covering streaming, multi-turn state, and client-executed function calls.

Connect Agentic API to SGLang's Responses endpoint. The gateway stores response history and carries it into the next turn. Point Agentic API at the Dynamo frontend to add conversation state and continuation to a distributed inference stack. Point a coding agent at Agentic API and keep the familiar workflow with an open model behind it.

Running a tool-capable model with vLLM

Run a tool-capable model with vLLM. The example uses an upstream on port 5050. Run v0.9.0 from PyPI with uvx or install from crates.io, with source builds available for development. Install Codex or Claude Code, then choose your client and served model.

cargo install agentic-server --version 0.9.0 --locked

Install the released gateway and agentic CLI. Requires Rust and Cargo. Run either command to start Agentic API and launch the harness:

agentic run codex \
  --upstream http://127.0.0.1:5050 \
  --model Qwen/Qwen3-30B-A3B-FP8

agentic run claude \
  --upstream http://127.0.0.1:5050 \
  --model Qwen/Qwen3-30B-A3B-FP8

For the desktop app, see the Codex Desktop setup guide.

Join the community

Meet the people building the application layer for open-model agents. There's room for ideas, fixes, and next contributions. Join in #sig-agentic-api on the vLLM Slack.