Recursant cuts AI agent costs by routing each step to the right model
A new open-source proxy called Recursant sits between AI agents and model providers, picking the cheapest model that can handle each step without breaking the agent's train of thought. The project claims 29 to 39 percent cost reduction at equal quality in its benchmarks. It runs locally, speaks the OpenAI API, and requires no changes to existing agent code.
Agents burn money on routine steps
An agent does not make one request per job. It makes dozens: read a file, run tests, read the error, fix it, try again. Most of those steps are routine. Sending all of them to a frontier model wastes money. Recursant intercepts every request and decides where it goes: a top model for the hard steps, a cheaper model for the routine ones, and a local model whenever personal data appears.
Four questions decide every request
For each incoming request, Recursant runs a fixed decision chain in milliseconds. First, does the request contain private data? The checker uses deterministic regexes covering tax file numbers, Medicare numbers, card numbers, phone numbers, email addresses, and any custom patterns you add. If anything matches, the request goes to your private model and nothing later can override that.
Second, is it safe to switch models now? Switching mid-conversation can confuse an agent or lose the cached-context discounts providers offer. Recursant tracks each conversation and only switches where it is safe.
Third, what just happened? If the agent's last action succeeded, the next step is routine and goes to the cheaper model. If it has failed twice in a row, a stronger model takes over. Helper agents start on the cheaper model.
Fourth, which allowed model is cheapest right now? Recursant weighs per-token prices, current load, and health status. No second AI model makes the call. No extra service runs.
Privacy stays local by design
Keys never touch the config file. They live in a separate environment file readable only by you. If your private model fails, Recursant never falls back to a public one. By default it binds to localhost. Opening it to your network is an explicit choice. The project documents the privacy checker as a strong safety net, not a guarantee that no personal data of any kind can ever leave.
Drop-in OpenAI compatibility
Recursant speaks the same language as OpenAI's API. Almost every agent and tool already supports it. The project has tested live with Hermes and pi. Configuration for pi takes three lines in a JSON file. For Hermes you enter the base URL, key, and model "auto" in the custom endpoint dialog. Any tool that asks for an OpenAI address, key, and model works the same way.
Benchmarks show the trade-off
On 980 MMLU-Pro questions, a local GLM-5.3-Flash scored 80.1 percent at zero cost. GPT-6 luna via OpenRouter scored 84.6 percent at fifteen cents. GPT-6.1 sol scored 88.3 percent at $1.53. The top model wins on hard questions, but nothing Recursant tried could predict which questions those were in advance. For single questions it uses the cheapest model you allow. The project also tested small decision models as advisers. They did not save money reliably, so they are disabled.
Written in C for speed and low overhead
The binary is written in C with minimal dependencies. It installs via a one-line script that builds from source, writes a starter config, and generates an API key. You get the program at ~/.local/bin/recursant, settings in ~/.config/recursant/config.json, and keys in ~/.config/recursant/recursant.env. A systemd service runs it at boot. Docker images are available for containerized deployments.
Roadmap adds observability
The core routing, privacy rules, and hybrid model pool are done and tested live with OpenRouter, vLLM, and Ollama. Twenty-eight provider types are predefined. A web console, audit history, and request tracing are planned. macOS support is underway. Windows remains tricky.