A new open-source tool takes a different approach to getting reliable answers from AI models: make them argue with each other. LLM Council, released last month, queries multiple frontier language models in parallel, anonymizes their responses, and forces them to critique each other's work before synthesizing a final recommendation.
How the three-stage deliberation works
The system runs in three distinct phases. First, each of the four models receives a persona and answers the user's question independently, with no cross-contamination between responses. Second, those answers are stripped of identifying information and labeled Response A, B, C, and D, then distributed to all models again. Each model tags the others with one of four labels: ADOPT, MERGE, DEFEND, or CHALLENGE. Third, a designated chairman model synthesizes everything into a recommendation complete with a confidence score, areas of consensus, preserved insights from individual responses, and documented dissent.
The anonymization step is deliberate. By removing brand labels, the system prevents models from deferring to or dismissing responses based on which company built the underlying system. The convergence tags force genuine engagement rather than polite agreement, a pattern that often plagues multi-model setups where models simply echo each other.
Four modes for different decision types
LLM Council ships with four preconfigured modes, each tailored to a different kind of thinking. The council mode pairs a Reframer, Devil's Advocate, Dreamer, and Operator for strategic decisions. The braintrust mode uses a Builder, Skeptic, Dreamer, and Operator for build-or-ship calls. The researcher mode deploys a Methodologist, Domain Expert, Critical Reviewer, and Statistician for investigative questions. The general mode rounds out the options with a Pragmatist, Skeptic, Strategist, and Architect for everyday problems.
Each mode assigns specific roles that push the models toward different perspectives, reducing the risk of groupthink across the deliberation. The Devil's Advocate in council mode actively challenges assumptions, while the Dreamer in braintrust mode pushes for creative solutions that might otherwise get dismissed by more conservative voices.
What it costs and how fast it runs
The system targets four frontier models by default: Claude Sonnet 4.6, GPT-5.4, Gemini 3.1 Pro Preview, and Grok 4.20, all accessed through OpenRouter. A full deliberation with frontier models runs in 30 to 60 seconds and costs between $0.50 and $2.00 depending on query complexity. A simpler configuration using lightweight models drops that to 20 to 40 seconds and $0.10 to $0.50.
Users can override the default models with any combination OpenRouter supports, and the chairman role can be assigned to a different model than the four participants. Every deliberation automatically appends to a chronicle file, creating architectural decision records that accumulate over time without manual documentation.
Built as a Claude skill, open to others
The tool is distributed as a Claude skill, installed by cloning the repository to the skills directory and running a setup script. It requires Python 3.10 or later, an OpenRouter API key, and two dependencies: httpx and python-dotenv, both handled by the installation script.
The project is MIT licensed and released as version 1.0.0. For developers building systems that need to make consequential decisions with AI assistance, the structured disagreement approach offers a practical alternative to simply picking one model and hoping for the best.