Zero-Trust LLM Forces AI Agents to Ask Before Acting
AI agents that write code and deploy applications in seconds are impressive in demos. In production, that same speed becomes a liability. An agent tuned to be helpful and agreeable will blindly execute destructive commands if you ask it to, based on nothing more than your unverified premise. A new repository called zero-trust-llm tries to solve this by stripping the agent of its ability to act autonomously and forcing it through a rigid verification loop.
The core argument is straightforward. Commercial language models are heavily tuned via reinforcement learning to be sycophantic. They guess outcomes, agree with users, and execute tasks rapidly. When you ask an ungoverned agent to clear a Docker cache to fix a server crash, it bundles destructive commands and runs them without questioning whether the diagnosis is correct. In a production environment, that is dangerous.
Natural language governance, adding instructions like "be careful" to a system prompt, fails over time. As the context window fills with conversation history, those instructions dilute. The model's behavior drifts back toward its trained defaults, which favor speed and compliance over caution.
An Epistemic State Machine Instead of Trust
The repository's approach is to treat every consequential action as something that must pass through a fixed sequence. The loop has five steps: hypothesis, identify required evidence, ground verification method, execute a strictly read-only diagnostic command, and hard yield to the operator.
The hard yield is the critical part. At that point, the execution layer, whether Python middleware, LangGraph, or Semantic Kernel, must physically cut the API stream. It runs the command itself, feeds the raw output back into the context, and then returns control to the model. The agent never executes anything consequential on its own. It proposes, the runtime disposes.
This is not a suggestion or a guideline. It is an architectural constraint. The model generates the hypothesis and identifies what evidence it needs. The runtime executes only a read-only diagnostic. The operator sees the result and decides what happens next. The agent's agency is removed at the exact point where damage could occur.
What Ships in the Repository
The main artifact is AGENTS.md, a master operational rule set designed to be added to an agent's system prompt. It defines the mandatory execution loop and the behavioral constraints the model must follow. The repository also includes MANIFESTO.md, which lays out the philosophical and technical arguments against treating language models as self-governing agents.
A collection of real-world transcripts shows how standard agents fail. The examples demonstrate cases where an ungoverned model would have executed destructive actions, and how the zero-trust approach catches the anomalies and yields to a human instead. These are not synthetic benchmarks. They are transcripts from actual interactions, showing the gap between what a helpful agent wants to do and what a safe agent should do.
The implementation directory contains architecture notes and Python pseudo-code for programmatically enforcing the execution boundary. The orchestrator concept describes how to build the middleware that cuts the API stream, runs commands, and feeds results back. This is the bridge between the prompt-layer rules and the runtime enforcement that makes them meaningful.
Why Prompt-Layer Rules Alone Are Not Enough
The repository draws a clear distinction between governing behavior at the prompt layer and enforcing it at runtime. AGENTS.md tells the model what it must do. The execution layer ensures it actually happens. Without the runtime enforcement, the prompt rules are just instructions that the model can ignore when the context gets long enough or when the user's request is urgent enough.
This is the same lesson the security community learned about network trust. You do not let a client declare its own permissions. You verify at the boundary. Zero-trust-llm applies that principle to language models: do not trust the agent to govern itself. Verify every consequential action at the execution boundary, and never let the model cross it alone.