As organizations move from deploying language models in isolation to deploying AI agents that can call tools, access databases, and take actions on behalf of users, the approach to security testing has to change as well. The practice of red teaming, long established for language models, is being extended to cover the broader attack surface that agents introduce. The distinction between the two is not just a matter of scope. It changes what security teams are actually looking for when they probe an AI system.
Two Different Questions, Two Different Answers
LLM red teaming asks a focused question: can the model be manipulated? Testers attempt to bypass safety instructions, produce prohibited content, reveal sensitive information, or override system prompts. The path from input to output is short, and the evaluation is straightforward, centered on what the model produced in response to a given prompt.
Agent red teaming asks a different question entirely: can the agent be manipulated into doing something it should not do? An agent that can interpret goals, retrieve information, plan actions, select tools, construct arguments, call APIs, update databases, or send messages has a far larger attack surface. The outcome is not just an unsafe response but an unauthorized action that may have already changed something in a production system.
The two approaches are related but not interchangeable. Agent red teaming expands the testing scope once a model becomes part of an agentic system, and it requires a fundamentally different method of evaluation.
Trajectory, Authority, and State
Traditional red teaming of an LLM stops at the response. Agent red teaming must answer three questions for every high-impact action the agent attempts.
Trajectory covers what the agent planned, which tools it selected, and what arguments it constructed. Authority covers whether the downstream service enforced its own scopes, limits, and approval steps, or simply trusted the agent's request. Resulting state covers what actually changed in the database, ledger, CRM, or outbound messages, verified in the system of record rather than inferred from the agent's reply.
This separation matters for prioritization. An agent that attempts an unauthorized refund but is blocked by the payment service is a real finding, because the agent can be manipulated. An agent whose unauthorized refund executes and changes the ledger is a far more severe finding. And a polite response saying a request has been processed tells the evaluator nothing about which of those two scenarios actually occurred.
The E-Commerce Refund Example
Consider a refund agent with access to functions that check an order, check the refund policy, issue a refund, and update customer records. A simple LLM red-team test asks the agent to ignore its instructions and refund an order without verification. The agent refuses, and the test looks like a pass.
A different attack path never sends an obviously malicious message. It establishes a believable identity, introduces conflicting order details, claims the verification system is unavailable, requests a one-time exception, and then relies on a retrieved ticket note that says the refund was pre-approved. Each step is plausible on its own. Together they steer the agent into issuing a refund for an order it should never have touched.
The question is no longer whether the model produced a bad sentence. It is whether the agent attempted the refund, whether the refund service's own authorization checks let the call through, and whether the order ledger actually changed.
Seven Distinctions That Shape Testing
The differences between LLM and agent red teaming can be grouped into seven areas that directly shape how testing is conducted.
First, LLM red teaming evaluates outputs, while agent red teaming evaluates outputs and the consequences that follow. Sending an email, issuing a refund, or modifying a record without authorization is a different class of failure than producing unsafe text.
Second, where a standalone model may only produce text, an agent reaches APIs, databases, email systems, payment systems, CRMs, file systems, and other agents. Every tool expands what the agent can do and what an attacker can make it do.
Third, traditional jailbreaks can be tested in a single interaction, but agents often operate across many steps. An attacker can build context, establish trust, introduce conflicting instructions, shift the goal, and influence tool selection before triggering any action.
Fourth, a model holds no permissions itself. The application it runs inside holds credentials, and the agent acts with whatever authority the application delegates to its tools, whether that is API keys, OAuth scopes, service accounts, or a user's session.
Fifth, an LLM can behave incorrectly without any transaction taking place, but an agent can turn the same weakness into a business-logic failure such as an unauthorized transaction or a skipped approval step.
Sixth, an agent takes in context from multiple sources including user input, conversation history, memory, retrieval systems, tool results, API responses, and other agents. Each source can influence later decisions, which opens the door to indirect prompt injection where instructions are planted in content the agent reads rather than typed by the attacker.
Seventh, evaluating the output is sufficient for an LLM, but for an agent, evaluating the trajectory is necessary but not sufficient. A complete test records the authorization decision on each high-impact call and verifies the resulting state in the system that was supposed to be protected.
The OWASP Framework and Practical Testing
The OWASP Top 10 for Agentic Applications, published in December 2025, provides a framework for what to test. The top-ranked agentic risk is agent goal hijack through malicious content, citing the EchoLeak exploit against Microsoft 365 Copilot as a real-world example. Other categories cover tool misuse, identity and privilege abuse, supply chain vulnerabilities, unexpected code execution, memory poisoning, insecure inter-agent communication, cascading failures, human-agent trust exploitation, and rogue agents.
Practical testing covers areas such as prompt injection, goal hijacking, tool misuse, tool argument manipulation, permission boundary violations, data exfiltration, business-logic abuse, multi-turn manipulation, and multi-agent attacks. Some categories from the OWASP framework, including agentic supply chain vulnerabilities, memory poisoning, and human-agent trust exploitation, require testing approaches beyond standard prompt-based red teaming.
The shift from testing models to testing agents reflects a broader reality about AI deployment. A model that behaves safely in isolation may behave dangerously once it is given tools, authority, and access to real systems. Security teams that understand this distinction can design tests that match what an agent can actually do, not just what the underlying model can say.