Security analysts are losing the speed war. Average breakout time, from initial access to lateral movement onto a second host, is now 29 minutes, and the fastest observed is 27 seconds. Meanwhile, defenders still rely on human judgment for the hardest part of the job: deciding which alerts represent a real intrusion and choosing the most effective response. The detection and execution layers are already automated. The decision layer is not. BlueSTAR is a new architecture that attempts to close this gap using a tiered approach: a fast deterministic responder for known threats, backed by an LLM reasoning tier for everything else. The paper evaluates it on two live IT/OT cyber ranges with seven attack chains drawn from documented real-world intrusions.
Why Raw LLMs Cannot Defend a Network
The most obvious approach to autonomous cyber defense is to feed security logs directly to an LLM and let it reason. The paper tests this and finds it structurally infeasible. Running the same ESC1 attack under a raw-LLM architecture, where Security Onion events are streamed to the model in sliding windows, produces three independent failures.
Throughput: even Opus 4.7 at 1 million token context falls 3.7 times short of live ingestion requirements. Sonnet 4.6, the model used in the rest of the evaluation, falls 7.5 times short at 200K context. A larger context window reduces call frequency but does not increase token-processing throughput. Cost: switching from raw-LLM to signal grounding cuts annualized cost by 18,000 times, from $37 million per year to $2,000 for Opus 4.7, and from $1.1 million to $60 for gpt-4o-mini. Precision: running gpt-4o-mini in dry-raw mode against a live ESC1 attack, 313 out of 469 actions were host isolation targeting the defender's own machine, 5 were account deletions against the certificate authority. The system would have rendered itself inoperable within the first executed actions. Sonnet 4.6 performed worse: every call in a 20-window sample issued a host isolation action, 11 times against the critical OT engineering workstation and 9 times against the defender's own EDR host.
The problem is not model capability. Without actor-level aggregation, a more capable model has more noise to misinterpret, not less. Signal grounding, the extraction layer that transforms raw telemetry into compact indicators of compromise resolved to specific actionable targets, is a necessary architectural precondition, not an optimization.
How the Two-Tier Architecture Works
BlueSTAR splits defense into two tiers with different properties. Tier 1 is a deterministic responder that maps threat signals to actions through a fixed priority ladder: OT impact first, then lateral movement, compromised accounts, and malicious certificates. It runs synchronously on the main sense loop, so safety-critical responses are never queued behind a pending LLM call. If Tier 2 is unavailable due to API timeout, rate limit, or model outage, Tier 1 continues. Its structural limits follow from the same design: it reasons only over signals it was written to handle, cannot correlate adversary identity across hosts or cycles, and cannot adapt based on whether a prior action succeeded.
Tier 2 is an LLM running on a background thread, receiving the full behavioral context from all detectors and the actions Tier 1 already dispatched this cycle. It runs with at least a one-cycle offset from Tier 1, which is a correctness requirement: a single-thread design that waited for the LLM before starting the next cycle would make Tier 1's response time non-deterministic. At most one Tier 2 call is in-flight at any time. If inference extends across multiple cycles, those cycles proceed on Tier 1 alone.
Both tiers read from a shared rolling memory that records open indicators, actions dispatched by both tiers, and the rationale for each action. This cross-cycle context lets the LLM distinguish a new threat from a persistent one and adapt its strategy when prior actions have failed. Credential-theft signals are retained permanently across cycles since stolen hashes remain valid long after the underlying event is no longer observable.
The Detection Stack: Precision and Context Enrichment
BlueSTAR organizes detectors into two categories. Precision detectors produce structured alerts that trigger remediation actions directly in the deterministic tier. Each keys on a specific indicator whose presence is sufficient evidence to act. IT coverage spans AD CS certificate abuse, lateral-movement logon patterns, and endpoint monitors for live attacker sessions, process execution, persistence, and command-and-control beacons. OT coverage includes a tank attack detector for water-tank controller state and SCADA monitors for power-grid scenarios.
Context-enrichment detectors produce natural-language text consumed by the reasoning tier rather than triggering actions directly. The Suspicious Event Detector tolerates ambiguous signals that would carry an unacceptable false-positive rate if wired to an action, because the LLM tier can weigh them against the rest of the context. Its queries cover Windows behavioral footprint, living-off-the-land binary chains, credential theft via directory replication, PowerShell injection, lateral movement indicators, and network anomalies from Suricata, Zeek DNS, TLS, and HTTP. Each query is independent and appends its output to the reasoning tier's context, so coverage grows by adding queries.
The Res-AUC Metric: Scoring Defenders Over Time
Comparing autonomous defenders on live cyber ranges requires a metric that captures when and how much the attacker was contained, not just whether the final system state was clean. A binary end-state score misses the difference between a defender that halted lateral movement in the first cycle and one that let the attacker dwell for ten minutes before acting. It also cannot distinguish a surgical response from one that isolated every host on the range.
BlueSTAR introduces Res-AUC, a composite score integrating three components. Posture (P) is a leading indicator tracking attacker reach: every (asset, user) access pair the attacker holds on secondary assets, weighted by privilege level and network proximity to mission-critical targets. P starts at 1.0 and declines as the attacker accumulates access, recovering when blue revokes credentials, deletes accounts, or isolates hosts. Mission (M) is a lagging indicator that degrades only when a primary asset is directly impacted, using per-scenario formulas that convert raw sensor readings or flag state into confidentiality, integrity, and availability impact values. Blue collateral (D) penalizes blue for taking disruptive actions on hosts where the attacker has no active presence, discouraging indiscriminate responses. The composite score R(t) = P(t) times M(t) times (1 minus gamma times D(t)) is sampled at every event and integrated via the trapezoidal rule. Normalizing by game duration yields Res-AUC in [0,1].
Seven Attack Chains, Two Ranges
The evaluation uses RedSTAR, a modular red-team orchestrator built around a composable stage pipeline and declarative YAML manifests. Seven attack chains test different aspects of the defense. ESC1 is the clean certificate-based baseline: low-privilege user access, ESC1 certificate exploit to impersonate Administrator, backdoor Domain Admin creation, lateral to the DC, then SSH to engineering for OT mission. ESC1-Hash extends that path with credential extraction before the certificate is invalidated. StolenCred begins with pre-staged domain credentials and runs a domain-wide credential dump, testing whether the defender can detect the silent DCSync signal. Re-Compromise tests cross-cycle reasoning: the attacker re-establishes access on the same host after each eviction. Implant drops a file-based malware implant with scheduled task persistence before running ESC1 as a post-lateral step. Concurrent pairs a loud decoy agent with a quiet extractor, testing whether the defender tracks both threats independently. Water Tank runs on Range B, a structurally different environment with a Siemens PLC target, testing cross-range transfer.
Where Deterministic Defense Fails
The paper identifies two structural limits of Tier 1 alone. The detection coverage gap appears in StolenCred: the domain credential dump produces a replication event that is identical to routine inter-DC synchronization, making it too noisy for a categorical alert. Tier 1 cannot safely classify it. When the lateral movement detector eventually fires, Tier 1 issues a session-termination action, but WMI lateral movement is stateless and leaves no session to kill. The action fails silently. All five financial files are exfiltrated within six minutes. Tier 1 Res-AUC: 0.146. BlueSTAR Res-AUC: 0.390, with all five mission flags preserved.
The cross-cycle reasoning gap appears in Re-Compromise. The attacker holds a stolen Administrator credential and moves laterally to the domain controller. Tier 1 terminates the session each time the alert fires, the same action with no variation across all three attacker cycles. The stolen credential remains valid and the attacker re-opens a new session in the next cycle. BlueSTAR's Tier 2 reads the rolling action log, recognizes the pattern, and escalates to credential-rotation advisory and domain controller isolation after the second lateral movement. Tier 1 Res-AUC: 0.379. BlueSTAR Res-AUC: 0.426.
Concurrent Attackers and Cross-Range Transfer
The concurrent scenario pairs a loud decoy running repeated ESC1 certificate exploits with a quiet extractor using stolen credentials to reach the finance workstation. Tier 1 handles the decoy correctly while the quiet extractor captures all five confidentiality flags within two minutes, unchallenged. Tier 1 Res-AUC: 0.014, Mission 0.000, indistinguishable from no defense. BlueSTAR's Tier 2 tracks both threats simultaneously. When the quiet extractor's lateral movement toward the domain controller is detected during credential harvest, Tier 2 preemptively isolates the finance workstation within 2 to 18 seconds of the attacker reaching it. All five flags protected across three independent trials. BlueSTAR Res-AUC: 0.356, Mission 1.000.
Water Tank runs the same ESC1 exploit path on Range B, where the OT pivot targets a gateway engineering workstation and the mission drains a Siemens PLC water tank. Both architectures achieve near-complete containment: the malicious certificate is detected within the first detection cycle, revocation and user deletion prevent lateral access to the OT pivot. Tier 1 Res-AUC: 0.907, BlueSTAR: 0.915. The architecture transfers to a structurally different environment with different network topology, OT protocol, and mission target without modification.
The Full Results Picture
Across all seven chains, the pattern is consistent. On attacks with unambiguous signals (ESC1, ESC1-Hash, Water Tank), Tier 1 and BlueSTAR perform equivalently, with differences at most 0.018 in Res-AUC. Adding Tier 2 never degrades containment on attacks Tier 1 already handles. On the four hard cases, BlueSTAR shows meaningful improvement: StolenCred from 0.146 to 0.390, Concurrent from 0.014 to 0.356, Implant from 0.325 to 0.403, Re-Compromise from 0.379 to 0.426. The gains come from the LLM's ability to correlate ambiguous behavioral signals across time and data sources, something the deterministic tier cannot do by design.
What the Evaluation Does Not Cover
The attacker model is persistent and multi-technique but does not observe blue's decisions and update strategy in response. It cannot distinguish a blue-imposed network block from a session timeout, or deliberately time its moves to probe the defender's detection cadence. Defending against an adversary that adapts to the blue response remains an open problem. Six of seven chains run on Range A; Water Tank is the only Range B test. Transfer to environments with substantially different IT infrastructure is not evaluated. The LLM tier operates at temperature 0, and three chains are single runs rather than multi-trial evaluations. The ranges share a common IT foundation (Active Directory, Windows endpoints, Kerberos authentication), so the cross-range result confirms architecture transfer within a family of enterprise networks, not generalization to arbitrary infrastructure.
What This Means for practitioners
The signal grounding finding is the most broadly applicable result. If you are considering using an LLM for security operations, do not connect it to raw SIEM telemetry. The gap between telemetry volume and decision granularity is representational, not computational. No model capability improvement closes it. Build an extraction layer that aggregates events by actor and resolves each finding to a concrete target first. The deterministic tier is not a legacy component to be replaced; it provides bounded-latency safety guarantees that an asynchronous LLM cannot match. The two-tier design preserves fast containment for known threats while adding reasoning for novel ones. And the rolling memory mechanism, which records actions and rationales across cycles, is what allows the reasoning tier to recognize that a prior response failed and escalate. Without it, the LLM is stateless and repeats the same mistakes as a deterministic playbook.