You Cannot Stop What You Cannot See

Prevention fails eventually. When it does, detection determines whether an intrusion is caught in hours or discovered months later. Detection engineering is the discipline of collecting the right signals, turning them into reliable alerts, and responding fast enough to matter.

Log Sources That Matter

Effective detection starts with comprehensive, centralized logging. High-value sources include:

  • Authentication: successful and failed logins, MFA events, password resets.
  • Endpoints: process creation, file writes, registry changes, script execution (EDR/Sysmon).
  • Network: DNS queries, flow records, proxy logs, firewall denies.
  • Cloud: control-plane API calls (CloudTrail, audit logs), IAM changes, storage access.
  • Applications: web server access logs, error logs, and audit trails of sensitive actions.

A single source is a fragment; correlation across sources is where detection power lives. A failed login alone is noise; a failed login followed by a new IAM key created from an unusual IP is a story.

Centralization and Normalization

Logs scattered across hosts are useless under pressure. Ship them to a central platform—a SIEM (Splunk, Elastic, Sentinel, Chronicle) or a log lake—and normalize fields so queries work across sources. Time synchronization (NTP) is mandatory: without consistent timestamps, correlation is impossible.

Detection Rules and Signals

Detections come in several flavors:

  • Signature rules: match known-bad patterns (a specific tool name, a known-malicious domain, a suspicious command line).
  • Behavioral/analytics: flag deviations from a baseline (a service account suddenly reading thousands of records, or a user logging in from two countries within an hour).
  • Threat-intelligence matches: indicators from feeds, useful but rapidly stale.

Rules should map to MITRE ATT&CK techniques, which gives a shared vocabulary and reveals coverage gaps. If you have no detection for credential dumping, lateral movement, or persistence, you know exactly where to invest.

Tuning and False Positives

An alert that fires constantly gets ignored—the worst outcome in detection. Tune rules with context and allowlisting, but do not over-suppress. Measure each rule's precision and volume; a rule that fires thousands of times a day and is always benign is worse than no rule, because it trains analysts to dismiss real signals. Maintain a runbook per detection so any analyst can triage it consistently.

The Detection Lifecycle

Detection is iterative, not a one-time build:

  1. Hypothesize a threat (from threat modeling, incidents, or ATT&CK gaps).
  2. Instrument the data needed to observe it.
  3. Write and test a rule against both malicious and benign scenarios.
  4. Deploy with a runbook.
  5. Measure precision and recall.
  6. Retire or refine as the environment and adversary evolve.

Purple teaming—where red and blue teams collaborate to test whether a technique is actually detected—is the most efficient way to drive this loop.

Response Integration

Detection without response is just documentation of a breach. Connect alerts to playbooks (SOAR): automatic enrichment, isolation of a host, revocation of credentials, and notification of the on-call analyst. The faster the loop from alert to containment, the smaller the blast radius.

Metrics

Track mean time to detect (MTTD) and mean time to respond (MTTR), alert volume and precision, and coverage by ATT&CK technique. These numbers tell you whether your detection program is improving or just accumulating rules. Good detection engineering is measurable, adversarial, and relentless about turning unknown unknowns into known signals.