Logging, Detection & SIEM

You Cannot Stop What You Cannot See#

Prevention fails eventually. When it does, detection decides whether an intrusion is caught in hours or discovered months later after the damage is done. Detection engineering is the discipline of collecting the right signals, turning them into reliable alerts, and wiring those alerts to a fast response. It is measured, adversarial work, not a one-time install.

Log Sources That Matter#

Detection starts with comprehensive, centralised logging. The high-value sources:

• Authentication: successful and failed logins, MFA prompts and denials, password resets, new device enrolments.
• Endpoints: process creation with command lines and parent process, file writes, registry changes, script execution. Sysmon (Windows) and EDR agents provide this. See malware types.
• Network: DNS queries, flow records, proxy logs, firewall denies.
• Cloud: control-plane API calls (CloudTrail, Azure Activity, GCP Audit Logs), IAM changes, storage access. See cloud security.
• Applications: web access and error logs, and audit trails of sensitive actions (role changes, exports, payments).

A single source is a fragment; correlation across sources is where detection power lives. A failed login alone is noise. A failed login, then a success, then a new IAM key created from an unfamiliar IP, then a bulk download, is a story.

Centralization and Normalization#

Logs scattered across hosts are useless under pressure. Ship them to a central platform, a SIEM (Splunk, Elastic Security, Microsoft Sentinel, Google Chronicle) or a log lake, and normalise fields so a query for "source IP" works across web logs, VPN logs and cloud logs that each name it differently. Two prerequisites are non-negotiable:

• Time synchronisation (NTP). Without consistent, ideally UTC, timestamps, cross-source correlation is impossible. See network time and synchronization.
• Tamper resistance. Ship logs off the host promptly and store them so an attacker who compromises a machine cannot edit or delete its history. This is also an incident response requirement.

Detection Rules and Signals#

Detections come in a few flavours, and a mature program uses all of them:

Map rules to MITRE ATT&CK techniques. Doing so gives a shared vocabulary and, crucially, reveals coverage gaps: if you have no detection for credential dumping, lateral movement or persistence, ATT&CK tells you exactly where to invest next.

A Worked Example: A Correlation Rule#

Real rules run inside a SIEM query language, but expressing one in Python makes the logic clear. This flags an account that fails to log in several times and then succeeds from a new IP within a short window, the classic brute-force or credential-stuffing success:

import csv
from collections import defaultdict
from datetime import datetime, timedelta

WINDOW = timedelta(minutes=10)
FAIL_THRESHOLD = 5

events = defaultdict(list)   # user -> list of (time, outcome, ip)
for row in csv.DictReader(open("auth.csv", encoding="utf-8")):
    t = datetime.fromisoformat(row["utc_time"])
    events[row["user"]].append((t, row["outcome"], row["ip"]))

for user, evs in events.items():
    evs.sort()
    known_ips = set()
    for i, (t, outcome, ip) in enumerate(evs):
        if outcome == "success":
            recent_fails = [e for e in evs[:i] if e[1] == "fail" and t - e[0] <= WINDOW]
            if len(recent_fails) >= FAIL_THRESHOLD and ip not in known_ips:
                print(f"ALERT {user}: success from new IP {ip} after "
                      f"{len(recent_fails)} fails at {t.isoformat()}")
            known_ips.add(ip)

An alert line such as ALERT dana: success from new IP 203.0.113.9 after 7 fails at 2026-09-30T14:03:00 is high-signal because it combines three weak indicators (failures, a success, a new IP) into one strong one. This also shows why raw thresholds alone are poor: "5 failed logins" fires constantly on fat-fingered passwords, but "5 failures then success from a new IP" rarely fires on anything benign. Production versions add allowlisting for known corporate IPs, awareness of impossible travel, and enrichment from threat intelligence.

Tuning and False Positives#

An alert that fires constantly gets ignored, which is the worst outcome in detection: a noisy rule trains analysts to dismiss the very signal you built it for. For each rule, measure its precision (of the alerts it raised, how many were real) and its volume. A rule that fires a thousand times a day and is always benign is worse than no rule. Tune with context and allowlisting, but do not over-suppress until you have hidden the real thing. Maintain a runbook per detection so any analyst can triage a given alert the same way: what it means, what to check, when to escalate.

The Detection Lifecycle#

Detection is iterative:

1. Hypothesise a threat, from threat modeling, a past incident, or an ATT&CK coverage gap.
2. Instrument the data needed to observe it (you cannot detect what you do not log).
3. Write and test the rule against both malicious and benign samples.
4. Deploy with a runbook.
5. Measure precision and volume in production.
6. Refine or retire as the environment and adversary change.

Purple teaming, where a red team executes a technique and the blue team checks whether it was detected, is the most efficient way to drive this loop, because it tests real detections against real attacker behaviour rather than assumptions.

Response Integration#

Detection without response is just a well-documented breach. Connect alerts to response, increasingly via SOAR playbooks that automate the mechanical steps: enrich the alert (look up the IP, the user, the asset), then, for high-confidence detections, isolate the host, disable the account or revoke the token, and page the on-call analyst. The faster the loop from alert to containment, the smaller the blast radius. Keep destructive automated actions gated behind high confidence so a false positive does not itself cause an outage.

Where Detections Come From#

You do not have to invent every rule. A practical detection library is assembled from several streams and then tuned to your environment:

• Sigma rules: a vendor-neutral YAML format for describing log-based detections, with a large open ruleset that converts to Splunk, Elastic, Sentinel and other query languages. Start here for common techniques rather than writing from scratch.
• Vendor and cloud content: managed detections shipped with your EDR, SIEM or cloud threat-detection service cover the well-known techniques out of the box.
• Your own incidents: every real incident should produce at least one new detection so the same technique is caught automatically next time.
• Threat intelligence: indicator feeds are useful for known-bad domains and hashes, but they go stale quickly, so weight behavioural detections above pure indicator matching.

Whatever the source, treat an imported rule as a draft: test it against your own benign traffic before trusting its precision, because a rule tuned for someone else's network is often noisy on yours.

Metrics That Tell You If It Is Working#

• MTTD (mean time to detect) and MTTR (mean time to respond), trending down over time.
• Alert volume and precision per rule, to find the noisy ones.
• Coverage by ATT&CK technique, to see where you are blind.

These numbers distinguish a detection program that is improving from one that is merely accumulating rules. Good detection engineering is measurable, adversarial, and relentless about turning unknown activity into known, actionable signals.

Practice#

Take an auth log you can access (or generate one) and implement the correlation rule above, then tune it: add an allowlist of known office IPs and measure how the alert count drops. Next, pick one ATT&CK technique you currently cannot detect, for example a document spawning PowerShell (see malware types), write a detection for it against Sysmon event ID 1, and test it by safely reproducing the behaviour in a lab.