Chapter 87 · Examples

Security Monitoring — Worked Examples

Five applied scenarios: SIEM correlation catching a credential attack, data transfer monitoring detecting exfiltration, inadequate archiving destroying breach evidence, alert fatigue from poor tuning, and a quarantine response containing a ransomware incident.

Example 1 — SIEM Correlation Catches a Credential Attack Invisible in Individual Logs
Scenario: Over 90 minutes on a Thursday night, an attacker uses a credential-stuffing tool to test 4,800 username and password combinations against the organization's VPN gateway. The attempts are distributed across 15 different source IPs to avoid triggering per-IP lockout thresholds. No individual server logs more than 350 failed logins — below the threshold for a per-source alert on any single device. At 2:17 AM, one set of credentials succeeds and the attacker logs in.

Why individual logs missed it: Each VPN server's authentication log shows failed logins from various IPs, all below per-source alert thresholds. Without correlation, the logs look like normal background noise. No single log reveals that 4,800 attempts against the same account namespace over 90 minutes from 15 coordinated IPs is a systematic attack.

What the SIEM sees: The SIEM aggregates authentication logs from all VPN servers. Its correlation rule detects: the same set of usernames being tested across multiple source IPs within the same time window. The rule fires at the 500th unique username attempt and generates a Critical alert at 1:44 AM — 33 minutes before the successful login. The on-call analyst receives an SMS and investigates. The geographic origin of the IPs (Eastern Europe) is flagged against the employee directory (all employees are domestic). The analyst implements a geo-block for the source country and prevents the eventual login that the log-only approach would have missed entirely.

The lesson: The attack was invisible at the individual log level. SIEM correlation across all authentication sources — connecting low-level signals from multiple systems into a pattern — made it visible. This is the core value proposition of log aggregation: attacks that span systems, time, and distributed sources become detectable only when the data is centralized and correlated.

Example 2 — Data Transfer Monitoring Detects Exfiltration in Progress
Scenario: An organization's financial database server typically transfers between 2 GB and 8 GB of data per day over its normal backup and reporting operations. The SIEM has a baseline-derived alert rule: alert if this server's outbound transfer volume exceeds 20 GB in a 4-hour window. On a Friday at 4:30 PM, the SIEM fires a High alert: the database server has transferred 47 GB in the past 3 hours to an external IP address the organization has never communicated with before.

What the alert revealed: An attacker had compromised the database server three weeks earlier via a SQL injection vulnerability in an internal application. For three weeks, the attacker performed quiet reconnaissance — reading table schemas, understanding what data was present, and identifying the most valuable records. On Friday afternoon (when IT staffing is at its lowest), the attacker began bulk-exfiltrating customer financial records to a cloud storage service they controlled.

The response: The on-call analyst confirmed the transfer was not a scheduled backup (all legitimate backups were to known internal addresses). The server was quarantined at 4:51 PM — 21 minutes after the alert fired. The exfiltration stopped. Approximately 2.1 million customer records had already been transferred. Forensic investigation using archived logs from the past three weeks reconstructed the full attack timeline, including the initial compromise three weeks earlier.

The lesson: The attack was only detected because a data transfer baseline existed and an alert threshold was configured. Without the baseline and alert, the exfiltration would have completed undetected over the weekend. The archived logs from the three-week dwell period were essential for the forensic investigation — without archiving, the initial compromise would have been unrecoverable.

Example 3 — Insufficient Log Retention Destroys Breach Evidence
Scenario: A healthcare organization discovers in November that patient records have been accessed and exfiltrated. Forensic investigators determine that the initial compromise occurred in approximately March of the same year — eight months before discovery. The organization retains firewall and authentication logs for 90 days. When investigators request the March through August authentication and network logs to reconstruct the attacker's initial access and lateral movement, they are told the logs no longer exist. They were purged after 90 days as part of automated retention cleanup.

What was lost: Evidence of: (1) the initial compromise vector (the investigators cannot determine how the attacker got in, making it impossible to close that entry point with confidence); (2) which specific patient records were accessed and when (the healthcare organization cannot accurately notify affected patients as required by HIPAA breach notification rules); (3) whether other systems were compromised beyond what was found (the investigators have no visibility into lateral movement that occurred before the 90-day window).

The regulatory consequence: HIPAA requires healthcare organizations to maintain security activity records for six years. The organization's 90-day retention policy was not only operationally inadequate — it was a regulatory violation. The organization faced regulatory scrutiny both for the breach and for the failure to maintain required records.

The lesson: The IBM 2022 nine-month average breach discovery time is not an abstract statistic. It directly determines the minimum log retention period needed for forensic investigation to be possible. A 90-day retention policy guarantees that any breach with a dwell period longer than 90 days will have its initial evidence destroyed before it is discovered. Retention policy must be set against the realistic dwell-time expectation, not operational convenience.

Example 4 — Alert Fatigue From Poor Tuning Causes a Real Threat to Be Missed
Scenario: A security team deploys a new SIEM with default alert rules. Within the first week, the SIEM generates an average of 2,400 alerts per day. The three-analyst security team quickly finds it impossible to investigate each alert. After two weeks, analysts begin acknowledging and closing alerts without full investigation — there are simply too many. In week four, a genuine ransomware pre-deployment alert (reconnaissance activity consistent with the attacker preparing to detonate ransomware) fires but is closed by an exhausted analyst who assumed it was another false positive. The ransomware deploys the following morning.

What went wrong: The SIEM's default rules were not tuned to the organization's environment. Alerts were firing on normal business activity that happened to match generic patterns. The high false positive rate caused alert fatigue: analysts lost confidence in the alerts and began dismissing them reflexively rather than investigating. The real threat signal drowned in noise.

The alert tuning failure: Alert tuning is the process of adjusting thresholds and rules to match the organization's actual baseline. In this environment, the team should have: (1) spent the first two to four weeks in observation mode, learning normal traffic patterns before enabling high-volume alerts; (2) progressively enabled alert rules starting with highest-confidence detections; (3) tuned thresholds based on the first weeks of false positive data. Default rules are a starting point, not a finished configuration.

The lesson: Too many false positives are nearly as dangerous as too many false negatives. An overloaded analyst who dismisses every alert without investigation creates a system that is monitoring in name only. Alert tuning is the ongoing process that keeps alert volume manageable, confidence high, and real threats visible.

Example 5 — Quarantine Containing Ransomware Before It Spreads
Scenario: At 10:43 PM on a Saturday, an endpoint detection and response (EDR) tool on a user laptop identifies behavior consistent with ransomware execution: rapid file encryption activity, deletion of shadow copies, and attempted connections to a known ransomware command-and-control domain. The EDR generates an alert that flows to the SIEM, which forwards an SMS to the on-call analyst. The analyst confirms the behavior in the SIEM dashboard and issues a quarantine command to the EDR platform at 10:51 PM — eight minutes after the initial detection.

What quarantine accomplished: The laptop was removed from all network communication at 10:51 PM. At that point, the ransomware had encrypted files on the local laptop and had begun attempting to reach shared network drives. The network connections to shared storage were severed before the ransomware could encrypt shared data. Approximately 4,200 local files on the laptop were encrypted. No shared network storage was affected.

Without quarantine: Ransomware typically spreads from an initial endpoint to all reachable network shares within minutes to hours, depending on network speed and the ransomware variant's propagation behavior. Without quarantine, the 10:51 PM network cut would not have occurred. By morning, the ransomware would likely have encrypted shared drives accessible to the compromised account — potentially hundreds of thousands of files.

The monitoring chain that made this work: EDR detection → SIEM aggregation → alert correlation → SMS notification → analyst investigation → quarantine command. Each link in the chain had to function: detection without alerting is useless; alerting without a responding analyst is useless; an analyst without quarantine capability is useless. Security monitoring is a system, not a single tool.