Key Takeaways
- Ticket‑centric metrics (e.g., tickets‑closed‑per‑hour) reward speed over accuracy and can cause analysts to close real attacks while chasing noise.
- Volume‑based measures such as rule count or log‑volume create alert inflation and give a false sense of coverage without improving detection.
- The only outward‑facing metric that truly reflects SOC effectiveness is time‑to‑detect (TTD) or time‑to‑respond (TTR); all others can be gamed.
- Red‑team, purple‑team exercises and MITRE ATT&CK‑decomposed test cases provide a controllable denominator for TTD when real intrusions are rare.
- Any metric that is watched—whether reported inward or outward—shifts analyst behaviour; therefore, non‑essential counts should be eliminated from all dashboards.
- Rebuilding a SOC around detection requires giving analysts time for hypothesis‑led hunting, setting strict false‑positive thresholds, and tracking skill‑based development metrics rather than throughput.
- Validation through sector‑specific purple‑team exercises confirms that the rebuilt SOC can detect simulated adversaries within the agreed TTD, turning metric reform into measurable improvement.
The ticket‑driven SOC trap
The NCSC’s latest advisory highlights a recurring pattern: analysts sit in front of a ticket queue where ≈ 99 % of alerts are false positives, yet their performance is judged by tickets‑closed‑per‑hour. Because each click—whether it dismisses noise or a genuine intrusion—increases the metric, analysts are incentivised to close tickets as fast as possible, even when that means shutting down real attacks alongside the noise. This mismatch between measurement and mission turns a well‑staffed, well‑tooled SOC into a blind spot for live threats.
Why traditional metrics mislead
The root cause is governance, not capability. Many SOCs inherit their key performance indicators from IT service desks, customer‑support teams, or development groups that share the same ticketing system. Metrics such as “tickets processed per shift” or “average time‑to‑close” are intuitive for non‑security executives and fit neatly on board dashboards, but they were never designed for an environment where the signal‑to‑noise ratio is extremely low. Optimising for these numbers therefore drives behaviour that is antithetical to effective threat detection.
Volume‑based metrics and alert inflation
A second class of broken metrics focuses on sheer volume: the number of detection rules or the amount of log data ingested. The NCSC observed SOCs creating a separate rule for every individual indicator of compromise (e.g., each IP address), inflating rule counts without improving coverage. Likewise, collecting massive log feeds—sometimes only the first 30 characters of a critical source—looks impressive on a capacity report but remains useless if no one is measured on whether those logs actually generate actionable alerts. Volume becomes a vanity metric that masks gaps in detection capability.
Time‑to‑detect as the only viable outward metric
The NCSC sanctions a single outward‑facing measure: whether the SOC detects and responds to attacks in a timely manner, expressed as time‑to‑detect (TTD) or time‑to‑respond (TTR). Unlike ticket‑count or rule‑count metrics, TTD/TTR directly reflects the core mission of a security operation centre. The challenge is that, in a healthy organisation, successful intrusions are rare, providing an insufficient natural denominator for the metric. To overcome this, the advisory recommends manufacturing a denominator through controlled adversarial testing, thereby turning TTD into a reliable, comparable indicator of SOC performance.
Using red‑team, purple‑team and MITRE ATT&CK to measure TTD
Red‑team exercises emulate an attacker’s covert posture, testing both detection and response capabilities. Purple‑team sessions sacrifice stealth for transparency, allowing the SOC to see exactly which attack steps triggered alerts and which did not. By decomposing adversary behaviour into individual MITRE ATT&CK techniques and testing each step in isolation, a SOC obtains a reproducible TTD per technique rather than a single opaque aggregate score. This granularity enables targeted improvements and provides defensible evidence to leadership that the SOC can find real threats when they occur.
Behavioural impact of any metric
The advisory stresses a counter‑intuitive truth: any metric that is watched—whether intended for internal health monitoring or external reporting—will influence analyst behaviour. Once analysts know a number is being tracked, they will optimise their actions to improve it, even if that optimisation harms the underlying security goal. Consequently, the NCSC advises that ticket counts and rule counts should not be reported at all, neither inward nor outward. By defining TTD/TTR as the sole board‑facing measure, the organisation removes the perverse incentive to game upstream counters and focuses attention on the outcome that truly matters.
Rebuilding the SOC: culture and investigation time
With the metric reset in place, the NCSC outlines a cultural shift: give analysts the time and authority to investigate before they are pressured to close tickets. This enables hypothesis‑led threat hunting, where an analyst forms a supposition based on threat intelligence about a likely attacker technique and then searches logs for evidence. Although most hunts return negative results, the real value lies in the new detection rule, hardening recommendation, or procedural improvement that emerges from the investigation—an activity that ticket‑throughput metrics make impossible.
Managing false positives and rule quality
To keep the alert stream tractable, the NCSC recommends setting hard false‑positive thresholds on detection rules. A worked example involves a rule for PowerShell execution by non‑IT staff; analysts iteratively refine the rule by working down the remaining false positives until any new PowerShell event is either an attack or a documented exception. Fortnightly or monthly false‑positive reviews turn a “noisy rule” into a “rule that needs different logic,” preventing the accumulation of ignored alerts and ensuring that rule‑based detections stay relevant and actionable.
Analyst development metrics and validation
Instead of counting rules or tickets, the SOC should track analyst‑development indicators such as threat awareness, tool expertise, and organisational fluency. Metrics like ATT&CK technique coverage, red‑team escape rates, and the depth of an analyst’s engagement with IT operations and business owners provide insight into whether the team understands what “normal” looks like—a prerequisite for spotting abnormal behaviour. These signals can sit on an internal manager’s dashboard, while the board continues to see only TTD. Finally, the rebuild is validated with a sector‑specific purple‑team exercise: select adversary playbooks relevant to the organisation, run each ATT&CK step in isolation, and measure how many fire alerts that analysts escalate within the agreed TTD. The exercise is used to refine the SOC, not to score it, ensuring that the metric reshapes behaviour toward genuine detection capability.