Skip to content
Tutorials Center
My learning

Topics

Lesson 62 of 80 ยท Security Operations and Incident Response

Security Metrics and KPIs

Learn which security metrics matter: operational, incident-response, risk and compliance KPIs such as MTTD and MTTR, how to compute them, and how to avoid misleading numbers.

"We are secure" is not a measurement. Leaders fund security with limited budgets, and they need evidence about where the organization stands, whether it is improving and where the next euro or hour should go. Security metrics turn the work of the team into numbers that can be tracked over time, and KPIs (key performance indicators) are the few metrics chosen because they tie directly to a goal.

Good metrics change decisions; bad ones decorate slides. This lesson shows the categories of security metrics, defines the most common ones (including MTTD and MTTR), calculates several from sample data in Python, and warns about pitfalls such as averages that hide outliers and numbers that reward the wrong behaviour. It connects to Vulnerability Management and Patching, The Incident Response Process and Compliance and Security Frameworks.

Metrics, KPIs and KRIs

  • A metric is any measurement ("number of phishing emails reported this month").
  • A KPI shows how well a function is meeting a goal ("95% of critical vulnerabilities fixed within 14 days").
  • A KRI (key risk indicator) warns that risk is rising ("share of internet-facing systems without MFA").
  • Leading indicators predict problems (unpatched systems, missing logs); lagging indicators report what already happened (incidents, breaches). You need both.

A useful metric is meaningful (linked to a goal), measurable from reliable data, actionable (someone can change it), comparable over time, and understood by its audience. A good starting method is: goal, question, metric. For example, the goal "reduce exposure to known exploited flaws" leads to the question "how fast do we fix them?", which gives the metric "median days to remediate known-exploited vulnerabilities".

Categories of metrics

Category Questions it answers Example metrics
Operational Are controls working day to day? Patch coverage, percentage of assets with endpoint protection, MFA coverage, backup success rate, firewall rule review age
Detection and response How quickly do we find and handle incidents? MTTD, MTTR, MTTC, alerts per analyst, false-positive rate, incidents by severity
Vulnerability management How exposed are we? Open critical vulnerabilities, age of oldest critical, SLA compliance, scan coverage
Risk How much risk remains? Risks above appetite, risk register trend, third-party risk scores, residual risk by business unit
Compliance Do we meet obligations? Controls passing, audit findings open and overdue, policy exceptions, training completion
People and awareness Do users help or hurt? Phishing-simulation click and report rates, training completion, time to report suspicious email
Program and cost Are we spending wisely? Security spend as a share of IT, cost per incident, tool utilization

Detection and response metrics

These are the best-known security KPIs. They describe segments of an incident's timeline:

  • MTTD (mean time to detect): from the start of the compromise (or first malicious activity) until it is detected.
  • MTTA (mean time to acknowledge): from alert to an analyst starting work.
  • MTTC (mean time to contain): from detection until the threat can no longer spread.
  • MTTR (mean time to respond or recover): from detection until the incident is resolved or systems are restored. The "R" is used for respond, remediate or recover in different places, so define it explicitly.
  • Dwell time: how long an attacker remains in the environment before being removed. It is close to MTTD plus time to contain.

Shorter is generally better, because damage grows with time (The Attack Lifecycle and MITRE ATT&CK). Note that the start of compromise is often only known after investigation.

Python Compute MTTD, MTTC and MTTR from incident records

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at MTTD: the mean is dragged up by the single incident that sat undetected for two weeks, while the median shows the typical case. Report both, plus the worst case, and talk about the outlier separately rather than letting one average hide it.

Vulnerability and operational metrics

For vulnerability management, a clear KPI is SLA compliance: set deadlines by severity (for example critical within 7 days, high within 30 days) and measure the share of findings fixed in time. Combine it with exposure metrics such as the age of the oldest open critical issue and coverage metrics (what share of assets is scanned at all). A scanner that covers 60% of your assets says nothing about the other 40%.

Python Patch SLA compliance and coverage

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A "percentage fixed" figure can look good while the one overdue critical issue that matters is still open, so keep the aging and overdue counts next to it.

Risk, compliance and awareness metrics

  • Risk: the number of risks above the organization's appetite, trend of the risk register, and exposure by business unit (Threats, Vulnerabilities and Risk).
  • Compliance: percentage of controls implemented and passing, open and overdue audit findings, policy exceptions and their age (Compliance and Security Frameworks).
  • Awareness: training completion and phishing-simulation results. The most useful number is not the click rate alone but the report rate and the time to report, because a user who reports quickly is a sensor (Phishing and Social Engineering).
  • Third parties: share of critical suppliers assessed, and findings open (Dependency and Supply Chain Security).
  • Resilience: backup restore tests passed and measured restore times against the recovery objectives (Backups, Resilience and Business Continuity).
  • Detection engineering: share of ATT&CK techniques covered by tested detections, and alert precision (true positives divided by all alerts), which shows how much analyst time is wasted.

Presenting metrics

  • Know the audience. Executives want risk, trend and decisions needed; managers want performance and bottlenecks; engineers want detail they can act on.
  • Show trends and targets, not single numbers: "median time to patch criticals fell from 21 to 9 days, target 7".
  • Few and stable: a dashboard of five to ten KPIs reviewed regularly beats fifty that nobody reads. Keep definitions fixed so trends are valid.
  • Give context: denominators, scope and data quality. "12 incidents" means little without knowing the number of systems or the previous quarter.
  • Document each metric: definition, formula, data source, owner, frequency and target.
  • Automate collection from scanners, ticketing, identity and SIEM systems, so reporting does not depend on manual spreadsheets.

Pitfalls

  • Vanity metrics: counting blocked attacks or scans ("we blocked 4 million attacks!") says little about risk, because most are automated noise.
  • Gaming (Goodhart's law): when a measure becomes a target it stops being a good measure. If analysts are rewarded for closing tickets fast, they may close them without investigating; if teams are punished for reporting incidents, reports will vanish. Pair speed metrics with quality metrics (reopen rate, post-incident review findings).
  • Means hiding outliers: use medians, percentiles and worst cases, as in the demo above.
  • Zero is not safe: "zero incidents" may mean zero detections; compare with detection coverage and red-team results.
  • Incomplete denominators: a patch rate over only the assets you know about ignores the unknown ones (shadow IT).
  • Changing definitions: redefining "critical" or "incident" mid-year breaks trend lines.
  • Measuring activity instead of outcome: hours of training delivered versus reduced risky behaviour.
  • Ignoring uncertainty: risk numbers from scoring models are estimates; label them as such.
Python Mean vs median, and a gamed metric

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A starter set of KPIs

If you must pick a handful, start with these, and adjust to your goals:

  1. Median and 90th-percentile time to detect and to contain incidents.
  2. Share of critical and high vulnerabilities fixed within SLA, plus the age of the oldest critical.
  3. Coverage: percentage of assets with EDR, logging, MFA and scanning.
  4. Phishing report rate and time to report.
  5. Backup restore test success and restore time versus objective.
  6. Open risks above appetite and overdue audit findings.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does MTTD measure?

    MTTD, mean time to detect, covers the gap between the beginning of malicious activity and the moment it is discovered.
  2. 2. Why should you report medians or percentiles alongside means for response times?

    Outliers such as a two-week dwell time pull the mean up. The median shows the typical case, and the worst case should be discussed separately.
  3. 3. A team is rewarded for closing alerts quickly and closure times drop, but reopened alerts rise. What does this illustrate?

    People optimize what is measured. Pair speed with quality measures such as the reopen rate.
  4. 4. Which is an example of a leading indicator?

    Leading indicators describe conditions that predict future incidents. Breaches, costs and fines are lagging results.

Summary

  • Metrics become KPIs when tied to a goal; KRIs warn about rising risk; you need leading and lagging indicators.
  • Core categories: operational, detection and response, vulnerability, risk, compliance, awareness and cost.
  • MTTD, MTTA, MTTC and MTTR describe the incident timeline; define each clearly and track dwell time.
  • Combine SLA compliance with aging, overdue counts and coverage so good averages do not hide problems.
  • Present few, stable, documented metrics with trends, targets and context, and automate their collection.
  • Avoid vanity metrics, gaming, misleading means and shifting definitions; pair speed with quality.

Next: Compliance and Security Frameworks, where metrics meet audits and standards.