Skip to content
Tutorials Center
My learning

Topics

Lesson 58 of 80 ยท Security Operations and Incident Response

The Incident Response Process

Learn the incident response lifecycle: preparation, detection, containment, eradication, recovery and lessons learned, with runbooks and communication.

Sooner or later something gets through: a stolen password, an infected laptop, a misconfigured storage bucket. What separates a bad day from a disaster is usually not the attack itself but how the team responds. People who have rehearsed a plan act calmly and in the right order. People who improvise tend to wipe the evidence, tell the wrong people, or fix the symptom while the intruder is still inside.

Incident response (IR) is the organized approach to handling a security incident so that damage is limited and recovery is fast and cheap. In this lesson you will learn the standard lifecycle, what each phase involves, how to write a runbook, and how to communicate during an incident. The code is small stand-ins that model severity and timing, because a real incident cannot be simulated in a browser. You may want to read Spotting Attacks in Logs first, since detection is where most incidents begin.

Event, alert, incident

These words are not interchangeable:

  • An event is anything observable: a login, a file change, a firewall drop.
  • An alert is an event (or a group of them) that a rule flagged as interesting.
  • An incident is an event that actually threatens, or already harms, the confidentiality, integrity or availability of your systems or data. See The CIA Triad and Core Principles.

Most alerts are not incidents. Deciding which ones are is called triage, and doing it well is a core skill.

The lifecycle

The best-known model comes from the US National Institute of Standards and Technology (NIST), which describes the work as a cycle in four major parts. Other frameworks, such as SANS, split the same work into six steps. They agree on the substance:

flowchart LR
    P["1. Preparation"] --> D["2. Detection<br/>and analysis"]
    D --> C["3. Containment,<br/>eradication, recovery"]
    C --> L["4. Post-incident<br/>activity"]
    L -.-> P
    D -. "new facts" .-> D
    C -. "new facts" .-> D

Incident response is a cycle: lessons from each incident feed the next round of preparation. Analysis often loops back as the team learns more.

1. Preparation

Most of the value of IR is created before anything happens.

  • A written plan and policy that says who may declare an incident and who is in charge.
  • A team with named roles, backups for each role and an on-call rota. Typical roles: incident commander, technical lead, communications lead, scribe.
  • Contact lists that work when email is down: phone numbers for staff, legal counsel, your cloud provider, insurers and law enforcement.
  • Tools and access ready: logging that is already switched on, forensic tooling, isolated "jump" machines, break-glass admin accounts stored safely.
  • Asset inventory and data classification, so you know what a compromised server holds.
  • Exercises. A tabletop exercise, where the team talks through a scenario around a table, finds gaps cheaply.

2. Detection and analysis

Signals arrive from monitoring, from antivirus or EDR tools, from users ("this email looks odd") and sometimes from outsiders (a customer, a researcher, the press). The team then:

  1. Validates: is this real, or a false positive?
  2. Scopes: which accounts, hosts and data are affected?
  3. Classifies severity and assigns an owner.
  4. Records everything, with timestamps, in one place. The log of what was seen and done becomes the incident timeline. For evidence handling, see Digital Forensics Basics.

A severity scheme keeps the response proportional. The stand-in below scores an incident from two questions: how sensitive is the affected data, and how widely has the attacker spread.

Python A small severity triage helper

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The numbers are arbitrary, and your organization will choose its own. What matters is that the rule is written down, so two analysts classify the same incident the same way and so that the people who must be woken up know why.

3. Containment, eradication and recovery

Once you know it is real, you act in this order:

  • Containment stops the damage from growing. Short-term containment is fast and blunt: disconnect a host from the network, disable an account, block an address. Long-term containment keeps the business running while you prepare a clean fix, for example by moving traffic to a clean system.
  • Eradication removes the cause: delete malware, close the vulnerability that let the attacker in, reset every credential the attacker could have seen, remove persistence such as new accounts or scheduled tasks.
  • Recovery restores normal operation from known-good sources: rebuild from clean images, restore from verified backups (see Backups, Resilience and Business Continuity), then watch the systems closely for signs of the attacker returning.

Two classic mistakes happen here. The first is powering off or wiping a machine too early, which destroys volatile evidence such as memory contents and active connections. Isolate it from the network first, and decide with your forensic lead whether to capture memory. The second is eradicating too soon: if you only remove the one backdoor you found while the attacker has three others, they simply return, and now they know you are watching. Scope fully before you evict.

4. Post-incident activity

Within a week or two of closing the incident, hold a blameless post-incident review. Ask what happened, what worked, what slowed you down and what will change. Turn the answers into tracked actions with owners and due dates: a new detection rule, a patch process fix, a training item. Update the plan and the runbooks. An incident that produces no changes will happen again.

Runbooks

A runbook (or playbook) is a short, step-by-step procedure for one kind of incident: phishing report, lost laptop, compromised account, ransomware. It lets a tired person at 3 a.m. do the right thing without having to remember everything. A good runbook has a trigger, roles, ordered steps, decision points, and the evidence to capture. Keeping it as structured data makes it easy to version, review and even check automatically.

Python A compromised-account runbook as data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the order inside the runbook: contain before you analyze. Disabling the account is cheap and reversible; a long investigation while the attacker still has a session is not.

Communication

Technical work is only half the job. Decide in advance:

  • Who is the single voice inside the team (the incident commander) and who talks to executives, staff, customers and the press (the communications lead). Everyone else sends updates through them.
  • A secure side channel, because the attacker may be reading company email and chat. Use a pre-agreed out-of-band channel.
  • Need-to-know. Share details with those who must act. Rumors and speculation cause harm.
  • Legal and regulatory duties. Many laws require notifying regulators or affected people within a fixed time after a personal-data breach (the EU GDPR sets 72 hours for notifying the authority, once you are aware of it). Involve legal counsel early. See Privacy and Data Protection.
  • Honest, regular updates: what we know, what we do not know, what we are doing, when the next update comes.

Measuring response

Teams track a few simple durations: time to detect (attack start to alert), time to respond (alert to first action) and time to recover (alert to normal service). Shorter is better, but the exact definitions matter less than using the same ones every time. The next stand-in computes them from a timeline.

Python Response metrics from an incident timeline

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The biggest number here is the time to detect: almost seven hours of unnoticed attacker activity. That is common, and it is why investment in monitoring and good alerts often pays more than speeding up an already quick response.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is the difference between an alert and an incident?

    Many alerts are false positives or harmless. Triage decides which of them are real incidents that need a response.
  2. 2. Why is it risky to immediately power off or wipe a suspected compromised server?

    Isolate the host from the network first, so it cannot cause more damage, and decide how to capture evidence before shutting it down or rebuilding it.
  3. 3. What is the main danger of eradicating too early?

    Full scoping before removal ensures that you evict the attacker once, completely, rather than starting a cat-and-mouse game.
  4. 4. What is the purpose of a blameless post-incident review?

    Blame discourages honest reporting. The goal is to find process and technology gaps and fix them with owners and deadlines.
  5. 5. Why should the team use an out-of-band channel during a serious incident?

    If the attacker controls or monitors normal communication systems, discussing the response there tips them off.

Summary

  • An event is observable, an alert is flagged, and an incident actually threatens security. Triage separates them.
  • The lifecycle is preparation, detection and analysis, containment/eradication/recovery, and post-incident review, repeated as a cycle.
  • Contain first, preserve evidence, scope fully, then eradicate and recover from known-good sources.
  • Runbooks turn lessons into step-by-step procedures that work under stress; keep them as reviewed, versioned documents.
  • Communicate through defined roles and a safe channel, and know your legal notification duties.
  • Measure time to detect, respond and recover, and turn every review into tracked improvements.

Next: Digital Forensics Basics, which shows how to handle and analyze the evidence an incident leaves behind.