Skip to content
Tutorials Center
My learning

Topics

Lesson 21 of 37 ยท IDS/IPS and Monitoring

Signature-Based Detection

Understand how signature-based intrusion detection works, write simple pattern rules, see why attackers evade them, and learn how to normalize input before matching.

A signature is a description of something known to be bad: a byte pattern, a header combination, a URL fragment or a sequence of events. Signature-based detection compares live traffic against a library of such descriptions. It is the oldest detection method, and still the backbone of most IDS and IPS deployments, because it is fast, explainable and has few false positives when the rules are good.

In this lesson you will build a small signature engine, see how simple evasion tricks defeat naive matching, and learn the fix that real engines use: normalization.

Anatomy of a signature

A useful signature answers four questions:

  1. Where do I look? (protocol, port, direction, a particular field such as the HTTP URI)
  2. What do I match? (a literal string, a regular expression, a byte sequence)
  3. How sure am I? (severity, classification, references to the vulnerability)
  4. What do I do? (alert, log, drop)

Good signatures match the vulnerability (what the exploit needs in order to work) rather than one specific exploit sample. A rule keyed to the exact bytes of one published exploit is trivially bypassed by changing a byte; a rule that describes "a header longer than 200 bytes in this field" catches every variant.

flowchart LR
    P["Packet / request"] --> N["Normalize"]
    N --> M{"Match rules"}
    M -- "no hit" --> D["Deliver"]
    M -- "hit" --> A["Alert / drop"]

Real engines decode and normalize first, then match.

A minimal signature engine

The stand-in below treats each request as text and checks it against rules described as data. Each rule has a name, a regular expression, a field and a severity. Nothing here touches the network.

Python A small signature engine

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Four of five requests raise an alert and the clean one passes. That looks great until an attacker tries to hide.

Evasion: why naive matching fails

Because attackers know signatures exist, they transform payloads so the pattern no longer matches while the target still understands them. Defensive teams need to know the classic tricks, because they explain why engines do so much work before matching.

  • Encoding. ../ can be written %2e%2e%2f (URL encoding), or doubly encoded %252e%252e%252f. The web server decodes it; a literal match on ../ does not see it.
  • Case and whitespace. SeLeCt, tabs or comments between SQL keywords.
  • Fragmentation. Splitting the payload over several TCP segments or IP fragments so no single packet contains the whole pattern. Engines fix this by reassembling the stream.
  • Protocol quirks. Extra slashes (//), /./ segments, mixed \ and /, odd header folding.

The next program shows an encoded request slipping past the rules, then caught after normalization.

Python Normalization defeats encoding tricks

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The naive column misses the encoded variants; the normalized column catches all four attacks and leaves the harmless PDF alone. The lesson: match on what the target will see, not on what arrived on the wire. Be careful, though: if your decoder differs from the server's decoder, an attacker can exploit the difference (a "parser differential"). Decode the way the protected application does.

Over-normalizing can also cause false positives. Decoding five times might turn an innocent literal %2525 into something that looks malicious. Match the number of rounds to what the target actually performs.

Writing good rules

  • Anchor to a field. Searching the whole packet for select matches blog posts about databases. Limit the rule to the URI or parameter where the vulnerability lives.
  • Combine conditions. "Port 445 and this SMB command and this length" beats any single clue.
  • Use a fast pre-filter. Engines first look for a simple fixed string (a "fast pattern") and only then run the expensive regular expression.
  • Include references. Link the rule to the CVE or advisory so an analyst can understand the alert without asking you.
  • Test on clean traffic. Run a new rule over a day of normal logs first; every hit is a false positive you can remove before blocking.

Strengths and limits

Signature detection is precise, quick to explain to an auditor ("rule 1002 fired because the URI contained a SQL tautology"), and cheap to run at line speed. Its limits are equally clear: it only finds known patterns, it needs constant updates, and it can be bypassed by anything the rule author did not foresee. That is why it is paired with anomaly detection, and why real rule languages such as Snort and Suricata rules add flow state, thresholds and protocol-aware keywords.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why do signatures that describe the vulnerability beat signatures that describe one exploit sample?

    An attacker can change an exploit's bytes easily, but the vulnerable condition (for example an oversized field) has to remain.
  2. 2. A request contains %252e%252e%252f. What does a single round of URL decoding produce?

    Decoding %25 gives a literal percent sign, so one round leaves %2e%2e%2f; a second round gives ../.
  3. 3. What problem does stream reassembly solve?

    Without reassembly, a pattern split across packets never appears whole in any single packet.
  4. 4. Which is a limitation of signature-based detection?

    Signatures encode prior knowledge, so brand new attack techniques pass until a rule exists.

Summary

  • A signature describes known-bad traffic: where to look, what to match, how severe, what to do.
  • Match the vulnerability condition, not a single exploit sample.
  • Attackers evade with encoding, case, whitespace, fragmentation and protocol tricks.
  • Normalize and reassemble before matching, and decode like the target does.
  • Test every new rule on clean traffic to find false positives before blocking.
  • Signatures are precise but blind to the unknown, so combine them with anomaly detection.

Next: Anomaly Detection and Baselines