Skip to content
Tutorials Center
My learning

Topics

Lesson 14 of 37 ยท Static Analysis

Entropy and File Structure Anomalies

Learn how Shannon entropy reveals compressed or encrypted data, compute it for whole files and sections in Python, and combine it with other structural anomalies.

When a sample shows very few strings and few imports, you suspect something is hiding the real code. Entropy is the number that backs up that suspicion. It measures how random the bytes of a file look: ordinary text and compiled code have structure and score low; compressed or encrypted data looks random and scores high. Analysts use it in minutes to decide whether a file is packed before spending hours on it.

What is Shannon entropy?

For a block of bytes, count how often each of the 256 possible byte values occurs. Entropy is

H = -sum(p * log2(p)) over all byte values with probability p.

The result lies between 0 and 8 bits per byte:

  • 0: every byte is identical (all zeros).
  • About 4 to 5: English text.
  • About 5 to 6.5: typical compiled code.
  • About 7.2 to 8: compressed or encrypted data.
flowchart LR
    A["0: padding, all zeros"] --> B["~4.5: text"]
    B --> C["~6: normal code"]
    C --> D["~7.5+: compressed or encrypted"]

Higher means more random. Very high entropy where you expect code is a red flag.

Python Compute entropy of different data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Compressed text scores much higher than the text it came from, even though it contains the same information. Compression removes the repetition that made the text predictable. Encryption does the same, which is why you cannot tell them apart by entropy alone.

Whole-file versus per-section entropy

A single number for a whole file hides detail. A packed executable usually has a few small readable parts (headers, a stub, a loader) and one large random-looking region. So analysts compute entropy per section or per window and look at the pattern.

The following example slices a synthetic "program" into windows, and prints a text bar chart, which is what entropy-plot tools show graphically. The synthetic file has a normal header, a code-like region, an area of random-looking data (standing in for a packed payload) and trailing padding.

Python An entropy map of a pretend file

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The whole-file score is moderate, which could look normal, but the per-window view shows a clear high-entropy block. This is why per-section checks matter. In real PE files you do the same per section, and also for the overlay (data appended after the last section) and for resources.

Entropy limits

  • Legitimate high entropy exists: images (JPEG, PNG), archives, installers, audio and video are compressed. Judge by what the file claims to be. A high-entropy .text section is odd; a high-entropy JPEG is normal.
  • Low entropy can still be encoded: authors can pad encrypted data with zeros or encode it as base64 text (about 6 bits per byte) to lower the score.
  • Small samples are noisy: entropy of a few dozen bytes means little.

Treat it as a pointer toward packing, not as a verdict.

Other structural anomalies

Combine entropy with the structure checks from earlier lessons (PE headers and sections). A checklist analysts apply:

Check Normal Suspicious
Entropy of code section 5 to 6.5 above 7
Number of imports dozens only LoadLibrary and GetProcAddress
Section names .text, .rdata, .data random or packer names
Section permissions code is RX, data RW writable and executable
Entry point in first code section in last section or outside
Overlay (data after the last section) none or a signature large and high entropy
Resources icons, version info big binary blobs
Timestamp plausible zero, future or identical across unrelated files
Digital signature valid, known publisher stolen or invalid signature
Version info complete and consistent missing or copied from a famous product

No single row decides anything. Triage means counting how many boxes are ticked.

Python Combine anomalies into a packing score

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

setup_installer.exe scores low because installers legitimately carry compressed payloads. That is the point of combining checks: context turns a number into a conclusion.

What to do when you suspect packing

  1. Note the evidence (entropy map, section table, imports).
  2. Check for a known packer signature (UPX is common and easy to unpack; custom packers are harder).
  3. Move to dynamic analysis: a packed program must unpack itself in memory, so running it in the lab and dumping memory often reveals the real code.
  4. Consider advanced unpacking, covered later in Packers and Obfuscation.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What entropy range (bits per byte) is typical for compressed or encrypted data?

    Compressed and encrypted data looks random, approaching the maximum of 8 bits per byte.
  2. 2. Why is per-section entropy more useful than one whole-file number?

    Averaging blends a random-looking payload with ordinary parts, so you look at each section or window separately.
  3. 3. A JPEG image has entropy of 7.9. How should you judge it?

    High entropy is expected for compressed formats. It is suspicious only where the content should be structured, like code.
  4. 4. Entropy alone cannot distinguish what?

    Both produce near-random bytes, so you need other evidence to tell which was used.

Summary

  • Shannon entropy measures randomness from 0 to 8 bits per byte; text is low, code medium, compressed or encrypted data very high.
  • Compute it per section and per window to find hidden payloads.
  • High entropy is only suspicious where structured data is expected.
  • Combine it with imports, section anomalies, entry point, overlay and resources.
  • Suspected packing leads to dynamic analysis and unpacking.

Next: Dynamic Analysis Basics