Skip to content
Tutorials Center
My learning

Topics

Lesson 24 of 37 ยท Packers and Obfuscation

Entropy and Section Analysis: Finding Packed Data

Measure Shannon entropy per section and per window to find compressed or encrypted regions in a binary, with runnable Python models and guidance on false positives.

Compressed and encrypted data looks like random noise. Ordinary program code and text do not: they have repeating patterns, a lot of zero bytes and a limited set of byte values. Entropy is a number that measures this randomness, and it is one of the cheapest, fastest ways to find the packed or encrypted parts of a file before you open a disassembler. Analysts use it in triage tools, and you can compute it in a dozen lines.

Everything here uses made-up byte strings generated in Python, so you can run it without any real sample.

Entropy in one picture

Text About 4.5 bits per byte.
Code Often 5.5 to 6.5.
Compressed Close to 8.
Encrypted Close to 8.

Maximum is 8 bits per byte.

1 / 2

Shannon entropy

Claude Shannon's entropy for a byte string counts how often each of the 256 possible byte values occurs, and computes

H = - sum( p * log2(p) )

where p is the fraction of the data that is a given byte value. The result runs from 0 to 8 bits per byte:

  • 0 means every byte is the same (a long run of zeros).
  • 8 means all 256 values are equally likely, which is what good encryption and good compression output looks like.

Typical figures, which vary by file:

Data Entropy (bits per byte)
Padding of zero bytes near 0
English text 4 to 5
x86 machine code 5.5 to 6.5
Resources with images or icons (uncompressed) 3 to 6
ZIP, PNG, JPEG, packed or encrypted payloads 7.2 to 8

A section with entropy above roughly 7.2 deserves attention. Because compressed data is also high-entropy, entropy cannot tell packing from encryption, and it cannot tell either from a legitimate compressed image.

Computing it

The first stand-in computes entropy for several kinds of generated data so you can see the ranges for yourself. The random bytes come from a seeded generator, so the numbers are reproducible. zlib stands in for compression.

Python Entropy of different kinds of data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that the code-like bytes sit well above the text, and the compressed words land close to the random bytes. The zero padding scores 0. Entropy measurements are noisy on tiny inputs, so compare sections of a reasonable size, a few kilobytes at least.

Per-section entropy

Most tools show entropy per PE section. A normal compiled program has a .text (code) section around 6, a .data section lower, and a .rsrc section whose value depends on its content. A packed sample often has one huge section near 8 and a tiny .text, or sections named by the packer. Here is a model of a table a triage tool prints, with a simple classification.

Python Classify sections by entropy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Windowed entropy: finding hidden blobs

A section can be mostly ordinary with a hidden encrypted blob inside, for example an encrypted configuration or an embedded second-stage payload in a data section or at the end of a file (an overlay). Per-section numbers average that away. A sliding window measures entropy in blocks (say 1 KB) and reveals the spike. Many tools draw this as a graph; each flat plateau near 8 is a candidate for investigation.

The third stand-in builds a made-up file with a plain-text region, an encrypted-looking region and a zero padding tail, then prints a text bar chart.

Python Find an encrypted blob with a sliding window

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The bars climb from about 4 for the text to near 8 for the blob, then drop to 0 for the padding. In a real investigation the next step is to look at what the high region is: its file offset, what references it, and whether a routine reads from it and decodes it. That ties back to reading disassembly and to string and configuration obfuscation.

False positives and limits

  • Legitimate compressed data. Images (PNG, JPEG), archives, fonts, installers and many resources are high-entropy for good reasons.
  • Digital signatures and certificates add a block of high-entropy bytes in the overlay of signed files.
  • Evasion. Authors who know this check can lower the apparent entropy by padding the payload with low-entropy bytes or encoding it as base64 (about 6 bits per byte, as it uses only 64 symbols). Entropy then looks "normal" while the data is still hidden.
  • Small samples. Short buffers give unstable numbers.

So use entropy as a pointer. It tells you where to spend your time and never what the data means.

Record the entropy of every section in your analysis notes. Comparing it with the entropy after unpacking is a quick confirmation that unpacking worked: the former packed section should drop to code-like values.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What entropy value, in bits per byte, is the maximum?

    With 256 equally likely byte values the entropy is log2(256) = 8.
  2. 2. A section has entropy 7.9. What is the best conclusion?

    High entropy indicates random-looking data, which can be compressed, encrypted or a legitimate resource.
  3. 3. Why use a sliding window instead of one value per section?

    Averaging over a section hides a small high-entropy region; windows reveal it.
  4. 4. How can an author make a payload look less random to an entropy check?

    Base64 and padding lower the measured entropy while the content stays hidden.

Summary

  • Entropy measures randomness from 0 to 8 bits per byte; code is roughly 5.5 to 6.5, packed or encrypted data 7.2 and above.
  • Compute it per section for triage and per window to find hidden blobs and overlays.
  • High entropy does not distinguish encryption from compression, or malicious from legitimate data.
  • Authors can lower it with encoding or padding, so use it as a pointer, not a verdict.
  • Comparing entropy before and after unpacking is a useful sanity check.

Next: String Obfuscation and Decoding