Skip to content
Tutorials Center
My learning

Topics

Lesson 9 of 37 ยท Static Analysis

Hashing and Fingerprinting Samples

Learn how cryptographic hashes, import hashes and fuzzy hashes identify and group malware samples, why one changed byte breaks a hash, and how to look samples up safely.

The first thing you do with any sample is fingerprint it. A fingerprint is a short value computed from the file's contents that identifies it without opening it, so you can compare it with what the world already knows. Hashes are the workhorse of triage: they let you search threat-intelligence sources, deduplicate a pile of files and prove that a file has not been altered.

This lesson covers three kinds of fingerprints: cryptographic hashes (exact match), import hashes (same build tools, same imports) and fuzzy or similarity hashes (almost the same file). All examples run on harmless byte strings you create yourself.

Cryptographic hashes

A cryptographic hash function takes any amount of data and returns a fixed-length value. Three properties matter:

  • Deterministic: the same input always gives the same hash.
  • Avalanche effect: change one bit and the hash changes completely.
  • One-way and collision-resistant: you cannot practically rebuild the file from the hash or craft two different files with the same hash (true for SHA-256; broken for MD5 and SHA-1 against deliberate attacks).

The common choices are MD5 (128 bits, 32 hex characters), SHA-1 (160 bits, 40 characters) and SHA-256 (256 bits, 64 characters). Many older feeds still list MD5 and SHA-1, so analysts record all three, but SHA-256 is the identifier you should prefer, because MD5 and SHA-1 collisions can be forged on purpose.

Python Compute and compare hashes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Compare the two sets of values: there is no visible similarity even though the files differ by one bit. That avalanche effect is why attackers can evade hash blocklists by changing a byte, and why defenders need more flexible fingerprints too.

Hash a file the right way

Real files can be large, so you read them in chunks rather than loading everything into memory. This pattern is the standard way to write a hashing script (shown here with an in-memory stream, which behaves like a file):

Python Hashing in chunks

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

On a command line you would use sha256sum file (Linux), shasum -a 256 file (macOS) or Get-FileHash file (PowerShell). Always hash the original sample before you unzip, rename or open it.

What hashes are used for

  • Lookup: paste a SHA-256 into a reputation service and see if others reported the file. This reveals nothing about the file's contents, so it is safe even for confidential documents.
  • Deduplication: twenty infected machines may all hold the same file.
  • Indicators of compromise: share hashes in reports so others can search their own systems.
  • Integrity: prove that the file you analyzed is the one you collected.

Hash-only detection has a weakness: change a byte and the match disappears. A new hash with no history says little about a file, so absence of a match is not a clean bill of health.

Import hash (imphash)

A Windows program lists the library functions it uses in its import table. The import hash, or imphash, is an MD5 of those imports in a normalized order. Two files built from the same source with the same libraries tend to share an imphash even when the compiled bytes differ, which makes it useful for grouping a family. The next example computes a simplified version from a made-up import list (the real algorithm lowercases and orders names in the file's import order; this one sorts them for clarity).

Python A simplified import hash

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

You will build a real import list from a PE file in Imports, Exports and Sections.

Fuzzy hashes and similarity

Fuzzy hashes (ssdeep, TLSH, sdhash) produce fingerprints where similar files give similar values. They are how analysts find "this looks like that other sample, but modified". They are read-only tools in this course because they need compiled libraries, but the principle can be shown with n-grams: split each file into short overlapping chunks and measure how many chunks two files share.

Python Why similarity hashing works

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The exact hashes of base and patched would be unrelated, but the similarity score stays high. Fuzzy hashing makes this cheap at scale by compressing the n-gram idea into a short string.

Hash lookups safely

  1. Hash the sample locally.
  2. Search the hash in your internal records and in reputation sources.
  3. Only upload the file itself if policy allows it and it does not contain confidential data.
  4. Record every result with the source and time, because reputation changes.

Common mistakes

  • Reporting only MD5.
  • Hashing a file after opening it in a program that may modify it (some documents are rewritten when saved).
  • Believing "unknown hash" means "safe".
  • Comparing hashes by eye; use a script or ==.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. You flip one bit in a file. What happens to its SHA-256 hash?

    The avalanche effect means a tiny input change produces an entirely different hash.
  2. 2. Why prefer SHA-256 over MD5 as the primary identifier?

    An attacker can craft different files with the same MD5, so SHA-256 is the trustworthy choice.
  3. 3. What does an import hash group together?

    Recompiled variants of the same program usually keep the same import table and therefore the same imphash.
  4. 4. Which problem do fuzzy hashes solve?

    Similar inputs produce similar fuzzy hashes, so modified versions of a sample can be found.

Summary

  • Hash every sample on receipt and record MD5, SHA-1 and SHA-256, preferring SHA-256.
  • A one-bit change produces a new hash, so hashes catch exact files only.
  • Imphash groups by imported functions; fuzzy hashes find similar files.
  • Hash lookups send no file contents and are safe for confidential files.
  • No match does not mean safe.

Next: Identifying File Types and Magic Bytes