Skip to content
Tutorials Center
My learning

Topics

Lesson 17 of 80 ยท Cryptography Basics

Identifying Hashes, Ciphers and Steganography

Learn to recognize hash types by length and prefix, tell encoding from encryption using entropy, spot classical ciphers, and understand LSB steganography and its detection.

Analysts regularly face unlabelled data. A database dump contains a column of strange strings: are they hashed passwords, and if so with which algorithm? A captured message looks like noise: is it encrypted, compressed, or just Base64? A forensic image holds an ordinary-looking photo: could it carry hidden data? Answering "what am I looking at?" is the first step of incident response, forensics and security auditing, and it also helps developers check that they are storing passwords safely.

This lesson teaches the clues: how to recognize common hash formats, how to distinguish encodings from encryption and from compressed data, how classical ciphers betray themselves, and how steganography hides data inside other files and how it can be detected. All examples are defensive: you inspect data you are authorized to examine. For the underlying concepts, see Hash Functions and Integrity and Cryptography in Plain Language.

Recognizing hashes

A hash has no label, but its length, character set and prefix are strong clues.

Clue Likely algorithm Notes
32 hex characters MD5 (or NTLM, MD4) Broken for security; unsalted MD5 is a red flag in a password table
40 hex characters SHA-1 Collision attacks exist; legacy
64 hex characters SHA-256 (or SHA3-256, BLAKE2s) Fine for integrity; not for passwords on its own
128 hex characters SHA-512 Same caution
$2a$, $2b$, $2y$ + cost bcrypt Designed for passwords; 60 characters
$argon2id$v=19$m=...,t=...,p=...$ Argon2 A modern password hash
$1$, $5$, $6$ MD5-crypt, SHA-256-crypt, SHA-512-crypt Unix crypt formats with salt
pbkdf2_sha256$600000$salt$... PBKDF2 Framework-specific text formats
or LDAP-style salted/unsalted SHA-1 Often base64

Length alone is ambiguous: many algorithms share an output size, and a truncated or differently encoded hash changes it (hex doubles size compared with raw bytes; Base64 is about 4/3 of raw). A SHA-256 hash is 32 bytes, 64 hex characters or 44 Base64 characters. The prefix, when present, is the most reliable clue because password-hashing formats embed the algorithm, cost and salt. Check where the value came from, and confirm by testing a known input, as you would not guess blindly.

Python A hash identifier

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

(The first value is the well-known MD5 of a common weak password, the second the SHA-1 of the same word, and the third the SHA-256 of hello. The other strings are made-up examples that follow each format.) The security finding is more important than the label: if a user table holds 32-character hex values, passwords are probably stored with a fast, unsalted hash, which must be fixed (Password Storage Done Right).

Encoding, compression or encryption?

Three transformations produce unreadable-looking data, but they differ in purpose:

  • Encoding (hex, Base64, URL encoding) changes representation for transport. It has no key, and anyone can reverse it. Base64 uses A-Z a-z 0-9 + / and often ends with =; hex uses 0-9 a-f; URL encoding uses %20 style sequences.
  • Compression (gzip, zip) shrinks data, has recognizable magic bytes (1f 8b for gzip, PK for zip), and needs no key.
  • Encryption needs a key, and its output looks like uniform random bytes.

A useful measurement is Shannon entropy in bits per byte (0 to 8). English text is about 4; Base64 text about 6; compressed and encrypted data close to 8. Entropy tells you random-looking or not, not which of compression or encryption it is, but magic bytes and structure settle that. High entropy inside a supposedly plain file is also a clue to look harder (Reverse Engineering Basics uses it to find packed code).

Python Classify data by encoding and entropy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that Base64 and hex "look scrambled" but decode instantly with no key. Security mistakes often come from confusing the two: Base64 in a cookie or config file is not protection. See Sessions, Cookies and JWTs, where a JWT payload is readable by anyone.

Spotting classical ciphers

Old ciphers still appear in puzzles, malware obfuscation and challenge exercises, and they teach why modern ciphers are built as they are. Clues:

  • Caesar / shift ciphers shift each letter by a fixed amount. The letter frequencies keep the same shape as English, just moved, and there are only 25 keys to try.
  • Monoalphabetic substitution replaces each letter by another. Frequencies are preserved, so frequency analysis (the most common letter is probably e) breaks it.
  • Vigenere uses several shifts in rotation. Letter frequencies look flatter, but the index of coincidence (IC) reveals structure: about 0.065 for long English text, near 0.038 for random letters, and in between for polyalphabetic ciphers.
  • XOR with a short repeating key (common in simple malware) leaves repeated patterns and low-entropy plaintext fragments.
  • Transposition reorders letters without changing them, so frequencies look exactly like English though the words are scrambled.

The index of coincidence is quick to compute, and it also gives a first-pass test for whether text was shifted or truly scrambled:

Python Frequency clues: shift cipher vs scrambled text

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Real tools combine many such tests. The lesson for defenders: a custom cipher or "homemade encryption" in a product is a finding, because standard algorithms exist and have been analyzed for decades.

Steganography: hiding that data exists

Cryptography hides what a message says; steganography hides that a message exists. A secret is embedded in an innocent cover (image, audio, video, document, network packet) so an observer sees nothing unusual. The two combine well: encrypt, then embed.

The classic technique is least significant bit (LSB) substitution. In an 8-bit image channel, the lowest bit of each pixel value contributes almost nothing visually, so replacing it with message bits changes colors by at most 1 out of 255. Other carriers include unused header fields, whitespace in text, timing patterns in network traffic, and data appended after the end of a file.

Why do defenders care? Steganography is used for data exfiltration (hiding stolen data in uploaded images), to deliver malware configuration inside image files, and for covert communication. Legitimate uses include watermarking and copyright marking.

Python LSB embedding and a simple detector

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The change is invisible (at most 1 level of brightness), yet the lowest-bit plane of the stego image looks random and its "roughness" rises. This is the basis of simple statistical detection (chi-square and RS analysis do it more rigorously). Real photographs contain sensor noise, so detectors compare the image with what a natural image of that type should look like, which gives false positives and misses.

Detecting steganography

  • Check the file, not just the pixels: unusual file size for the resolution, data after the end-of-file marker, odd metadata, or an image that was clearly re-saved.
  • Statistical steganalysis: chi-square, RS analysis, and sample-pair analysis on the LSB plane; machine-learning detectors trained on clean and stego images.
  • Compare with the original if you have it; hash differences reveal change (Hash Functions and Integrity).
  • Visualize bit planes to see the structure in the lowest plane.
  • Prevent and monitor: in sensitive environments, restrict outbound file uploads, apply content inspection and re-encode images (lossy re-compression destroys most LSB payloads), and watch for unusual volumes of image transfers (Intrusion Detection and Network Monitoring). Forensic workflows are in Digital Forensics Basics.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A password table contains 32-character hexadecimal strings. What is the most important concern?

    32 hex characters match MD5 or NTLM. Fast, usually unsalted hashes let attackers test billions of guesses per second, so a password-hashing function is needed.
  2. 2. Which statement about Base64 is correct?

    Base64 only changes representation for transport; it provides no confidentiality.
  3. 3. Data with entropy near 8 bits per byte and no recognizable header is most likely:

    Uniformly random-looking bytes indicate encryption or strong compression. Magic bytes help tell them apart.
  4. 4. What distinguishes steganography from cryptography?

    Cryptography makes the message unreadable. Steganography conceals the existence of any message, and the two can be combined.

Summary

  • Identify hashes by prefix first (bcrypt, Argon2, crypt formats), then by length (MD5 32, SHA-1 40, SHA-256 64, SHA-512 128 hex characters); length alone is ambiguous.
  • Fast unsalted hashes in a password store are a finding.
  • Encoding (hex, Base64) and compression need no key; entropy near 8 bits per byte means encrypted or compressed, and magic bytes help tell them apart.
  • Classical ciphers leak statistical structure, which frequency analysis and the index of coincidence expose; custom homemade ciphers are a red flag.
  • Steganography (for example LSB substitution) hides that data exists; it is used for exfiltration and covert communication, and is detected through file and statistical analysis.
  • Always analyze only data you are authorized to examine.

Next: Hash Functions and Integrity to revisit how hashes behave, or continue to TLS, Certificates and PKI.