Skip to content
Tutorials Center
My learning

Topics

Lesson 25 of 37 ยท Packers and Obfuscation

String Obfuscation and Decoding

Recognise XOR-encoded strings, stack strings, base64 layers and API hashing, and practice decoding made-up examples with runnable Python stand-ins.

Strings are the analyst's favourite shortcut: a URL, a file path, a registry key or an error message can reveal what a program does in seconds. Authors know this, so they hide them. Instead of storing "http://example.invalid/update" in plain view they store a scrambled version and decode it just before use. If you can recognise the decoder, you can usually decode every string at once without running the sample.

All examples below are made-up strings with invalid example domains. The Python programs decode data that the lesson itself generates; they are teaching stand-ins for the decoding routines you would find in a disassembler.

Common ways to hide strings

XOR One key over every byte.
Stack strings Built one character at a time.
Base64 layers Text that is really bytes.
API hashing Names replaced by numbers.

Find the decoder, decode everything.

1 / 2

Single-byte XOR

XOR with a constant is the simplest scheme and still very common. Each byte b is replaced by b ^ key. XOR is its own inverse, so the same loop encodes and decodes. In disassembly it is the loop you saw in control flow: a counter walking a buffer with xor byte ptr [esi+ecx], 0x5a.

Because a single byte has only 256 possible values, you do not even need to find the key in the code. Brute force all 256 keys and keep the results that look like readable text. The first stand-in encodes a made-up URL with a secret key and then recovers it by scoring each candidate on how many bytes are lowercase letters, digits or common URL characters.

Python Brute-force a single-byte XOR key

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The best candidate scores 1.00 and reads as the URL; wrong keys produce a mixture of unlikely bytes. On very short strings several keys can score well, so add a second test: look for expected substrings such as http, .exe, \\ or HKEY.

Multi-byte and rolling keys

A longer key, such as a four-byte repeating key, or a key that changes as it goes (add the previous byte, rotate, add the index), defeats naive brute force but not careful reading. Once you identify the decoding loop in the disassembler and rewrite it in Python, the key and algorithm are right there, so decoding the whole sample's blobs is a copy-paste job. This is the central skill: re-implement the decoder rather than debug it again and again.

Python Re-implement a repeating-key decoder

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In a real case, the blobs live at fixed offsets in the file or in a configuration structure. A script that walks them and prints every decoded string is a reusable artefact you can attach to your report.

Stack strings

Not every hidden string is in the data section. A stack string is built at run time by writing characters (or 4-byte chunks) onto the stack, one mov at a time:

TEXT
mov   byte ptr [ebp-0Ch], 63h   ; 'c'
mov   byte ptr [ebp-0Bh], 6Dh   ; 'm'
mov   byte ptr [ebp-0Ah], 64h   ; 'd'
mov   byte ptr [ebp-09h], 2Eh   ; '.'
mov   byte ptr [ebp-08h], 65h   ; 'e'
mov   byte ptr [ebp-07h], 78h   ; 'x'
mov   byte ptr [ebp-06h], 65h   ; 'e'
mov   byte ptr [ebp-05h], 0     ; terminator

The strings tool sees nothing because no run of printable bytes sits in the file; the instructions contain the characters as immediate values. The giveaway in a disassembler is a long run of mov with small constants into consecutive stack offsets. Analysis tools can recover these automatically; you can also read the immediates and convert them. The next stand-in does the conversion from a list of (offset, value) pairs.

Python Rebuild a stack string from its immediates

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Base64 and other text encodings

Base64 turns bytes into text made of letters, digits, + and /, ending with = padding. It is not encryption: it only changes the alphabet, and anyone can reverse it. Malware uses it to carry binary data in text channels and to hide strings from naive scans. A telltale sign is a long string with a limited alphabet and a length divisible by four. Layers are common: base64 of XOR of base64.

Python Peel three layers of encoding

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Order matters: the layers must be removed in the reverse of the order they were applied. When one step produces gibberish, try the other likely transforms (zlib, gzip, base64, hex, XOR) instead of assuming you have the wrong key.

API hashing

To hide which Windows functions it uses, a sample may skip the import table and compute a small hash of each function name it needs, then walk the export table of a library in memory comparing hashes until it finds a match. The hash values (for instance 0x0726774C) replace names in the disassembly. Defenders cannot read the imports, so the call looks like an opaque number.

The countermeasure is a lookup table. Take the hash algorithm from the sample, compute it for every exported function name in the system libraries, and map the number back to the name. The stand-in uses a simple made-up hash (not the algorithm of any real family).

Python Resolve API hashes with a lookup table

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Once the constants are named, the sample's real capabilities appear (memory allocation, internet access), and you can rename the calls in the disassembler. Together with behavioural patterns, this recovers what the author tried to hide.

The XOR loop and the hash routine are described and re-implemented here for analysis. The skill is reading and reproducing a decoder; it does not require, and this course does not teach, writing evasive code.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is brute-forcing a single-byte XOR key practical?

    A one-byte key has 256 possibilities; the correct one yields readable text.
  2. 2. What is a stack string?

    Characters appear as immediate values in mov instructions, so strings finds nothing.
  3. 3. Is base64 a form of encryption?

    Base64 has no key; it changes the representation, not the secrecy.
  4. 4. How do you defeat API hashing?

    A table from hash to name restores the hidden function names.

Summary

  • Authors hide strings so that strings and signatures find nothing; decoders must run before use.
  • Single-byte XOR can be brute-forced; longer keys need the decoder loop re-implemented in a script.
  • Stack strings are built from immediate values; rebuild them from the mov constants.
  • Base64 is encoding, not encryption; layers are removed in reverse order.
  • API hashing hides function names behind constants; resolve them with a lookup table.
  • Scripts that decode every blob are reusable evidence for your report.

Next: Indicators of Compromise