Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 80 ยท Web Application Security

Input Validation and Sanitization

Learn allow-list validation, canonicalization, path traversal and file upload checks, and how regex pitfalls such as ReDoS can hurt, with Python validators.

Every web application is a machine for accepting input from strangers: form fields, query strings, headers, cookies, uploaded files, JSON bodies, even filenames. Most of the attacks in this chapter, SQL injection, XSS and SSRF, begin with data the programmer did not expect. Input validation is the habit of deciding, in code, exactly what a field may contain, and refusing everything else before the data goes anywhere.

Validation is a first filter, not the whole defense. You still parameterize queries and encode output. But a strict check at the edge shrinks the attack surface, catches bugs early and gives users clear error messages. In this lesson you will learn the allow-list approach, why input must be canonicalized before it is checked, how path traversal and unsafe file uploads work, and how a careless regular expression can freeze a server.

Validate, sanitize, encode: three different jobs

These words are often mixed up:

  • Validation decides whether input is acceptable and rejects it if not ("age must be a whole number from 0 to 130").
  • Sanitization changes input to make it acceptable (trimming whitespace, removing characters). Use it sparingly: silently altering data can surprise users and hide attacks.
  • Encoding / escaping protects output in a specific context (HTML, SQL, shell). It happens when data is used, not when it arrives.

The strongest pattern is validate on arrival, encode on use. And always validate on the server. Browser-side checks (HTML required, pattern) are a convenience for honest users; an attacker simply sends the request without the page.

Allow-lists beat block-lists

A block-list rejects known bad input ("no <script>"). An allow-list accepts only known good input ("3 to 20 letters, digits or underscores"). Attackers are creative and the set of bad inputs is endless, so block-lists fail; the set of good inputs is usually small and describable. Define the format, the length, the range and the type for every field.

Python Strict validators

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A few lessons are hidden here. Use re.fullmatch, not re.match or re.search, or the pattern may match only part of the value. Check the type before the content. Be careful with email: the official grammar is huge, so check the shape and prove ownership by mailing a link. And in Python, \d in a regular expression on a str matches digits from other scripts too, which the next demo shows.

Canonicalize first, then validate

The same data can be written in many forms: %2e%2e%2f is ../ after URL decoding, the fullwidth letters in ๏ผก๏ผค๏ผญ๏ผฉ๏ผฎ normalize to ADMIN, trailing spaces and case differences make two usernames look different to a human and identical to a database collation. If you validate one form and use another, the check is bypassed. So: decode and normalize to a single canonical form first, then validate that form, then use that same value.

Python Canonicalization and lookalikes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Pick a rule and apply it consistently: decode exactly once at the edge, normalize Unicode (usually NFKC for identifiers), strip control characters, and reject anything that still looks wrong, such as a leftover % after decoding.

Path traversal

If a program builds a file path from user input, an attacker can include .. segments to climb out of the intended folder and read files like configuration or keys. This is path traversal (or directory traversal). The robust defense is not to look for ../ in the string, but to resolve the final path and verify that it still lives inside the allowed base directory. Better still, never use user-supplied names as paths at all: store files under generated IDs.

Python Keeping file access inside a base folder

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Note the absolute path /etc/passwd: posixpath.join throws away the base when the second part is absolute, and the commonpath check catches it. In real code also resolve symbolic links (os.path.realpath) because a link inside the folder can point outside, and run the process with file permissions that cannot read secrets even if a check fails.

File uploads

Uploads are risky because attackers control the name, the type, the size and the bytes. A reasonable checklist:

  • Limit size at the web server and in code.
  • Allow-list extensions that you expect (.png, .jpg, .pdf), compared after lowercasing and using only the last extension, since photo.php.jpg and evil.jpg.php are classic tricks.
  • Check the content, not the Content-Type header (which the client sets). Look at the first bytes ("magic numbers"), or better, decode the file with an image library and re-encode it.
  • Rename to a random identifier and store outside the web root, or on separate storage that cannot execute code. Never use the original filename in a path.
  • Serve safely: correct Content-Type, X-Content-Type-Options: nosniff, and a download header for types the browser should not render. See HTTP Security Headers.
  • Scan for malware if you accept documents from the public.
Python An upload checker

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The photo.php.jpg row shows why magic bytes matter: the extension looks right, but the content is not a JPEG. Magic bytes alone are still forgeable (a file can start like a PNG and contain more), so re-encoding images and storing them where they cannot run is the stronger defense.

Regex pitfalls and ReDoS

Regular expressions are the everyday validation tool, and they come with two traps:

  1. Anchoring mistakes. Without anchors, [a-z]+ happily "validates" hello<script>. Use re.fullmatch (or ^...$ in engines where $ also allows a trailing newline, so prefer \Z or fullmatch).
  2. Catastrophic backtracking, or ReDoS. Many engines, including Python's re, try alternatives by backtracking. Patterns with nested repetition such as (a+)+$ can take exponential time on input that almost matches, so one crafted request can tie up a CPU core and take a service down.

Timing on a shared machine is noisy, so this demo counts the matcher's steps for ^(a+)+$ against aaa...a! with a tiny backtracking matcher written for the purpose:

Python Counting steps in catastrophic backtracking

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Each extra two characters multiplies the work by about four, so 30 characters would already need around a billion steps. Defenses: avoid nested quantifiers and overlapping alternatives, limit input length before running a regex, prefer simple parsers over regexes for complex formats, and use a linear-time engine (RE2, Rust's regex) for patterns that come from users or run on untrusted text. Python's re has no built-in timeout.

Where validation fits in the bigger picture

Validate at every trust boundary, not only at the first one, since internal services receive data too. Return clear but non-revealing errors. Log rejected input (without logging secrets) because bursts of failures are a signal (Logging, Log Analysis and SIEM). And remember that validation cannot make dangerous characters impossible: a name like O'Brien is valid, so you still need parameterized queries and output encoding.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why is an allow-list usually better than a block-list?

    Attackers invent new bad inputs, so a block-list always has gaps. An allow-list accepts only what you have defined as valid.
  2. 2. Why must you decode and normalize input before validating it?

    If %2e%2e%2f passes a check for ../ and is decoded later, the traversal sneaks through. Validate the canonical form you will actually use.
  3. 3. What is the most reliable way to stop path traversal?

    String filters miss encodings and tricks like ....//. Resolving the path and checking its location catches all of them.
  4. 4. A file is named photo.jpg but starts with <?php. What should the upload check conclude?

    The extension and Content-Type header are attacker-controlled. Checking the actual bytes (or decoding the image) reveals the mismatch.
  5. 5. What makes a regular expression like ^(a+)+$ dangerous?

    Inputs that almost match force the engine through an exponential number of ways to split the text, a ReDoS condition.

Summary

  • Validate on the server, with allow-lists for type, length, range and format, and reject everything else.
  • Validation, sanitization and output encoding are separate jobs; validate on arrival and encode on use.
  • Canonicalize (decode once, Unicode-normalize) before validating, and use the same value afterwards.
  • Stop path traversal by resolving the path and confirming it lies inside the base folder, or by never using user names as paths.
  • Check uploads by size, extension allow-list and real content, store them under random names where they cannot execute.
  • Use fullmatch, avoid nested quantifiers, cap input length and consider linear-time regex engines to avoid ReDoS.

Next: SSRF, Unsafe Deserialization and XXE, where the dangerous input is a URL, a serialized object or an XML document.