Skip to content
Tutorials Center
My learning

Topics

Lesson 59 of 133 ยท Networking and HTTP

File Uploads and Multipart Parsing

Accept file uploads in Node.js safely - parse multipart bodies as a stream, enforce size and count limits, write to disk with backpressure, sanitize filenames and check file signatures.

A file upload is the hardest request body a server handles. It can be gigabytes long, it mixes text fields with binary data, the client controls every byte of it (including the file name), and a single careless line can fill your disk or overwrite a file you meant to protect. In Request Bodies, JSON and Forms you wrote a small multipart parser that holds the whole body in memory. That is fine for a demo and dangerous for production.

This lesson turns it into a real upload handler. You will write a streaming parser that never keeps more than a few kilobytes of a file in memory, writes each file to disk with backpressure, enforces limits on size and count, and treats everything the client sends as hostile: the file name, the declared type and even the content. Frameworks and libraries such as busboy or multer do the same job; knowing how it works helps you configure them correctly.

What a multipart body looks like

A browser form with enctype="multipart/form-data" sends one request whose Content-Type header names a boundary string. The body is a sequence of parts, each separated by that boundary:

TEXT
--XYZ\r\n
Content-Disposition: form-data; name="title"\r\n
\r\n
Sunset\r\n
--XYZ\r\n
Content-Disposition: form-data; name="photo"; filename="a.png"\r\n
Content-Type: image/png\r\n
\r\n
...binary bytes...\r\n
--XYZ--\r\n

Every part has headers, an empty line, and a body. The body ends at the next \r\n--XYZ. The final boundary has two extra dashes. Because file bytes can contain any value, you must work on Buffers and never convert a file to a string (see Buffers).

The hard part of streaming is that a request arrives in chunks of arbitrary size. A boundary can be cut in half by a chunk edge, so the parser must always remember the last few bytes of the previous chunk.

The parser as a state machine

Our parser has five states. It consumes chunks, moves between states when it finds a marker, and emits events to a handler.

stateDiagram-v2
    [*] --> start
    start --> afterDelimiter: first delimiter found
    afterDelimiter --> headers: CRLF follows
    afterDelimiter --> done: two dashes follow
    headers --> body: blank line found
    body --> afterDelimiter: delimiter found
    done --> [*]

The parser states. A file part loops through headers, body and afterDelimiter until the closing boundary arrives.

In the body state, the parser can safely hand over every byte except the last delimiter.length - 1 bytes of its buffer, because those might be the start of the delimiter. They stay in the buffer until the next chunk shows what they are.

Streaming a file to disk

The next example is the heart of the lesson. receive reads the request with for await, so Node.js stops reading from the socket while we wait for the disk (this is the backpressure idea from Pipeline and Backpressure). Text fields are collected in memory with a small cap; files go to a randomly named temporary file while we count bytes and compute a SHA-256 hash on the fly.

The client sends one field and two files, one of them 300,000 bytes, so many chunk boundaries fall inside the data. It then compares the hash the server computed with the hash of the original bytes.

Node.js A streaming multipart upload handler

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The two hashes match, and the number of bytes on disk is exactly what was sent, even though the data arrived in many pieces. Memory use stays small because only the current chunk plus a short tail is ever buffered; the file itself flows straight to the disk.

Limits: size, count and time

An upload endpoint without limits is a denial-of-service endpoint. The parser above already checks:

  • File size (maxFileBytes): counted while streaming, so an attacker cannot send 50 GB and have you accept it before you notice.
  • Field size (maxFieldBytes): text fields live in memory, so they get a much smaller cap.
  • Number of parts (maxParts): thousands of tiny parts can burn CPU and file handles.
  • Header size: part headers beyond 8 KB are rejected.

You should also look at the declared Content-Length first and reject obviously oversized requests before reading anything. Be aware that it can be missing (chunked transfer) or wrong, so it is a fast path, not the real defense. Finally, set timeouts on the server (server.requestTimeout, server.headersTimeout) so a client that sends one byte per minute cannot hold a connection forever; see The http Module: Servers.

When a limit is hit, abort the current file, delete what was written so far, and answer with the right status code: 413 Content Too Large for size, 415 Unsupported Media Type for a wrong content type, 400 Bad Request for malformed bodies. The next example uses the same receive idea in a smaller form to show the cleanup rule: a failed upload must leave nothing behind.

Node.js Rejecting an oversized upload and cleaning up

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Both checks matter. The declared length catches honest clients early and cheaply. The running counter catches the clients that lie or use chunked encoding. Only the first upload remains on disk.

Never trust the file name

The filename in a part is a string chosen by the client. It can be ../../etc/cron.d/job, C:\Windows\x.dll, a name with newlines that breaks your logs, or one that is 5,000 characters long. The rule from Paths, Permissions and Safe File Handling applies fully: do not build a storage path from it. Store files under an id you generate (as above) and keep the original name only as display metadata, after cleaning it.

A cleaning function should keep the base name only, replace anything outside a small safe alphabet, limit the length, and decide about the extension from an allow-list, not from a block-list.

Never trust the declared type either

The Content-Type of a part, and the file extension, are also claims by the client. A file called cat.png with type image/png can contain a script. The cheapest strong check is the signature ("magic number") at the start of the file: PNG files begin with the eight bytes 89 50 4E 47 0D 0A 1A 0A, JPEG with FF D8 FF, PDF with %PDF-, and so on. Check the first chunk of the part before you commit to saving it.

Node.js Safe names and signature checks

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The signature check is a filter, not a guarantee. A file can start with a valid PNG header and still contain extra data after it. For serious needs, decode the image with a library, re-encode it, and store the re-encoded copy. For other types, scan with antivirus software, serve user uploads from a separate domain, and send Content-Disposition: attachment plus X-Content-Type-Options: nosniff so browsers never run them as pages. More on this in Web Application Security.

Practical checklist

  • Store uploads outside your public web root, under generated names, and serve them through a handler that checks permissions.
  • Limit size, count and time. Count bytes while streaming, never after.
  • Write to a temporary location first, validate, then move the file into place (a rename on the same disk is atomic).
  • On any error, delete partial files. Run a cleanup job for temporary files older than a day.
  • For huge files or many users, let browsers upload directly to object storage with a signed URL, so the bytes never pass through your server.
  • Consider resumable uploads for large files: the client sends the file in numbered pieces, and the server assembles them.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does the streaming parser keep the last delimiter.length - 1 bytes of its buffer instead of writing everything?

    Chunks have arbitrary sizes, so a boundary can be split across two chunks. Holding back a short tail lets the parser recognize it once the rest arrives.
  2. 2. What is the best way to choose the path where an uploaded file is stored?

    The file name is attacker-controlled and can contain path segments or reserved names. A generated name removes the whole class of path attacks.
  3. 3. Why is checking the Content-Length header alone not enough to enforce a size limit?

    Use the header as a cheap early rejection, and a running byte counter as the real defense.
  4. 4. What should happen to a partially written file when an upload fails?

    Leftover partial files waste disk space and can be abused. Clean up in the error path, and run a periodic job for anything that slips through.
  5. 5. What does the signature (magic number) check give you?

    Signatures reveal what the bytes claim to be, which the client cannot fake as easily as a name. A file can still carry extra malicious data, so combine this with re-encoding, safe serving headers and scanning.

Summary

  • A multipart body is a series of parts split by a boundary; parse it as bytes, in a state machine that survives chunk boundaries.
  • Stream files to disk with backpressure (write + drain) and hash or count them on the way.
  • Enforce limits on file size, field size, part count, header size and time; delete partial files when anything fails.
  • The file name and the declared content type are untrusted. Store under generated names and verify the content signature.
  • Keep uploads outside the web root, serve them with safe headers, and consider direct-to-storage uploads for large files.

Next: Child Processes