Skip to content
Tutorials Center
My learning

Topics

Lesson 127 of 133 ยท Projects

Project: File Sharing Platform

Build a file sharing service in Node.js. Stream uploads to disk with size limits and hashing, issue unguessable expiring links, limit downloads and stream files back with Range support.

"Here, I put the file on a link for you." Services such as WeTransfer, Firefox Send and a hundred company-internal tools do the same four things: accept an upload, give back a link, let someone download through that link, and delete the file after a while. It is a wonderful project for Node.js because almost everything is about streams: a 4 GB upload must not be held in memory, and a download should start flowing before the file has been read completely.

You will build the core of such a platform in one runnable program: a streaming upload endpoint that enforces a size limit and computes a checksum on the fly, unguessable links with an expiry time and a download limit, a download endpoint that streams the file and supports Range requests (resumable downloads and video seeking), and a cleanup job. Along the way you meet the security traps of file hosting: path traversal, content sniffing and stored cross-site scripting.

Plan

  1. Upload (PUT /upload): stream the request body to a file in a private folder, counting bytes and hashing as it goes. If the limit is exceeded, stop and delete the partial file.
  2. Link store: metadata for each file (token, original name, size, SHA-256, expiry, download limit) with the rules for when a link is still valid.
  3. Download (GET /f/:token): validate the link, then stream the file with correct headers and Range support.
  4. Delete and cleanup: an owner key for early deletion, and a sweeper that removes expired files.

We use the plain PUT body for uploads to keep the code short. Browsers upload forms as multipart/form-data; the File Uploads and Multipart lesson shows that parser, and everything after parsing (limits, hashing, storage) is identical.

Stage 1: streaming an upload safely

The upload handler must do four things at once while the bytes flow through: write them to disk, count them, hash them, and abort if the count passes the limit. Streams make that natural: pipeline connects the request to a counting transform (which can fail the stream when the limit is exceeded) and then to a file write stream. We hash inside the same transform, so the file is read only once (Pipeline and Backpressure).

Two design points. First, files are stored under a random id that we generate, never under the name the client sent. A client-chosen name such as ../../etc/passwd would be a path traversal attack; with random ids there is no way to influence the path. The original name is kept only as metadata, for the download header. Second, the limit is enforced while streaming. Checking Content-Length alone is not enough, because a client can lie or use chunked encoding.

Node.js Streaming upload with a size limit and a checksum

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Notice the last line: after one success, one over-limit upload and one aborted upload there is exactly one file left. Cleaning up partial files on every failure is part of the contract. The file is created with mode 0o600 (owner only) and the flag 'wx', which fails if the file already exists, so a random-id collision can never overwrite someone else's data. Also notice that the hash comes out when the stream ends, with no second pass over a possibly huge file.

A link is a promise: "whoever has this URL can download until it expires or has been used N times". The token in the URL is the only secret, so it must be unguessable: 16 random bytes encoded as base64url (22 characters) gives 128 bits of entropy, and never a counter or a hash of something guessable.

The rules are small but easy to get wrong:

  • An expired link and a link that does not exist must look the same to the visitor (404), so strangers cannot probe which tokens were once valid.
  • The download counter must be decremented before serving the file and in the same step as the check. If you check first and decrement later, two simultaneous requests can both pass a "one download left" link. In Node's single-threaded model a synchronous check-and-decrement is atomic; with a database use a single UPDATE ... WHERE downloads_left > 0.
  • Rejected or failed downloads should not burn a download. We count at the moment we start sending, which is a reasonable and simple choice.
  • An owner key (a second random secret returned at upload time) lets the uploader delete the file before it expires.
Node.js LinkStore: expiry, download limits and owner keys

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The results for the exhausted link, the expired link and the guessed token are all the same null, so the HTTP layer will answer all of them with the same 404. The sweep returns the file ids of links that are no longer usable, so the caller can delete the files from disk in the same pass.

Stage 3: the server with streaming downloads and Range support

Downloading is the second place where streams shine. fs.createReadStream reads a file in chunks; piping it to the response sends data as it is read, with constant memory however big the file is. A few headers make downloads behave properly:

  • Content-Length, so the browser can show a progress bar.
  • Content-Type: application/octet-stream and Content-Disposition: attachment; filename="...", which tell the browser to download the file, not to display it. This is a security measure. If you served an uploaded .html file as text/html from your own domain, any script inside would run with your site's origin: stored XSS. Add X-Content-Type-Options: nosniff so the browser does not guess a more dangerous type.
  • Accept-Ranges: bytes and support for the Range request header. A client that lost its connection at 600 MB can resume with Range: bytes=600000000- and receive just the rest, with status 206 Partial Content and a Content-Range header. Media players use the same mechanism to seek.
  • The file name from the upload is user input. Strip anything that is not safe in a header (quotes, control characters, path separators) and provide an RFC 5987 filename*=UTF-8''... form for non-ASCII names.

The program below is the full platform. It uses a temporary folder, a virtual clock for expiry, and fetch to play the client: upload, download, ranged download, expired link, size limit, deletion and cleanup.

Node.js The complete file sharing server

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Go through the output slowly. The hostile file name ../../rรฉsumรฉ "final".txt was reduced to a safe name, and the header carries both an ASCII fallback and the encoded UTF-8 form. The ranged request returned 206 with exactly the last ten bytes and a Content-Range header, and the fourth normal download was refused because the link allowed three. The oversized upload was rejected and left nothing on disk. After the virtual clock passed the expiry, the link answered 404 and the sweep deleted the file.

A file hosting service is a magnet for abuse: malware distribution, copyright infringement, phishing pages. Real services add virus scanning, per-user and per-IP quotas, rate limits, abuse reports and a takedown process. Never serve uploads from your main domain (use a separate one, such as usercontent.example.com), so that even a mistake cannot give uploaded scripts access to your site's cookies.

Going further

  • Use object storage. Store the bytes in S3 or Cloudflare R2 and hand out presigned URLs, so downloads never touch your server. The metadata and rules in this project stay the same.
  • Resumable uploads. The tus protocol and chunked upload APIs let a client continue an interrupted 5 GB upload. The server stores chunks by index and joins them.
  • Encryption. Encrypt files at rest with a per-file key (aes-256-gcm, see Encryption); or let the client encrypt before upload and keep the key in the URL fragment, which browsers never send to the server.
  • Per-user accounts and quotas with the auth building blocks from the Blog API project.
  • Rate limiting the download endpoint to slow down token guessing (Rate Limiting).
  • Cleanup as a scheduled task using the cron scheduler you built before, running sweep() every few minutes.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does the server store each upload under a random id instead of the name the client sent?

    If any part of the path comes from the client, an attacker can try to escape the folder. Random ids chosen by the server remove the whole class of attack; the original name is kept only as metadata.
  2. 2. Why is the size limit enforced inside the stream and not only by reading Content-Length?

    The header is a claim, not a guarantee. Counting the real bytes while they arrive is the only reliable check, and it lets you stop reading as soon as the limit is passed.
  3. 3. Which headers stop a browser from displaying an uploaded HTML file as a web page on your domain?

    Together they tell the browser to download the bytes and not to guess a type, which prevents stored cross-site scripting through uploaded files.
  4. 4. A client resumes a download with Range: bytes=600-. What should the server answer?

    206 with Content-Range lets the client continue where it stopped. If the range is outside the file, answer 416.
  5. 5. Why do an expired link and a link that never existed return the same 404?

    Different answers would let an attacker probe for tokens and learn about files that used to exist. Identical answers reveal nothing.

Summary

  • Stream uploads to disk through pipeline, counting and hashing in a transform; enforce limits on actual bytes and delete partial files on any failure.
  • Name stored files yourself, keep user-supplied names as sanitized metadata and write them into Content-Disposition safely.
  • Issue links with at least 128 bits of randomness, an expiry and a download limit; check and consume a download in one step, and answer every invalid case with the same 404.
  • Stream downloads with createReadStream, send Content-Length, support Range with 206 and 416, and force a download with octet-stream, attachment and nosniff.
  • Sweep expired files regularly; for real deployments use object storage with presigned URLs, quotas, rate limits and a separate domain for user content.

Next: Project: Online Code Runner with child_process