Skip to content
Tutorials Center
My learning

Topics

Lesson 62 of 133 ยท Processes and Threads

Worker Threads

Use Node.js worker_threads to run CPU-heavy JavaScript in parallel. Learn Worker, workerData, messages, transferables, SharedArrayBuffer, Atomics, worker pools and when to choose them.

JavaScript in Node.js runs your code on one thread. That is wonderful for I/O-heavy programs, because the event loop can juggle thousands of waiting connections. It is a problem for CPU-heavy work: while a loop crunches numbers, compresses an image or parses a huge file, nothing else can run. Timers fire late, requests wait, and the whole server feels frozen.

The node:worker_threads module solves this by letting you start extra threads, each with its own JavaScript engine instance, its own event loop and its own memory. Threads talk to each other by passing messages, and can share raw memory when you ask for it explicitly. In this lesson you will start workers, send them data, move big buffers without copying, share counters safely with Atomics, and build a small worker pool.

A worker is not a child process. It lives inside your Node process, starts in a few milliseconds instead of tens, and can share memory. A bug in a worker cannot corrupt your main thread's variables, because nothing is shared unless you share it. If you have not read Child Processes yet, a quick look makes the comparison at the end of this lesson clearer.

Your first worker

A worker runs a JavaScript file. Our editor has a single file, so the program starts itself as the worker and uses isMainThread to decide which role it plays. In a normal project the worker is usually a separate file, new Worker(new URL('./worker.js', import.meta.url)).

Node.js A first worker with workerData and messages

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Follow the flow:

  1. new Worker(path, options) starts a new thread that runs the file from the top. The same file therefore runs twice: once on the main thread, once in the worker.
  2. workerData is a copy of the data you pass in options, available in the worker as the named import workerData.
  3. parentPort is the worker's end of the message channel to its parent. postMessage(value) sends a value; the parent gets it in a 'message' event.
  4. When the worker has nothing left to do (no pending timers, no open message listeners), it exits, and the parent sees 'exit' with the exit code.

Messages are copied using the structured clone algorithm, the same one behind structuredClone(). Plain objects, arrays, strings, numbers, Map, Set, Date, BigInt, typed arrays and errors work. Functions, class instances' methods and symbols do not: posting a function throws a DataCloneError.

The worker has its own global scope. Variables, module state and caches in the main thread are not visible there. Each worker also loads its own copy of every module it imports, which costs a little memory and start-up time.

Why bother: CPU-bound work

Threads help when the bottleneck is computation, not waiting. Counting prime numbers is a classic example: it is pure arithmetic with no I/O. We will count the primes below 200,000 by splitting the range into four slices and giving each slice to its own worker.

Node.js Splitting CPU work across four workers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Promise.all returns the results in the order of the jobs, so the output is the same on every run even though the workers finish at different times. We never print how long it took, because timings differ between machines. On a computer with several cores the four slices really run in parallel. Our sandbox has a single CPU, so you would see no speed-up here, but on your own machine the work finishes in roughly a quarter of the single-thread time plus a small start-up cost.

The important lesson is the first printed line: the main thread did not block. In a server, that means other requests keep being served while a worker handles the heavy request.

Do not use threads to make I/O faster. Reading files, calling APIs and querying databases already run outside the JavaScript thread (see How Node.js Works). Adding workers to an I/O-bound program adds overhead and no speed.

Handling errors and exits

An uncaught exception inside a worker does not crash the main thread. It terminates that worker and surfaces as an 'error' event on the Worker object. If you have no listener, the error is thrown in the main thread like any EventEmitter error, so always attach one.

Node.js Errors and exit codes from workers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The worker that throws reports exit code 1, process.exit(5) inside a worker ends only that thread with the code you gave, and worker.terminate() stops a thread from outside (it returns a promise for the exit code). Because terminate() is abrupt, the worker gets no chance to clean up, which is why it is best kept for stuck or cancelled work.

Transferables: moving memory instead of copying it

Copying is fine for small messages. For a 100 MB buffer, copying wastes time and doubles memory use. An ArrayBuffer can be transferred: ownership moves to the receiver in constant time, and the sender's view of it becomes empty. List the buffers to transfer as the second argument of postMessage.

Node.js Copying versus transferring an ArrayBuffer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

After the transfer the original Uint8Array has byteLength 0: the memory now belongs to the worker. Using a transferred buffer afterwards is a bug that shows up as empty arrays, not as an exception, so treat a transferred buffer as "gone". This is the technique behind fast image, audio and file processing in workers. (Node's Buffer objects from file reads usually come from a shared internal pool, so transfer new Uint8Array(size).buffer or a .slice() copy rather than a pooled Buffer.buffer.)

Note how the worker keeps running while it has a 'message' listener. A listener on parentPort keeps the thread alive, so the worker must receive a "done" message and call parentPort.close(), or the parent must call terminate().

Sharing memory with SharedArrayBuffer and Atomics

Messages copy or move. A SharedArrayBuffer is different: the same block of memory is visible to several threads at once. That is the fastest way to share a large table, a counter or a flag, and also the easiest way to create race conditions.

Consider counter[0]++. It looks like one step but is three: read, add, write. If two threads do it simultaneously, one update can overwrite the other and the final total is too small, in a different way on every run. The Atomics object provides operations that are indivisible: Atomics.add, Atomics.load, Atomics.store, Atomics.compareExchange, and Atomics.wait/Atomics.notify for waiting.

Node.js Four workers, one shared counter

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The total is exactly 100,000 on every run. If you replaced Atomics.add(counter, 0, 1) with counter[0]++, the result would usually be smaller and would change from run to run on a multi-core machine. We do not run that version here because its output is not repeatable, but you can try it on your own computer.

Shared memory also allows a lock-free signal: one thread calls Atomics.wait(int32, 0, 0) to sleep until another calls Atomics.notify(int32, 0). Use Atomics.wait only inside workers, never on the main thread, because it blocks the thread. (Atomics.waitAsync is the non-blocking variant for the main thread.)

Shared memory is for experts and for hot paths where copying has been measured to be too slow. Start with messages. They are safe, simple to reason about and fast enough for most programs. Only numbers and bytes can live in a SharedArrayBuffer; you cannot share objects, strings or functions.

A worker pool

Starting a worker is cheaper than starting a process, but it still costs a few milliseconds and several megabytes. Creating a new worker for every request would be slow and could use all your memory. The standard solution is a pool: a fixed number of workers that are reused, with a queue for tasks that arrive while all workers are busy.

This pool keeps size workers, each running the same file in worker mode, and hands out tasks one at a time.

Node.js A small reusable worker pool

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Each worker runs one task at a time, so its 'message' event always matches the task it was given. In a real pool you would also add a timeout per task, replace workers that crash, and expose the queue length for monitoring. The popular piscina package does all of this (read-only example, it is not installed in our editor):

JavaScript
// Read-only: needs `npm install piscina`
import Piscina from 'piscina';

const pool = new Piscina({ filename: new URL('./worker.js', import.meta.url).href });
const result = await pool.run({ word: 'node' });

A good pool size for CPU-bound work is the number of CPU cores (os.availableParallelism()), because more threads than cores only add context switching.

Message channels and ports

parentPort is just one end of a MessagePort. You can create your own channel with new MessageChannel() and send one of its ports to a worker, so that two workers (or a worker and a library) can talk to each other directly without going through the main thread. Ports are themselves transferable: worker.postMessage({ port: port2 }, [port2]). This is how you wire up pipelines of specialised workers. Remember that an open port with a listener keeps its thread alive until you call port.close().

Workers, child processes or cluster?

worker_threads child_process cluster
Isolation Separate JS heap, same OS process Separate OS process Separate OS processes
Start-up cost Low (milliseconds) Higher (tens of ms) Higher
Share memory Yes (SharedArrayBuffer) No No
Crash impact Worker dies; the process survives if you handle 'error' Only the child dies Only that worker process dies
Typical use CPU-heavy functions: hashing, parsing, image work Running other programs; hard isolation Scaling an HTTP server across cores

Rules of thumb: use a worker thread when you have a JavaScript function that burns CPU; a child process when you need another program or strong isolation; cluster (next lesson: Cluster, Scaling and Load Balancing) when you want many copies of a whole server. Also remember that some Node APIs already use background threads for you: crypto.pbkdf2, zlib and file-system calls run on libuv's thread pool, so you rarely need your own worker for them.

Pitfalls professionals watch for

  • Starting a worker per task. Use a pool.
  • Forgetting to close. A worker with a live parentPort listener, timer or open handle never exits, and neither may your program.
  • Huge messages. Cloning a 500 MB object blocks the sender while it serialises. Transfer buffers or share memory.
  • Unhandled 'error'. Always listen for it, and decide whether to restart the worker.
  • Assuming shared state. A Map or a database connection in the main thread does not exist in the worker. Each worker needs its own.
  • Debugging and ordering. Console output from different threads interleaves in any order, which is why this lesson printed only after collecting results.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. When are worker threads the right tool?

    Workers help with computation. I/O such as database calls and fetch already runs without blocking the main thread, so threads add overhead there.
  2. 2. What happens to a Uint8Array after you transfer its buffer to a worker with postMessage(value, [value.buffer])?

    Transferring moves ownership of the memory to the receiver in constant time. The sender's view is detached, so its length becomes 0.
  3. 3. Why use Atomics.add(counter, 0, 1) instead of counter[0]++ on a SharedArrayBuffer?

    counter[0]++ is read, add, write. Two threads can interleave those steps and overwrite each other. Atomics operations are indivisible.
  4. 4. A worker throws an uncaught exception. What is the effect on the main thread?

    Errors in a worker end that worker only. The Worker object emits 'error', and it is your job to handle it, for example by logging and starting a replacement.
  5. 5. Why do applications use a worker pool instead of creating a new Worker per task?

    Each worker has its own engine instance with start-up time and memory cost. A pool reuses a bounded number of workers and queues extra tasks.

Summary

  • worker_threads runs JavaScript in parallel threads, each with its own heap and event loop. Use it for CPU-bound work, not for I/O.
  • Start a worker with new Worker(file, { workerData }); communicate with postMessage and parentPort. Messages are copied with structured clone.
  • Listen for 'error' and 'exit'; a crash in a worker does not take down the main thread. terminate() stops a worker abruptly.
  • Transfer ArrayBuffers to move memory without copying (the sender's copy becomes empty); use SharedArrayBuffer with Atomics when threads must share memory.
  • Reuse workers through a pool sized near the number of CPU cores, and make sure workers can exit.
  • Choose workers for CPU-heavy functions, child processes for other programs or isolation, and cluster for scaling servers.

Next: Cluster, Scaling and Load Balancing