Node.js Reference ยท Concepts and Pitfalls
Backpressure in Node.js streams
Why Node.js streams need flow control, how write() returning false and the drain event work, and how pipeline() and async iteration handle it for you.
Backpressure is the way a slow consumer tells a fast producer to wait. Without it, a program that reads a huge file quickly and writes it to a slow network connection would pile the whole file into memory, because the writer cannot keep up with the reader.
In Node.js streams, backpressure is a small protocol built on two signals: the return value of writable.write() and the 'drain' event. Each stream has a buffer limit, the highWaterMark. When the buffered data reaches it, write() returns false: "stop sending". When the buffer has emptied, the stream emits 'drain': "you may continue".
if (!writable.write(chunk)) { await once(writable, 'drain'); }
Try it
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.
How the protocol works
- A writable keeps incoming data in an internal queue and passes chunks to its
_write()implementation one at a time. write()returnstruewhile the queued size is belowhighWaterMarkandfalseonce it reaches or exceeds it. The data is still accepted; thefalseis advice, not a refusal.- When the queue empties after a
false,'drain'fires exactly once. - A readable-to-writable
pipe()implements this loop for you: it pauses the readable onfalseand resumes it on'drain'.
Because the stream always accepts the data, a producer that ignores false still works, but its memory use grows with every unread chunk. The second example shows the careless loop buffering all 2000 bytes at once, while the careful one stays within a few chunks.
Usage notes
- Prefer
stream.pipeline()over manualwrite()loops: it handles backpressure, forwards errors and destroys every stream on failure. Seestream.pipeline(). - Async iteration respects it too.
for await (const chunk of readable)pulls data only as fast as your loop body finishes. highWaterMarkis a threshold, not a hard cap. For byte streams it counts bytes; in object mode it counts objects. See highWaterMark.- Readable streams have their own side:
push()returnsfalsewhen the internal buffer is full and_read()is called again when there is room. - Do not mix
datalisteners withpipe()on the same stream in ways that bypass pausing. - A missing
'drain'wait is a classic leak. Servers that copy a large upload to a slow disk or a slow client can run out of memory when each write ignores the result. - Check
writableNeedDrainif you need to know whether a drain is pending without tracking the last return value.