Skip to content
Tutorials Center
My learning

Topics

Linux Command Reference ยท Concepts

Pipes and subshell variables

Why a variable set inside a while loop at the end of a pipe is lost in Bash, which constructs create subshells, and the fixes: process substitution, lastpipe, here-strings.

A classic puzzle: you count lines in a loop that reads from a pipe, and after the loop the counter is still zero. Nothing is wrong with your arithmetic. In Bash, each stage of a pipeline runs in its own subshell, a forked copy of the shell. A copy can change its variables, but the changes die with it and never flow back to the parent.

Knowing which constructs create subshells tells you when a variable assignment will be lost and what to do about it.

Terminal
count=0
printf 'a\nb\n' | while read -r l; do count=$((count+1)); done
echo "$count"     # prints 0

Try it

Bash The bug and three fixes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Bash Which constructs make a subshell

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

What creates a subshell

Construct Subshell? Variable changes survive?
( commands ) yes no
{ commands; } no yes
$( commands ) and backticks yes no
Every stage of a | b | c yes (even the last, unless lastpipe) no
cmd & (background) yes no
<( cmd ) and >( cmd ) yes (the command inside) no
coproc yes no
exec cmd replaces the shell (no return) not applicable
source file / . file no yes
Running a script ./script a new process, not a subshell no, except through exported variables going down
while ... done < file no yes
for, if, functions no yes

A subshell inherits variables, functions, options, the working directory and open files. It cannot send anything back except through output, exit status or files.

The fixes in order of preference

  1. Redirect instead of piping: while read ...; done < file or < <(command). This keeps the loop in the main shell and the producer in the subshell, which is the right way round.
  2. Here-string <<< "$var" for data already in a variable.
  3. shopt -s lastpipe runs the last pipeline stage in the current shell. It works in scripts and non-interactive shells; interactive shells have job control on, which disables it unless you run set +m.
  4. Collect with command substitution: count=$(... | wc -l). Let the pipeline compute the number and capture the output instead of mutating a variable inside.
  5. Write to a file or use a fifo if several processes have to communicate.

Usage notes

  • cd inside a subshell does not move you. That is a feature: (cd dir && make) runs the build and leaves your directory alone.
  • Exit status of a pipeline is that of its last stage, unless set -o pipefail. PIPESTATUS lists every stage's status, but only until the next command.
  • set -e and subshells: an error inside $( ) or ( ) stops that subshell, not necessarily the script.
  • Arrays and associative arrays are lost the same way. Fill them from mapfile -t arr < <(cmd).
  • Functions in pipes run in subshells too, so echo x | f where f sets a global will not set it.
  • Other shells differ: zsh and ksh run the last pipeline stage in the current shell, which is why this "bug" seems not to exist there. Do not rely on it in portable scripts.
  • Debugging: echo $BASHPID shows the real process ID of the current (sub)shell, whereas $$ always shows the original shell's ID.