Skip to content
Tutorials Center
My learning

Topics

Lesson 76 of 95 ยท Practice and Review

Practice Problems: Scripting and Pipelines

15 graded Bash scripting problems with hints and verified solutions: functions, arrays, getopts, regex, traps, retries, background jobs, xargs and pipelines.

The first set of problems was about commands. This set is about scripts: functions that return values, arrays, option parsing, error handling, parallel jobs and the pipeline pitfalls that catch everyone once. It is ordered from beginner to advanced inside each group and assumes you have worked through the Shell Scripting and Advanced Bash chapters.

Most problems come with a stub: a function that only prints TODO, plus the test calls that exercise it. Replace the body of the stub, press Run, and compare your output with the comments at the bottom. A hidden solution follows each problem, with a short explanation of the idea.

  • Run the tests, not just the function. The calls below your code are the specification. If one prints something different, your function is wrong, not the test.
  • Think about the unusual inputs. Empty lists, names with spaces, a missing trailing newline and values like 09 are what the tests probe, and what real scripts meet.
  • One reference style. Solutions use [[ ]], quoted variables, local, and printf instead of echo -e. You may solve a problem differently as long as the output matches.
  • Every editor is a fresh sandbox. Files you create disappear after the run, so nothing you try can do harm.

If you get stuck, the answers are in Functions, Arrays, Exit Codes, set -e and Error Handling and Performance, Tricks and Pitfalls. The cheat sheet has the syntax, and the interview questions are a good next step.

Functions, parameters and arrays

These problems practise functions, parameter expansion, arrays and option parsing. In most of them the editor contains a stub: a function that prints a placeholder. Replace its body, then compare the output with the comments.

1. Clamp a number (Beginner)

Write the function clamp MIN MAX VALUE that prints VALUE limited to the range from MIN to MAX: values below MIN become MIN, values above MAX become MAX.

Hint: arithmetic comparison with (( value < min )) works on integers; use local for the variables.

Bash Exercise 1: Clamp a number

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Show solution
Terminal
clamp() {   # clamp MIN MAX VALUE
  local min=$1 max=$2 value=$3
  (( value < min )) && value=$min
  (( value > max )) && value=$max
  echo "$value"
}

(( cond )) && action runs the action only when the arithmetic test is true. Because the variables are local, the function does not leak min, max or value into the caller. Negative numbers work because (( )) treats its contents as an arithmetic expression, not as strings.

2. Split a path into name and extension (Intermediate)

Write split_name PATH that prints the file name without its last extension and the extension, separated by a space, using only parameter expansion (no basename, sed or other commands). When there is no extension (or the name only starts with a dot, like .bashrc), print - as the extension.

Hint: ${var##*/} removes everything up to the last slash; ${var%.*} and ${var##*.} split at the last dot; the pattern ?*.* means "at least one character, then a dot somewhere".

Bash Exercise 2: Split a path into name and extension

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Show solution
Terminal
split_name() {   # split_name PATH -> "NAME EXTENSION"
  local base=${1##*/}
  if [[ $base == ?*.* ]]; then
    printf '%s %s\n' "${base%.*}" "${base##*.}"
  else
    printf '%s -\n' "$base"
  fi
}

${1##*/} strips the longest prefix ending in a slash, so only the file name remains. ${base%.*} removes the shortest suffix of the form .something, i.e. only the last extension, while ${base##*.} removes the longest prefix ending in a dot, leaving the extension. For .bashrc the test ?*.* fails (the only dot is the first character), so it is treated as a name without an extension.

3. Join array elements (Intermediate)

Write join_by SEPARATOR ITEM... that prints all items separated by the separator, without a separator before the first or after the last item. It must handle zero items (print an empty line) and items that contain spaces.

Hint: take the separator with $1 and shift, start the result with the first item, then append "$sep$item" for the rest.

Bash Exercise 3: Join array elements

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Show solution
Terminal
join_by() {   # join_by SEPARATOR ITEM...
  local sep=$1 out item
  shift
  if (( $# > 0 )); then
    out=$1
    shift
    for item in "$@"; do
      out+="$sep$item"
    done
  fi
  printf '%s\n' "$out"
}

out+=... appends to a string variable. Handling the first item separately avoids the classic leading-separator bug. Passing "${fruits[@]}" keeps each element as one argument even when it contains spaces. The same job in one line is (IFS=,; echo "${arr[*]}"), which works for single-character separators only.

4. Count words with an associative array (Intermediate)

Write count_words that reads text from standard input (the Input box), counts how often each word occurs, and prints WORD COUNT for each word in alphabetical order.

Hint: declare -A count creates an associative array; ${!count[@]} lists its keys in no useful order, so pipe the keys through sort.

Bash Exercise 4: Count words with an associative array

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Show solution
Terminal
count_words() {
  declare -A count
  local line word
  local -a words
  while IFS= read -r line || [[ -n $line ]]; do
    read -ra words <<< "$line"
    for word in "${words[@]}"; do
      count[$word]=$(( ${count[$word]:-0} + 1 ))
    done
  done
  for word in $(printf '%s\n' "${!count[@]}" | sort); do
    printf '%s %d\n' "$word" "${count[$word]}"
  done
}

|| [[ -n $line ]] handles a last line without a trailing newline: read then returns failure but still fills line. ${count[$word]:-0} supplies 0 for a word seen for the first time. Iteration order of an associative array is unspecified, which is why the keys are sorted before printing.

5. Parse options with getopts (Intermediate)

Write cli [-v] [-n NAME] [-c COUNT] [FILE...]. It prints hello NAME COUNT times (default name world, default count 1). With -v it adds a line files: ... listing the remaining arguments. A COUNT that is not a positive whole number, a missing option value and an unknown option must each print a message to standard error and make the function return 2.

Hint: start getopts with : to handle errors yourself; inside the loop the cases : (missing value) and * (unknown option) get the option letter in OPTARG. Declare local OPTIND=1.

Bash Exercise 5: Parse options with getopts

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Show solution
Terminal
cli() {
  local OPTIND=1 opt verbose=0 name=world count=1 i
  while getopts ':vn:c:' opt; do
    case $opt in
      v) verbose=1 ;;
      n) name=$OPTARG ;;
      c) [[ $OPTARG =~ ^[1-9][0-9]*$ ]] || { echo "cli: -c needs a positive number" >&2; return 2; }
         count=$OPTARG ;;
      :) echo "cli: option -$OPTARG needs a value" >&2; return 2 ;;
      *) echo "cli: unknown option -$OPTARG" >&2; return 2 ;;
    esac
  done
  shift $((OPTIND - 1))
  for (( i = 0; i < count; i++ )); do
    echo "hello $name"
  done
  if (( verbose )); then echo "files: $*"; fi
  return 0
}

With a leading : in the option string, getopts stays silent and reports problems through opt: : means a missing value, ? an unknown option (our * branch). local OPTIND=1 matters because getopts remembers its position between calls, and the function is called several times. Ending with an explicit return 0 avoids returning the status of the last if.

Control flow and robustness

These problems practise loops over files, exit codes and traps, regular expressions in Bash, and reading input.

6. Loop over files with spaces (Beginner)

The loop below breaks on file names with spaces because it parses the output of ls. Fix it: print found: NAME for every .txt file (names with spaces intact), followed by total: N.

Hint: never loop over $(ls ...). A glob in the for list expands to exactly one word per file, and you must quote "$f" when you use it.

Bash Exercise 6: Loop over files with spaces

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Show solution
Terminal
count=0
for f in *.txt; do
  echo "found: $f"
  count=$((count + 1))
done
echo "total: $count"

$(ls *.txt) produces text that the shell then splits at every space, so my file.txt becomes two words. A glob is expanded by the shell as separate words, with no splitting afterwards, so each file name stays intact. Globs expand in sorted order, which makes the output predictable.

7. Retry a flaky command (Intermediate)

Write retry MAX COMMAND... that runs the command up to MAX times until it succeeds. After each failed attempt print attempt N failed. On success print succeeded on attempt N and return 0; if all attempts fail return 1.

Hint: until "$@"; do ... done runs the command as the loop condition, so the loop body only runs after a failure.

Bash Exercise 7: Retry a flaky command

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Show solution
Terminal
retry() {   # retry MAX COMMAND...
  local max=$1 attempt=1
  shift
  until "$@"; do
    echo "attempt $attempt failed"
    (( attempt >= max )) && return 1
    attempt=$((attempt + 1))
  done
  echo "succeeded on attempt $attempt"
}

Using "$@" runs the command with its arguments intact. Because retry calls flaky directly (no subshell), the counter file changes between attempts. In real life you would add a sleep that grows with the attempt number (exponential backoff), which we leave out to keep the run fast.

8. Clean up after failure with trap (Intermediate)

Write process as a function whose body is a subshell ( ... ). Inside it: enable set -e, create a temporary directory with mktemp -d, record its path in the file workdir.txt (in the original directory), register an EXIT trap that deletes the temporary directory and prints cleanup done, then cd into it, print working in a temp dir and run false (a failing command) followed by an echo that must never run. The caller prints the function's status and whether the temporary directory still exists.

Hint: the trap runs when the subshell exits, whatever the reason; set -e makes false end the subshell with status 1.

Bash Exercise 8: Clean up after failure with trap

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Show solution
Terminal
process() (
  set -e
  work=$(mktemp -d)
  trap 'rm -rf -- "$work"; echo "cleanup done"' EXIT
  echo "$work" > workdir.txt
  cd "$work"
  echo "working in a temp dir"
  false
  echo "not reached"
)

A function body in parentheses is a subshell, so set -e and the trap affect only that function. The EXIT trap fires when the subshell ends after false failed, which removes the directory even though the work did not finish. The status 1 is false's status, passed through by set -e.

9. Parse a date with a regex (Intermediate)

Write parse_date TEXT for dates in the form YYYY-MM-DD (always two digits for month and day). On a match print year=Y month=M day=D with month and day as plain numbers without leading zeros. Otherwise print not a date: TEXT to standard error and return 1.

Hint: [[ $text =~ ^([0-9]{4})-([0-9]{2})-([0-9]{2})$ ]] fills the array BASH_REMATCH. A number like 09 is read as invalid octal by arithmetic and printf %d, unless you force base 10 with 10#09.

Bash Exercise 9: Parse a date with a regex

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Show solution
Terminal
parse_date() {
  if [[ $1 =~ ^([0-9]{4})-([0-9]{2})-([0-9]{2})$ ]]; then
    printf 'year=%d month=%d day=%d\n' \
      "${BASH_REMATCH[1]}" "$((10#${BASH_REMATCH[2]}))" "$((10#${BASH_REMATCH[3]}))"
  else
    echo "not a date: $1" >&2
    return 1
  fi
}

BASH_REMATCH[0] is the whole match and [1]..[3] are the parenthesized groups. The 10# prefix is the fix for the leading-zero problem: without it $((09)) fails with "value too great for base". The regex only checks the shape; it does not know that 2026-13-45 is not a real date. For that you need date -d.

10. Read a CSV without a trailing newline (Advanced)

items.csv contains name,quantity,price_in_cents lines, and its last line has no newline. Write report FILE that prints for every item NAME QTY x PRICE = TOTAL (prices in dollars with two decimals, name left-aligned in 9 columns) and then total: AMOUNT. Use integer arithmetic only.

Hint: while IFS=, read -r name qty cents || [[ -n $name ]]; do reads the final unterminated line too. Format cents as $((x / 100)).$((x % 100)) with %d.%02d.

Bash Exercise 10: Read a CSV without a trailing newline

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Show solution
Terminal
report() {   # report FILE (prices in cents)
  local name qty cents total=0 line_total
  while IFS=, read -r name qty cents || [[ -n $name ]]; do
    line_total=$(( qty * cents ))
    total=$(( total + line_total ))
    printf '%-9s %d x %d.%02d = %d.%02d\n' "$name" "$qty" \
      $((cents / 100)) $((cents % 100)) $((line_total / 100)) $((line_total % 100))
  done < "$1"
  printf 'total: %d.%02d\n' $((total / 100)) $((total % 100))
}

Money in integer cents avoids floating-point rounding, and Bash arithmetic is integer-only anyway. When the last line has no newline, read returns a failure status but has already stored the fields, so the || [[ -n $name ]] test keeps the loop going one more time. IFS=, applies only to that read, because it is a prefix assignment.

Pipelines, processes and design

The last group uses process substitution, background jobs, xargs, subshells and the dry-run pattern from Designing Maintainable Scripts.

11. Compare two directories with comm (Intermediate)

The directories a and b contain different files. Print three lines: only in a: ..., only in b: ... and in both: ..., each listing the file names on one line separated by spaces. Use comm with process substitution, not temporary files.

Hint: comm -23 shows lines only in the first input, -13 only in the second and -12 common lines. Both inputs must be sorted. paste -sd' ' joins lines with a space.

Bash Exercise 11: Compare two directories with comm

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Show solution
Terminal
echo "only in a: $(comm -23 <(ls a) <(ls b) | paste -sd' ')"
echo "only in b: $(comm -13 <(ls a) <(ls b) | paste -sd' ')"
echo "in both:   $(comm -12 <(ls a) <(ls b) | paste -sd' ')"

<(ls a) is replaced by a file name (like /dev/fd/63) that yields the command's output, so comm sees two "files" without anything being written to disk. ls output is sorted in the C locale, which comm requires. In a script, add LC_ALL=C so the order does not depend on the user's locale.

12. Run jobs in parallel and collect failures (Advanced)

The function job N takes a short while, writes job N done to result-N.txt and fails when N is 3. Start jobs 1 to 4 in the background, remember their process IDs, then wait for each one individually. Print finished 4 jobs; failed: job 3 (or failed: none when nothing failed), then show all result files.

Hint: $! holds the PID of the last background job. wait PID returns that job's exit status. Store PIDs in an array.

Bash Exercise 12: Run jobs in parallel and collect failures

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Show solution
Terminal
pids=()
for i in 1 2 3 4; do
  job "$i" &
  pids+=($!)
done
failed=()
for i in "${!pids[@]}"; do
  if ! wait "${pids[i]}"; then
    failed+=("job $((i + 1))")
  fi
done
echo "finished ${#pids[@]} jobs; failed: ${failed[*]:-none}"
cat result-*.txt

job "$i" & starts a background process and $! is available immediately. Waiting for each PID in turn does not slow anything down: jobs that finished earlier are simply collected. A bare wait would succeed without telling you who failed. ${failed[*]:-none} prints none when the array is empty.

13. Count lines of many files safely (Intermediate)

Print LINES FILE for every .txt file under data (including the directory with a space in its name), sorted by file name. File names must survive intact, so use NUL-separated names.

Hint: find ... -print0 | xargs -0 -n 1 wc -l, then sort -k2.

Bash Exercise 13: Count lines of many files safely

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Show solution
Terminal
find data -name '*.txt' -print0 | xargs -0 -n 1 wc -l | sort -k2

-print0 and -0 separate the names with a NUL byte, the only character that cannot appear in a file name, so spaces and even newlines are safe. -n 1 runs wc once per file, so no total line appears. sort -k2 sorts from the second whitespace-separated field to the end of the line, which includes the whole path.

14. The variable that disappears in a pipeline (Intermediate)

This snippet counts lines but prints count: 0. Explain to yourself why, then change it so that it prints count: 3, still reading the lines of sort list.txt one by one in a while read loop.

Hint: every stage of a pipeline runs in its own subshell, and a subshell's variable changes are lost. Feed the loop with < <(command) instead of command |.

Bash Exercise 14: The variable that disappears in a pipeline

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Show solution
Terminal
count=0
while read -r line; do
  count=$((count + 1))
done < <(sort list.txt)
echo "count: $count"

In sort ... | while ... the loop is a pipeline stage and therefore runs in a child shell; when it ends, its copy of count disappears. With < <(sort list.txt) the loop runs in the current shell and only the sort is a separate process. Another fix is shopt -s lastpipe (in scripts, where job control is off), which runs the last pipeline stage in the current shell.

15. A dry-run wrapper with printf %q (Advanced)

Write run COMMAND... for a script with a variable DRY_RUN. When DRY_RUN=1 it prints [dry-run] followed by the command, each argument quoted so that it could be pasted back into a shell, and does not run it. Otherwise it runs the command. The test calls show both modes.

Hint: printf ' %q' "$@" prints every argument quoted with backslashes where needed.

Bash Exercise 15: A dry-run wrapper with printf %q

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Show solution
Terminal
run() {
  if (( DRY_RUN )); then
    printf '[dry-run]'
    printf ' %q' "$@"
    printf '\n'
  else
    "$@"
  fi
}

printf ' %q' "$@" reuses the format for every argument, quoting spaces, quotes and other special characters, so the printed line shows exactly what would have run. In dry-run mode nothing is deleted: ls still shows tmp dir. The same one-function pattern is used in the backup and deployment projects.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does sort list.txt | while read -r line; do count=$((count + 1)); done leave count unchanged in the calling shell?

    The loop is a pipeline stage, so Bash runs it in a child shell. Reading with done < <(sort list.txt) keeps the loop in the current shell.
  2. 2. What does the 10# prefix in $((10#09)) do?

    Numbers with a leading zero are octal in Bash arithmetic, so 09 is an error. 10#09 says explicitly: this is decimal.
  3. 3. How can a script collect the exit status of each background job separately?

    wait PID blocks until that process ends and returns its exit status; plain wait returns 0 once all jobs are done.
  4. 4. Why is for f in $(ls *.txt) wrong for file names containing spaces?

    Unquoted command substitution is subject to word splitting. A glob in the for list gives one intact word per file.
  5. 5. In retry, why is until "$@" used instead of running the command inside $(...)?

    until "$@" runs the command directly in the current shell and tests its exit status, which is exactly what a retry loop needs.

Summary

  • Pipeline stages run in subshells; feed loops with < <(command) (or use lastpipe) when they must change variables.
  • getopts with a leading colon lets you write your own error messages; declare local OPTIND=1 in functions.
  • Use 10#NN for numbers that may have leading zeros, || [[ -n $var ]] for files without a final newline, and globs, never ls, for file loops.
  • Subshell function bodies f() ( ... ) keep set -e, traps and cd local; an EXIT trap cleans up on every path.
  • Collect background results with pids+=($!) and wait "${pids[i]}".
  • printf ' %q' "$@" shows commands exactly as they would be typed, which is what a dry run needs.

Next: Linux and Bash Interview Questions and Answers