Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 95 ยท Shell Scripting

Parameter Expansion and String Handling

Handle strings in pure Bash: defaults ${var:-x}, ${var:?}, length, substrings, removing prefixes and suffixes, replace, case conversion, trimming and path parts.

You often need to change a piece of text: cut the extension off a filename, use a default when a value is missing, make a word upper-case, find the part of a path after the last slash. The beginner's answer is to start sed, cut or basename in a pipe. The Bash answer is parameter expansion: special forms of ${...} that transform a variable's value on the spot, without starting another process.

Parameter expansion is faster (no fork), safer (no word-splitting surprises when you quote), and often shorter. It is also dense and cryptic at first sight, because the syntax is made of punctuation. This lesson groups the forms by purpose and shows each with real output, so you can recognise them when you read other people's scripts.

Defaults and checks

Four forms deal with variables that might be unset or empty. They differ in what they do:

Form If var is unset or empty Changes var?
${var:-word} expands to word no
${var:=word} assigns word to var, then expands to it yes
${var:?message} prints message to stderr and aborts the script (non-interactive) no
${var:+word} expands to word only if var is set and non-empty, else to nothing no

Without the colon (${var-word}) the test is only "is it unset?", so an empty value counts as set.

Bash Default values

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The typical uses: ${PORT:-8080} for a setting with a default, : "${LOG_DIR:=/var/log/app}" to set a default once (the : is the do-nothing command), and ${flag:+--flag} to add an option only when wanted.

${var:?message} is the safest check at the start of a script. Used in a command substitution it stops the script with a clear message:

Bash Requiring a value with :? (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The second call prints main.sh: line 1: 1: usage: require TARGET (the script name, the line and the parameter name come first) and the subshell exits with status 1. In a real script this is how you guard dangerous commands: rm -rf -- "${build_dir:?}/"* can never turn into rm -rf /*.

Length and substrings

${#var} is the number of characters in the value. ${var:offset:length} extracts a substring; offsets start at 0, and a negative offset counts from the end (put a space or parentheses before the minus, so it is not read as :-).

Bash Length and substrings

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Positional parameters work as well: ${1:0:3} gives the first three characters of the first argument, and ${@:2} expands to all arguments from the second on.

Removing prefixes and suffixes

These four operators delete text that matches a glob pattern (the shell kind with * and ?, not a regular expression) at the start or the end of the value:

Form Meaning
${var#pattern} remove the shortest match from the beginning
${var##pattern} remove the longest match from the beginning
${var%pattern} remove the shortest match from the end
${var%%pattern} remove the longest match from the end

A mnemonic: on a US keyboard # is left of %, just as "beginning" is left of "end"; and the doubled symbol means "greedy".

TEXT
   path=/home/ann/docs/report.final.txt

   ${path##*/}   removes the longest  "*/" from the front   ->  report.final.txt
   ${path%/*}    removes the shortest "/*" from the back    ->  /home/ann/docs
   ${path%.*}    removes the shortest ".*" from the back    ->  /home/ann/docs/report.final
   ${path%%.*}   removes the longest  ".*" from the back    ->  /home/ann/docs/report
   ${path##*.}   removes the longest  "*." from the front   ->  txt
Bash Prefixes, suffixes, filenames and extensions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

These forms are the Bash way to split paths, and they replace basename, dirname and many cut and sed calls. Mind the edge cases: basename and dirname treat trailing slashes and the empty string specially, and ${path%/*} does not (for file.txt without any slash it returns file.txt unchanged). If a script must handle every odd input, use dirname/basename.

Search and replace

${var/pattern/replacement} replaces the first match of a glob pattern; ${var//pattern/replacement} replaces all matches. With # or % after the first slash the pattern must match at the start or end; an empty replacement deletes the match.

Bash Replace inside a variable

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

For anything needing regular expressions, use sed -E (sed: The Stream Editor) or the [[ =~ ]] operator.

Case conversion

Bash 4 and later changes case without tr:

Form Effect
${var^} first character to upper case
${var^^} all characters to upper case
${var,} first character to lower case
${var,,} all characters to lower case
Bash Changing case

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

macOS ships Bash 3.2 by default, which lacks these. On such systems install a newer Bash or use tr '[:lower:]' '[:upper:]'. See POSIX sh, Portability and Compatibility.

Trimming whitespace

There is no built-in trim, but the pattern operators can do it with the extglob shell option and the character class [:space:]:

Bash Trim leading and trailing whitespace

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

+(...) is an extended glob meaning "one or more of". It needs shopt -s extglob, and the option must be on before the function is defined or the line is parsed. Wildcards and Globbing covers extglob.

Building a string safely

Putting values together is just writing them next to each other inside double quotes. Add += to append:

Bash Join, append and repeat

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The last loop uses ${list:+, } to put a comma only if the list already has content, a very common idiom for joining items.

Arrays and positional parameters

The same operators work on each element when you write [@] or [*] (more in Arrays and Associative Arrays):

Bash Expansion applied to every element

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Indirection and other forms

A few more forms appear in real scripts:

  • ${!name} is indirect expansion: the value of the variable whose name is stored in name. ${!prefix*} lists variable names with a prefix; ${!array[@]} lists an array's indices.
  • ${var@Q} quotes the value so it can be reused as shell input (Bash 4.4+), ${var@U} and ${var@L} change case, ${var@E} interprets backslash escapes (Bash 5).
  • ${var:offset} on a variable that holds a number works as a string, not arithmetic.
Bash Indirection and @Q

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Indirect expansion is handy but obscure; modern Bash prefers associative arrays or local -n namerefs (Functions).

String comparison and testing

Strings are compared inside [[ ]] (more in Conditionals). The parameter forms above combine with these tests:

Bash Comparing and testing strings

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does ${name:-guest} do when name is unset?

    :- only provides a default for this expansion. := would also assign it, and :? would abort.
  2. 2. With f=archive.tar.gz, what does ${f%%.*} give?

    %% removes the longest match of .* from the end, which is everything from the first dot: .tar.gz. That leaves archive. ${f%.*} (single %) would give archive.tar.
  3. 3. How do you get the file name part of p=/a/b/c.txt without calling basename?

    ##/ removes the longest prefix ending in a slash, leaving c.txt. ${p%/} would give the directory /a/b.
  4. 4. Which expansion upper-cases the whole value of word?

    ^^ upper-cases every character, ^ only the first one, and ,, lower-cases. These forms need Bash 4 or newer.
  5. 5. Why is rm -rf -- "${dir:?}"/* safer than rm -rf $dir/*?

    ${dir:?} aborts with an error when dir is empty, and the quotes prevent word splitting of the value. The unguarded version would delete from the root directory.

Summary

  • ${v:-d} default, ${v:=d} assign default, ${v:?msg} require, ${v:+w} use only when set. Without the colon only "unset" counts.
  • ${#v} length; ${v:off:len} substring (negative offsets need a space or parentheses).
  • #/## remove the shortest/longest prefix, %/%% the shortest/longest suffix. They use glob patterns.
  • ${v/p/r} replaces the first match, ${v//p/r} all matches; /# and /% anchor at the start or end.
  • ^, ^^, ,, ,, change case (Bash 4+).
  • Trimming uses shopt -s extglob with +([[:space:]]); building strings uses +=.
  • ${!name} is indirect expansion; ${var@Q} quotes a value for reuse.
  • These forms are pure Bash, so they avoid starting basename, sed, tr and friends.

Next: Arithmetic and Numbers