Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 38 ยท sed Scripting

sed Script Files and Classic Recipes

Organize sed programs in script files with comments and shebangs, pass shell variables safely, and learn a dozen classic recipes: trim, number, strip tags, CRLF and more.

A sed program starts as a one-liner and, if it is useful, grows. At some point you want comments, a name, a place in version control. This lesson shows how to put sed programs in script files, how to pass shell data into them safely, and then collects a set of classic recipes you will use again and again. Each recipe is short, tested, and explained.

Script files

Put commands in a file, one per line, and run it with -f:

TEXT
sed -f cleanup.sed input.txt

Rules for script files:

  • A line starting with # is a comment. If the first two characters of the file are #n followed by a newline, it acts like -n.
  • Commands are separated by newlines (or ;). Labels and a/i/c text end at the newline.
  • Blank lines and leading spaces are ignored.
  • A script can start with #!/bin/sed -f (or #!/usr/bin/sed -f) and be made executable: ./cleanup.sed input.txt. Many systems also need -nf in the shebang for -n.
Bash A commented script file

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Each section is a recipe in itself, and the comments tell the next person (or you in six months) why it exists. For a few commands you can still use -e; for anything that people will read, use a file.

Mixing -f and -e, and several files

You may repeat -f and -e. The fragments are joined in order. That lets you keep a reusable library of scripts and add a project-specific tweak:

TEXT
sed -f common.sed -e 's/staging/production/' app.conf

Passing shell variables safely

A script file cannot see the shell's variables. For one-liners you may switch to double quotes, but then the shell processes $, backticks and backslashes, and the value is pasted into the regex or replacement without escaping. A value containing /, & or \ can break the command or change its meaning.

Bash Variables in sed commands

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The last example escapes \, & and | (the delimiter used) so the value is inserted literally. For anything more complex or for untrusted values, a better approach is to give the value to AWK via -v, which does no regex interpretation of the data (see AWK variables).

Recipes

Every recipe below was tested. Try them one at a time.

Bash Recipe set 1: whitespace

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The last one works like cat -s: on an empty line, N appends the next line; if that is empty too (\n$), D deletes the first of the two and restarts.

Bash Recipe set 2: numbering, ranges, reversing

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Bash Recipe set 3: files, paths, markup

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The last recipe uses the empty regex: inside /BEGIN/,/END/, // means "the last regex tried", which is BEGIN on its first line and END on the last, so //!p skips both marker lines.

Bash Recipe set 4: editing configuration safely

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

#& puts a hash before the whole match: a common way to comment out lines without retyping them.

Writing maintainable scripts

  1. Name your scripts for what they do (normalise-config.sed), not how.
  2. Comment each section, including why a regex looks the way it does.
  3. Test on a small sample file and compare with diff before using -i.
  4. Keep sample input and expected output next to the script so changes can be checked.
  5. Know when to stop: if the script has several nested loops or hold-space juggling, rewrite it in AWK or another language.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a first line #n in a sed script file do?

    #n on the very first line of a script (alone on that line) works like -n.
  2. 2. How do you run the script file fix.sed on data.txt?

    -f reads the script from a file; -e would treat the text fix.sed as a command.
  3. 3. Which command deletes the lines between markers while keeping the markers?

    Inside the range, // is the last regex used, so //!d deletes every line except the two marker lines.
  4. 4. Why is putting $value into a double-quoted sed command risky?

    The value is pasted into the script as text, so special characters change its meaning unless you escape them.

Summary

  • Put real programs in .sed files with comments; run them with -f; #n on line 1 means -n.
  • Combine -f and -e; avoid pasting untrusted shell values into scripts.
  • Memorize the recipes: trim, squeeze blank lines, number lines, strip tags, CRLF conversion, section extraction, comment out lines.
  • Test with diff before -i, keep samples, and switch to AWK when the script gets hard to read.

Next: Your First AWK Program