Skip to content
Tutorials Center
My learning

Topics

Lesson 34 of 38 ยท Real-world Tasks

Bulk Edits and Refactoring Source Code with sed and AWK

Refactor many files safely: rename functions and variables, update imports and versions, add file headers, report TODOs, rename files in bulk and preview every change first.

Editors can rename a symbol in one project, but sometimes you need the command line: a rename across hundreds of files, a version bump in six config files, a license header on every source file, a report of every TODO. These are the jobs where sed's s command and a file loop shine, and where a careless command can damage a whole code base.

The rule of this lesson: preview, then apply, then review. With version control (Git) you have a safety net, because git diff shows every change and git checkout -- . undoes them. Without it, keep backups.

The safe workflow

  1. Find the files: grep -rl 'oldName' src/.
  2. Preview the edit without changing anything: run sed without -i (or diff its output against the original).
  3. Apply: sed -i.bak ... on exactly those files.
  4. Review: git diff, run the tests, delete the .bak files.
flowchart LR
  A["grep -rl<br/>find files"] --> B["sed (no -i)<br/>preview"]
  B --> C["sed -i.bak<br/>apply"]
  C --> D["git diff / tests<br/>review"]

Never skip the preview stage on a large change.

Renaming an identifier

A naive s/count/total/g also changes counter, account and discount. Use word boundaries so only the whole word matches. GNU sed supports \b and \< / \>; -E with \b is the portable-enough choice on modern systems.

Bash Rename a function, whole words only

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

xargs hands the file list to sed -i.bak. If a file name might contain spaces, use find ... -print0 | xargs -0 instead.

Updating versions and settings

For one value in many files, anchor the pattern on surrounding text so you do not touch unrelated numbers.

Bash Bump a version everywhere

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Escape dots in version numbers (1\.4\.2): an unescaped . matches any character, so 1.4.2 would also match 1x4y2.

Adding and removing headers, trailing spaces and tabs

Bash Normalise source files

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The grep -q ... || guard makes the operation idempotent: running the script twice does not add the header twice. Idempotent scripts are much safer to re-run after an interruption.

Reports with AWK: TODOs, long lines, function lists

Not every refactoring is an edit. Sometimes you need an inventory first.

Bash TODO report and function list

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The "file:line: text" layout is the same one compilers and grep -n use, so editors can jump to those locations.

Renaming files in bulk

Use AWK or sed to generate the mv commands, read them, and only then run them. That makes a dry run free.

Bash Generate, inspect, then execute

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

For a plain extension change, shell parameter expansion is simpler: for f in *.txt; do mv "$f" "${f%.txt}.md"; done. Be careful with names that already exist: mv -n refuses to overwrite.

What sed cannot do for you

Text tools do not understand code. They cannot tell a variable named count from a string containing the word, a comment from real code, or two scopes with the same name. For anything beyond simple, distinctive names, use the language's refactoring tooling (an IDE, gopls rename, jscodeshift, rope...) or add the review step above. Treat sed as the fast, blunt tool and verify the result with tests.

Common mistakes

  • Forgetting \b, so s/id/user_id/g rewrites valid, identity and width.
  • Unescaped dots and slashes in the pattern (use another delimiter for paths: s|/old/|/new/|).
  • Running sed -i on binary files or on the .git directory. Restrict with --include or find -name '*.js'.
  • No backups and no version control.
  • Replacing in generated files or dependencies (node_modules, vendor).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why use \bgetUser\b instead of getUser in the pattern?

    \b matches a word boundary, so only the complete word is replaced.
  2. 2. What does grep -rl pattern . | xargs sed -i.bak 's/a/b/' do?

    grep -rl lists matching file names, xargs passes them to sed, and -i.bak edits each in place keeping a backup.
  3. 3. Why should a header-adding script be idempotent?

    Guarding with grep -q makes repeated runs harmless, which helps after interruptions.
  4. 4. Why generate mv commands with AWK before running them?

    Printing the commands lets you check them, then pipe the same output to sh to execute.

Summary

  • Preview, apply with a backup, then review with git diff and tests.
  • Use word boundaries and escape dots to avoid unintended matches.
  • grep -rl | xargs sed -i.bak edits just the files that contain a pattern.
  • Make scripts idempotent with a grep -q ... || guard.
  • AWK reports (TODOs, function lists, long lines) and generates mv commands for dry runs.
  • Blunt text tools do not understand syntax: verify with tests.

Next: Project: A Log Dashboard in AWK