Skip to content
Tutorials Center
My learning

Topics

Lesson 69 of 95 ยท Projects

Project: A Todo Manager in Bash

Build a todo manager in Bash with git-style subcommands, a tab-separated file store, getopts per command, optional colours, atomic updates and a test script.

Tools like git, docker and apt share a shape: one program, many subcommands (git commit, docker run), each with its own options. In this project you build a small todo manager in that style. todo add -p high Fix the tap, todo list, todo done 3: tasks live in a plain text file, the output can be coloured when you are looking at it and plain when a pipe is reading it, and a test script proves that the whole thing works.

The project pulls together functions, option parsing with getopts, case statements, awk, sort and cut, error handling and the design habits from Designing Maintainable Scripts. Like the other projects, we build it in stages and every stage is a complete script you can run.

The plan

TEXT
  todo [--color=MODE] COMMAND [ARGS]
        |
        +-- add [-p PRIORITY] TEXT...   append a line to the store
        +-- list [-a] [-p PRIORITY]     filter, sort by priority, format
        +-- done ID... / undo ID...     change the state of a line
        +-- rm ID... / clear            delete lines
        +-- help

  store: $TODO_FILE (default ~/.todo.tsv), one task per line
         ID <TAB> STATE <TAB> PRIORITY <TAB> TEXT

Design decisions:

  • A plain text store. One task per line, fields separated by a tab. You can read it with cat, search it with grep, back it up with cp, and fix it by hand. No database needed.
  • The location is configuration. TODO_FILE is an environment variable with a default. Tests point it at a temporary file, so they never touch your real list.
  • Updates are atomic. To change a line the script writes a complete new file next to the old one and renames it over the old one. A crash in the middle leaves the old list intact.
  • Output depends on who is reading. Colour appears only when standard output is a terminal and NO_COLOR is unset, or when asked for explicitly.

Step 1: the file format and safe updates

Why tab as the separator? Task text may contain spaces, commas and colons, but a tab is almost never typed by accident, and the script can flatten the rare one. cut, sort and awk all understand a tab as a field separator, and cat -A shows tabs as ^I, which makes the format easy to inspect.

New tasks need an ID. The simplest scheme is "highest ID plus one", which awk computes in one pass. Changing or deleting a task means rewriting the file. The function below runs awk over the store, writes to a mktemp file in the same directory, and renames it with mv. A rename inside one filesystem is atomic, so readers see either the complete old file or the complete new one, never a mixture.

Bash Step 1: the store, next ID and an atomic rewrite

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Note the OFS='\t' in rewrite. When awk assigns to a field ($2 = "done") it rebuilds the line using the output field separator, which defaults to a space. Without OFS set to a tab, the first update would silently corrupt the whole file format. Also notice that the task text Fix: the tap, today survives untouched, colon and comma included: the tab is the only separator that matters.

The weak spot of "highest ID plus one" is that deleting the newest task makes its ID available again. For a personal todo list that is acceptable; if you ever need stable IDs, keep the last-used number in a separate counter file.

Step 2: subcommands with case and getopts

A subcommand program has two levels of parsing. The top level looks at the first word and picks a function. Each subcommand function then parses its own options with getopts and treats whatever is left as arguments. Two details trip people up:

  1. getopts keeps its position in the global variable OPTIND. Every function that parses options needs local OPTIND=1, otherwise the second call starts where the first one stopped.
  2. The top level must shift the subcommand name away before calling the function, so that the function sees only its own arguments.
Bash Step 2: a first version with add and list

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The text of a task is "$*": the remaining words joined by single spaces. That is why you do not have to quote the task on the command line. todo add Buy milk and todo add "Buy milk" store the same thing. (Quoting is still needed for characters the shell treats specially, such as ; or &; see Quoting and Expansion.)

This version works, but it trusts its input. Priorities are not validated, an empty add creates an empty task, and list has no sorting. The next steps fix that, add the other commands and make the output pleasant.

Step 3: colours that behave

Colours are made with ANSI escape sequences: the byte ESC followed by [31m switches the text to red, [0m resets. Terminals interpret them; a file or a pipe just receives the raw bytes, which look like garbage in a log. The convention among command-line tools:

  • Colour when standard output is a terminal ([[ -t 1 ]]).
  • Never colour when the environment variable NO_COLOR is set to anything (see the community convention at no-color.org).
  • Let the user override with a flag such as --color=always or --color=never.

The sandbox is not a terminal, so auto mode prints plain text here, which is exactly right. cat -v makes the invisible ESC character visible as ^[.

Bash Step 3: auto, always and never

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Notice || return 0 in the auto branch. In a function, a bare return reports the status of the last command, and a failed test is status 1. Under set -e that would end the whole script just because stdout is not a terminal. This is a real bug that the first draft of this project had.

Step 4: the complete tool

Here is the finished script. It adds validation, done, undo, rm, clear, priority sorting, a help text and the colour handling. Read the cmd_list function carefully: it is a small pipeline that decorates each line with a sort key, sorts, undecorates and formats.

TEXT
 store --awk--> "rank TAB line" --sort -k1,1n -k2,2n--> cut -f2- --awk--> aligned, coloured
         filter, add rank   high=1 normal=2 low=3     drop rank      printf formatting

sort -s keeps the original order for equal keys (a stable sort), and the second key makes ties break by ID, so tasks of the same priority appear in the order you added them.

After the script, the demo runs a short session, and then a test script. The tests use a temporary store, a todo wrapper that forces plain output, and a tiny expect function. The next project, A Mini Test Framework, turns that idea into a reusable tool.

Bash Step 4: todo.sh, a session and its tests

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

How the complete script works

Walk through todo --color=always add -p high Fix the tap and todo list:

  1. main first consumes global options: arguments that start with -- before the subcommand. --color=always sets COLOR. Only then does it call setup_colors, because the colour variables must exist before any output is produced.
  2. local command=${1:-help} picks the subcommand and defaults to help when none was given. (( $# == 0 )) || shift drops the name only when there is one; a plain shift with no arguments would fail under set -e.
  3. A nested case runs ensure_store (create the directory and an empty file) for every command that touches data, then dispatches. done and undo share cmd_set_state with a different target state, and everything else calls cmd_<name> directly through "cmd_$command". The dispatch only happens after the outer case has accepted the command name, so user input never builds an arbitrary function name.
  4. cmd_add parses -p with its own getopts loop, validates the priority with a nested case, flattens tabs (a tab inside the text would add a fake field to the line), rejects empty text, computes the ID and appends one line with printf.
  5. done, undo and rm take several IDs. They validate every ID first and only then change anything, so todo rm 3 99 fails without deleting task 3. Doing all the checks before the first side effect is a general rule for scripts that change data.
  6. cmd_list is the pipeline from the diagram: filter, rank, sort, strip, format.

Look at how the errors in the earlier output were reported: every message goes to standard error, prefixed with the program name, and the exit status is 1. An unknown command additionally prints the usage text to standard error. A script that reports failures this way can be used inside other scripts (todo done 3 || notify), and its normal output stays clean for pipes.

Ideas for extending it

  • Due dates. Add a fifth field in YYYY-MM-DD form (that format sorts correctly as text), a -d option for add, and a list --overdue filter that compares against date +%F. For tests, inject "today" the way the backup project injects the time.
  • A edit ID TEXT... command. One more rewrite with $4 = text.
  • search PATTERN. grep -i -- "$pattern" on the text column.
  • A lock so two terminals cannot rewrite the file at the same moment (see flock and mkdir locks in Advanced Patterns).
  • Tab completion, written as a small function registered with complete -F. It works in an interactive shell only, so it is read-only in this course.
  • Reading from a pipe: todo add with no arguments could read one task per line from standard input when [[ ! -t 0 ]] (see Reading Input).

A script with this many features is near the point where Designing Maintainable Scripts suggests asking whether Bash is still the right tool. If you want due-date arithmetic, a real database or JSON output, a small Python or Go program will be easier to keep correct. Bash is at its best as the glue and for tools of this size.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does each subcommand function declare local OPTIND=1 before calling getopts?

    OPTIND is a global that tracks how far getopts has parsed. A fresh parse in another function (or a second parse in the same shell) must start again at 1.
  2. 2. In the script's rewrite function, why is the new file written with mktemp in the same directory and then renamed with mv?

    The complete new content exists before the old file is replaced. The temporary file must be on the same filesystem as the target, otherwise mv turns into a copy and the atomicity is lost.
  3. 3. What goes wrong in an awk update like $1 == id { $2 = "done" } { print } if OFS is not set to a tab?

    When a field is modified awk reassembles $0 using OFS. The default OFS is a single space, so one update would turn the whole line into a space-separated one.
  4. 4. Why does todo rm 3 99 leave task 3 in place?

    cmd_rm loops over every ID with require_id first. A failure there stops the script before any rewrite happens, so the operation is all-or-nothing as far as validation goes.
  5. 5. When does --color=auto print colour codes?

    [[ -t 1 ]] tests whether file descriptor 1 is a terminal, so pipes and redirected files get plain text. A non-empty NO_COLOR also disables colour.

Summary

  • Build subcommand tools with one case on the first word, one function per command, and getopts with local OPTIND=1 inside each function.
  • A tab-separated text file is a perfectly good store: awk, sort, cut and cat -A all work with it. Set OFS when awk rewrites fields.
  • Update files atomically: write a mktemp file in the same directory, then mv it over the original.
  • Validate all arguments before the first side effect, send errors to stderr, and exit non-zero.
  • Colour only when stdout is a terminal and NO_COLOR is unset; offer --color=always|never. A bare return after a failed test ends a set -e script, so write return 0.
  • Make the file location configurable (TODO_FILE) so tests run against a temporary store.

Next: Project: A Mini Test Framework