Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 95 ยท Working with Text

Viewing and Slicing Text

Learn the Linux tools that show and slice text: cat, tac, head, tail, nl, cut, paste, tr, fold and expand, with CSV and log examples you can run.

On Linux almost everything interesting is text: logs, configuration files, source code, CSV exports, command output. A small family of tools reads text and cuts it into pieces, and each one does a single job well. Learn them and you can answer questions like "what are the last five lines of this log?", "which column holds the email address?" or "how do I turn commas into tabs?" in seconds, without opening an editor.

The tools in this lesson share one design: they read lines from files or from standard input and write lines to standard output. That is why they combine so well in pipelines. Here we use them one at a time first. The next lessons add grep, sed and awk for searching and transforming.

cat, tac and nl: showing a whole file

cat (concatenate) prints files one after another. With a single file it just shows the file; with several it joins them. Handy options:

  • -n numbers every line, -b numbers only non-blank lines.
  • -A makes invisible characters visible: $ marks the end of each line, ^I is a tab. It is the fastest way to find trailing spaces or Windows line endings (^M).
  • -s squeezes runs of blank lines into one.

tac is cat backwards: it prints the last line first. nl is a more capable line numberer (it can skip blank lines, restart numbering and pad with zeros).

Bash cat, tac, nl and invisible characters

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

In cat -A odd.txt you can see that the first line ends with two spaces, a carriage return (^M) and then the end of line: a Windows-style line ending that can break scripts with errors like bad interpreter or make comparisons fail mysteriously. If a file looks right but behaves wrong, run cat -A on it.

A famous beginner habit is cat file | command. It works, but most commands can read a file directly (grep pattern file) or through redirection (command < file), which saves a process. People call the pattern a "useless use of cat". It is harmless in a quick experiment, and you will see it in tutorials, but prefer the direct form in scripts.

head and tail: the beginning and the end

head prints the first 10 lines, tail the last 10. Both take -n N (or -N) to change the count and -c N to count bytes instead. The forms with a sign are powerful:

Command Meaning
head -n 3 file first 3 lines
head -n -2 file everything except the last 2 lines (GNU)
tail -n 3 file last 3 lines
tail -n +4 file everything from line 4 onward (skip the first 3)
tail -c 5 file last 5 bytes

tail -n +2 is the standard way to skip a CSV header. Combine both and you can print any range: lines 3 to 5 are head -n 5 file | tail -n 3.

Bash head and tail ranges

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Here seq 1 12 generates numbers and sed (covered later) prefixes them with the word "line"; we only need a test file. When head or tail get more than one file they print a ==> name <== banner before each; -q suppresses it.

Following a growing file with tail -f

tail -f logfile keeps the file open and prints new lines as they are appended, the classic way to watch a log live. tail -F also survives log rotation (the file being renamed and recreated). -f never ends by itself, so it needs Ctrl+C and does not make sense in our run-to-completion editor; the read-only example below shows the idea.

CONSOLE
$ tail -f /var/log/syslog
Oct 22 09:14:02 host sshd[812]: Accepted publickey for ada
Oct 22 09:14:09 host cron[455]: (root) CMD (run-parts /etc/cron.hourly)
^C

A useful trick for pipelines: command | head -n 1 stops after one line. The producer receives a signal (SIGPIPE) when head exits, which is why yes | head -3 ends instead of running forever. We explain this in Pipes and Pipelines.

less: reading large files (read-only)

less is the pager you use to read long output without flooding the terminal: space or f goes forward a page, b back, /word searches, n repeats the search, g and G jump to the start and end, q quits. man pages open in less. Because it is interactive, it does not run in our editor, so treat the following as a reference:

CONSOLE
$ less /var/log/syslog          # open a file
$ ls -l /etc | less             # page any command's output
$ less -S wide.csv              # do not wrap long lines; use arrow keys to scroll sideways
$ less +F app.log               # follow mode, like tail -f (Ctrl+C to stop, F to resume)

Compared with a text editor, less never loads the whole file, so it opens a multi-gigabyte log instantly.

cut: choosing columns

cut extracts parts of each line. You choose either fields with a delimiter (-d and -f), characters (-c) or bytes (-b). Ranges work as in -f1,3, -f2-4, -f3- (from 3 to the end).

Bash cut on delimited data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Know the limits of cut:

  • The delimiter is one character. Data separated by runs of spaces (like the output of ls -l or ps) cannot be cut reliably, because each space is a separator and empty fields appear. For that, use awk (which splits on any run of whitespace).
  • It cannot reorder: cut -f3,1 still prints field 1 first. Use awk or paste for that.
  • Real CSV can contain commas inside quotes ("Smith, John"). cut -d, cannot understand quotes. For simple generated data it is perfect; for arbitrary CSV use a CSV-aware tool.

paste: gluing lines side by side

paste is the horizontal counterpart of cat: it joins the n-th line of each file with a tab (or the delimiter you choose with -d). With -s it puts all lines of one file on a single line.

Bash paste: columns and joined lines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The paste - - - trick (each - is another read from the same stdin) reshapes a long list into rows of three. paste -sd+ ages.txt prints 36+28+45, which you can hand to $(( )) or bc to add numbers.

tr: translating characters

tr works on characters, not lines. It reads standard input only (you cannot give it a file argument; use <). Common jobs:

  • Translate sets: tr 'a-z' 'A-Z' upper-cases text, tr ',' '\t' turns commas into tabs.
  • Delete with -d: tr -d '\r' removes carriage returns (fixing Windows files), tr -d '[:digit:]' removes digits.
  • Squeeze repeats with -s: tr -s ' ' collapses runs of spaces into one.
  • Complement with -c: tr -cd '[:alnum:]\n' keeps only letters, digits and newlines.
Bash tr in action

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

tr only maps single characters, so it cannot replace a word with another word; that is a job for sed. When the second set is shorter than the first, GNU tr pads it with its last character, so tr 'abc' 'x' turns all of a, b and c into x. Remember that tr has no knowledge of multi-byte UTF-8 characters in some versions, so use it for ASCII data.

fold, fmt, expand and more

A few smaller helpers round out the set:

Tool Purpose
fold -w 20 hard-wrap lines at 20 characters (-s breaks at spaces)
fmt -w 40 re-flow paragraphs to a width, joining short lines
expand / unexpand convert tabs to spaces / spaces to tabs (-t 4 sets the tab stop)
rev reverse the characters of each line
column -t align columns into a table (read-only here, see below)
Bash fold, fmt, expand and rev

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

column -t aligns whitespace-separated text into neat columns (column -t -s, for CSV), and it is widely used for pretty-printing. It comes from the bsdextrautils/util-linux package and is not installed in our sandbox, so we align with printf above. On a normal Linux machine:

CONSOLE
$ column -t -s, people.csv
id  name   city      age
1   Ada    London    36
2   Linus  Helsinki  28

Reading logs and CSV-ish data

Real files combine these tools. A typical question: "How many requests did each user make?" Starting from a log, you cut out the field and count (the counting tools come in sort, uniq, cut, tr and wc).

Bash A small log and CSV workflow

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The rev | cut | rev line is a famous idiom for "the last field": reverse each line, take the first field, reverse it back. (With awk it is simply awk '{print $NF}'.) The <( ... ) in the last command is process substitution, which hands the output of a command to another command as if it were a file; it is explained in xargs, Command Substitution and Process Substitution.

Choosing the right tool

TEXT
Show everything          cat, tac, less (interactive)
Show the start / end     head, tail, tail -f
Pick columns             cut (one-character delimiter), awk (flexible)
Join side by side        paste
Change characters        tr
Wrap or reflow           fold, fmt
Number lines             cat -n, nl

A last pitfall: all of these split on the newline character. A file whose last line lacks a trailing newline is still read, but cat a b then glues the last line of a to the first line of b. That is a good reason to end every text file with a newline.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which command prints a CSV file without its header line?

    tail -n +2 starts output at line 2, skipping the first line. tail -n 2 prints the last two lines, and head -n -1 drops the last line instead.
  2. 2. You run cat -A report.txt and see ^M$ at the end of every line. What does it mean?

    ^M is a carriage return shown visibly and $ marks the end of the line, so the file has CRLF endings. tr -d '\r' removes them.
  3. 3. What does cut -d, -f3,1 print for the line a,b,c?

    cut always outputs the selected fields in file order, so it prints a,c. Reordering columns needs awk.
  4. 4. Why does tr 'a-z' 'A-Z' file.txt fail?

    tr takes no file arguments. Feed it with redirection or a pipe.
  5. 5. What does printf '1\n2\n3\n4\n' | paste - - produce?

    Each dash reads the next line from the same standard input, so paste takes two lines per output row, separated by a tab.

Summary

  • cat shows or joins files (-n, -A, -s), tac reverses line order, nl numbers lines with more control.
  • head -n N, tail -n N, tail -n +N and head -n -N select line ranges; tail -f follows a growing file; less pages through big output interactively.
  • cut -d -f picks delimited fields and cut -c picks characters; it needs a one-character delimiter and cannot reorder.
  • paste joins files side by side or, with -s, joins lines into one; paste - - - reshapes a list.
  • tr maps, deletes (-d), squeezes (-s) and complements (-c) single characters, reading standard input only.
  • fold, fmt, expand and rev reshape text; cat -A reveals tabs and Windows line endings.

Next: grep and Searching Text