Skip to content
Tutorials Center
My learning

Topics

Lesson 15 of 95 ยท Working with Files

Disk Usage and File Information

Learn how to measure disk space on Linux with du, df, ls -lh and stat, find the biggest files, count files, and understand sparse files and apparent size.

"Where did my disk space go?" is one of the oldest questions in system administration, and the command line has precise tools to answer it. In this lesson you will measure single files, whole directory trees and entire file systems, find the biggest offenders, count files, and learn why two programs can report different sizes for the very same file.

Before the commands, one idea matters more than any option: a file has two sizes. The apparent size is how many bytes you can read from it. The disk usage is how much storage the file system reserved for it. They are usually close, but they are not equal, and every tool in this lesson picks one or the other. Knowing which one is the difference between a confident answer and a confusing one.

Two sizes for every file

A file system does not hand out space byte by byte. It hands out blocks, typically 4,096 bytes each. A file of 3 bytes still occupies one whole block, and a file of 4,097 bytes occupies two. The classic tools therefore disagree in a predictable way:

TEXT
file contents: 1,500 bytes

apparent size  (ls -l, wc -c, stat %s, du --apparent-size):   1500
disk usage     (du, ls -s, stat %b x %B):                     one 4096-byte block = 4.0K

Let's see both numbers side by side. The example builds files of known sizes (we use head -c N /dev/zero, which writes exactly N zero bytes) and fixes their timestamps so the output is the same on every run.

Bash Apparent size versus disk usage

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

A 1,500-byte file costs 4.0K of disk, while --apparent-size reports 1.5K. The 3-byte note.txt shows 8 blocks of 512 bytes in stat: stat always counts in 512-byte units, so eight of them are one 4 KiB allocation unit. Remember the rule of thumb: ls -l and wc -c tell you what the file contains; du tells you what it costs.

Sizes in this lesson were produced on a Linux file system with 4 KiB blocks. On other file systems (or with other settings) the "cost" numbers can differ, which is exactly why we print them rather than guess. The apparent sizes never change.

ls -l and human-readable sizes

You met ls -l in Navigating. Three options help with sizes:

  • -h prints sizes as K, M, G (powers of 1024). It needs -l or -s to be useful.
  • -S sorts by size, largest first. Add -r to reverse.
  • -s adds a column with the disk usage in blocks (1 KiB units by default in GNU ls).

The total line at the top of ls -l is also disk usage (in 1 KiB units), summed over the listed files; it is not the sum of the byte column.

Many tools accept -h for human-readable output, and GNU tools also know --si, which uses powers of 1000 (kB, MB) like disk vendors do. Mixing the two units is a classic source of "my 1 TB drive is only 931 GB" confusion: 1 TB is 10^12 bytes, which is about 931 GiB.

du: measuring trees

du (disk usage) walks a directory tree and adds up the space used. Its default output lists every sub-directory, which is usually too much, so you will almost always combine it with one of these options:

Option Effect
-s Summary: one total per argument, no sub-directories
-h Human-readable sizes (4.0K, 12M)
-d N / --max-depth=N Show directories down to N levels (-d 1 = immediate children)
-a Also list files, not only directories
-c Print a grand total as the last line
-b Bytes, and implies --apparent-size
--apparent-size Count content bytes instead of blocks
-x Stay on one file system (do not cross mount points)
--exclude=PATTERN Skip names matching a glob

A directory is itself a file with a size, so du counts it too: an empty directory is usually 4,096 bytes of apparent size. Keep that in mind when a total is "a little more" than the sum of the files.

Bash du with -s, -d and -c

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Look at the numbers: logs is 45596, which is 40,000 + 1,500 plus 4,096 for the directory entry itself. The . line is the grand total of everything below it. Because du prints one line per directory, the totals are cumulative: a parent includes all of its children.

Sorting for the biggest items

Raw bytes sort correctly with sort -n. With human-readable sizes, sort with sort -h, which understands K, M and G:

Terminal
du -sh * | sort -h          # smallest to largest, human readable
du -sh * | sort -rh | head  # top ten

This du | sort | head pattern is the classic "what is eating my disk" recipe. We cover sort in depth in sort, uniq, cut, tr and wc. A related warning: du -sh * skips hidden files because * does not match a leading dot (see Wildcards and Globbing). Use du -sh .[!.]* * or du -h -d 1 instead, which includes dotfiles.

df: the file system view

du adds up files by walking them. df (disk free) asks the file system how full it is. They answer different questions, and they can disagree: df also counts metadata, files that were deleted but are still held open by a program, and reserved blocks.

CONSOLE
$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda2        98G   61G   33G  66% /
tmpfs           3.9G  1.2M  3.9G   1% /run
/dev/sda1       511M  6.1M  505M   2% /boot/efi

The output above is typical of a Linux desktop or server and is shown read-only, because the numbers in our sandbox describe a container and differ from machine to machine. The useful df options are:

  • -h human-readable, -T adds the file system type (ext4, xfs, tmpfs, ...).
  • -i shows inodes instead of blocks. A disk can be "full" with plenty of free space if it ran out of inodes, which happens with millions of tiny files. See links and inodes.
  • df . or df /path shows the file system that contains a given path, the easiest way to answer "which disk is this folder on?".

The column layout is stable and scriptable with --output, so you can ask only for what you need:

Bash df headers and file system type

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

-P forces the POSIX layout (one line per file system, 1024-byte blocks), which keeps long device names from wrapping and breaking scripts.

When df says a disk is 100% full but du cannot find the files, suspect a deleted file still held open by a running program (a log that was removed while the service kept writing). The space returns when the program closes the file or restarts. You will learn to find such processes in Processes.

stat: everything about one file

stat prints the file's metadata: size, blocks, inode, links, permissions, owner, and three timestamps. The default output is long; the -c (format) option picks what you want, like printf. Useful format sequences:

Sequence Meaning
%n file name
%s size in bytes (apparent)
%b / %B allocated blocks / size of each block (512 for %B on Linux)
%F file type (regular file, directory, symbolic link)
%a / %A permissions in octal / in rwx form
%h number of hard links
%i inode number
%y last modification time

Allocated bytes are %b times %B. This is how you compute disk usage for one file without du. Because stat is just printing numbers, it is a good building block in scripts, for example size=$(stat -c %s "$file"). (On macOS and BSD, the option is stat -f with different sequences; this is one of the GNU-versus-BSD differences to remember.)

Sparse files: apparent size is not a cost

Some files claim to be large but store almost nothing. A sparse file has "holes": regions that read back as zeros but have no blocks allocated. The truncate command creates one instantly by just setting the length.

Bash A sparse file

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

truncate -s 1M made a file of 1,048,576 bytes that cost nothing; after writing one byte, a single block is allocated and du reports 4.0K. Virtual-machine disk images, database files and some backups are sparse, which is why copying them naively (cp without --sparse=always, or tar without -S) can suddenly use much more space than the original did. The opposite also happens: a file that looks small in ls may be a hole-filled monster when "restored".

Finding what is big

With the pieces above you can answer most "what is using the space" questions.

Bash Largest files, big files, file counts

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Notes on what you saw:

  • find -printf '%s %p\n' prints the size and path of each file without starting a process per file, so it is fast even on large trees. %h is the directory part, handy for grouping. find has far more tests (-size, -mtime, -newer) than this lesson uses.
  • -size +10k means "more than 10 KiB, rounded up to whole units". find rounds file sizes up to the unit before comparing, so -size -1k matches only empty files. Use -size +10000c for exact bytes.
  • The shell total is a small trick: find prints 3000+700+, and the extra 0 finishes the sum inside $(( )) (see Arithmetic). du --files0-from=- totals a list of files safely: names are separated by NUL bytes, so spaces and newlines in names are harmless.
  • wc prints lines, words and bytes; wc -l counts lines, wc -c bytes, wc -m characters, wc -w words. Counting files is just find ... | wc -l, with the caveat that a file name containing a newline counts twice. Use find ... -printf '.' | wc -c when you need an exact count.

Quotas and limits (concepts)

On shared servers, administrators often set quotas: a cap on how much space (blocks) or how many files (inodes) a user or group may own. When you exceed a hard limit, writes fail with Disk quota exceeded. The quota command shows your limits and repquota summarizes everyone's (both need a file system with quotas enabled and are not available in our sandbox). In containers and cloud systems the same effect comes from cgroup limits and per-volume sizes. A different, per-process limit is ulimit -f, the largest file a shell and its children may create; you will see ulimit in kill and signals.

Three habits make disk work safer:

  1. Look before you delete. Run the du | sort | head recipe, then inspect the top items; do not guess.
  2. Stay on one file system with du -x and find -xdev so you do not wander into network mounts or /proc.
  3. Remember the two sizes. When a number surprises you, ask "apparent or disk usage?" first.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A file contains 1,500 bytes. On a file system with 4 KiB blocks, what does plain du -h report for it?

    du reports disk usage, which is rounded up to whole blocks: one 4 KiB block. Use --apparent-size to see 1.5K.
  2. 2. Which command prints the number of bytes in a file's contents, not the space it occupies?

    wc -c counts the bytes it can read. du and stat %b report allocated blocks, and df reports the whole file system.
  3. 3. You create a file with truncate -s 1M disk.img. What does du -h disk.img show?

    truncate only sets the length, creating a sparse file with no allocated blocks. ls -l shows 1048576 bytes, while du shows 0 until data is written.
  4. 4. Why can df report a full disk when du -sh / shows much less used?

    df reports the file system's own accounting. du only adds up the files it can reach by walking directories.
  5. 5. Which pipeline lists the five largest entries directly inside the current directory, including hidden ones?

    du -d 1 lists the immediate children (dotfiles included) and sort -rh orders human-readable sizes from largest. The * glob skips hidden files and the list is unsorted.

Summary

  • Every file has an apparent size (ls -l, wc -c, stat %s) and a disk usage (du, ls -s) that is rounded up to whole blocks.
  • du -sh, -d N, -a, -c, -x and --apparent-size cover most needs; du | sort -h | head finds the biggest items.
  • df reports file-system capacity and can disagree with du; df -i checks inodes, df path finds the disk behind a path.
  • stat -c prints exact metadata for scripts; sizes are bytes, blocks are counted in 512-byte units.
  • Sparse files (truncate -s) are large in ls but small in du; copying them carelessly can expand them.
  • Use find -printf and -size to hunt for large files and find | wc -l to count them.

Next: Viewing and Slicing Text