Disk Usage (df, du)

Every Linux filesystem has a fixed amount of space, and eventually something fills it up — a runaway log file, an old kernel package, a careless download. The two tools you reach for are df ("disk free"), which reports how full each mounted filesystem is, and du ("disk usage"), which measures how much space a specific file or directory tree actually consumes. They sound like they answer the same question, and people often confuse their output, but understanding how they differ is what lets you actually track down a full disk instead of guessing.

Overview: How Disk Usage Really Works

To understand why df and du can report different numbers for what looks like the same thing, it helps to have a rough picture of how a filesystem stores data. A filesystem divides the underlying block device into fixed-size blocks (commonly 4096 bytes on ext4). Every file is described by an inode, a metadata record holding its owner, permissions, timestamps, size, and the list of blocks on disk that hold its contents. A directory is just a special file that maps names to inode numbers. When you write a 10-byte file, the filesystem still allocates at least one full block to hold it, because a block is the smallest unit it can hand out — which is why the space a file uses on disk is often larger than its apparent size.

df does not walk any directory tree. It asks the kernel for statistics about a mounted filesystem as a whole — total blocks, free blocks, and free inodes — numbers the kernel already tracks for every mounted filesystem. That is why df returns instantly even on a filesystem with millions of files: it is reading a handful of counters, not scanning anything.

du, by contrast, does walk the directory tree. It recursively visits every file and subdirectory under the path you give it, calls stat() on each one to find out how many disk blocks it occupies, and adds them up. This is why du can be slow on a large tree or a network filesystem, and why its number means "the sum of what these specific files use", not "how full the filesystem is".

Those two facts explain the classic mismatch: a filesystem can report itself as nearly full in df while du on every visible directory adds up to far less. The usual cause is a file that has been deleted but is still open by a running process. Linux only actually frees a file’s blocks once both its link count (the number of directory entries pointing to it) and its open-file-handle count reach zero. Once you delete the file, the directory entry disappears, so du never sees it — but if some process still has it open, the kernel keeps the blocks allocated, so df still counts them as used. A log file that was deleted or rotated away while a service was still writing to it is the textbook example.

One more distinction worth knowing: apparent size versus disk usage. A sparse file can report a large logical size (say, a 10 GB virtual disk image) while occupying almost no actual disk blocks, because large stretches of it are unwritten "holes" the filesystem never allocated. du reports real block usage by default; pass --apparent-size to see the logical size instead, which is why a sparse file’s du and ls -l sizes can differ wildly.

Syntax

df — filesystem-level free space

df [OPTIONS] [FILE...]
Option Meaning
-h Human-readable sizes (K, M, G), using powers of 1024
-H Human-readable sizes using powers of 1000 (matches drive-vendor marketing units)
-T Show the filesystem type (ext4, xfs, tmpfs, …) in an extra column
-i Report inode counts and usage instead of block usage
-a Include pseudo-filesystems normally hidden (proc, sysfs, tmpfs, etc.)
--total Add a grand-total row summing all listed filesystems
FILE Limit output to the filesystem containing this file or mount point (default: all mounted filesystems)

du — space used by files and directories

du [OPTIONS] [FILE...]
Option Meaning
-h Human-readable sizes
-s Summarize: print only a total for each argument instead of every subdirectory
-c Add a grand-total line after the individual totals
--max-depth=N Show totals only down to N directory levels deep
-a Show sizes for individual files too, not just directories
-x Stay on one filesystem; don’t descend into other mounted filesystems
--apparent-size Report logical file size instead of actual disk blocks allocated
--exclude=PATTERN Skip files or directories matching a glob pattern

Examples

Example 1: Check overall filesystem usage

df -h

Output:

Filesystem      Size  Used Avail Use% Mounted on
udev            3.9G     0  3.9G   0% /dev
tmpfs           798M  1.5M  797M   1% /run
/dev/sda1        98G   42G   51G  46% /
tmpfs           3.9G     0  3.9G   0% /dev/shm
tmpfs           798M     0  798M   0% /run/user/1000

The root filesystem /dev/sda1 is 98 GB total, 42 GB used, 46% full. The tmpfs and udev lines are RAM-backed pseudo-filesystems (used for /dev, /run, and per-user runtime files) — they matter for memory accounting, not disk space. If you only care about your real disk, you’d focus on the /dev/sda1 line.

Example 2: Find the size of a specific directory

du -sh /var/log

Output:

128M	/var/log

-s collapses the recursive walk into a single summary line, and -h converts the raw block count into a human-readable 128 MB. This tells you the whole /var/log tree — every log file and subdirectory under it — adds up to 128 MB, without listing each file individually.

Example 3: Find which subdirectory is the biggest offender

du -h --max-depth=1 /home/alice | sort -h

Output:

4.0K	/home/alice/.cache
16M	/home/alice/Documents
1.2G	/home/alice/Videos
1.3G	/home/alice/projects

--max-depth=1 shows a total for each immediate subdirectory of /home/alice instead of every file at every depth, and piping through sort -h (which understands human-readable suffixes like M and G) puts the biggest space consumers at the bottom, right where you’re already looking. Here, projects and Videos are clearly where alice’s disk space is going.

How It Works Step by Step

Walking through du -h --max-depth=1 /home/alice | sort -h:

  1. Bash sees the pipe | and creates a kernel pipe — a pair of connected file descriptors, one for writing and one for reading.
  2. It forks two child processes: one execs du with its standard output connected to the pipe’s write end, the other execs sort with its standard input connected to the pipe’s read end. Both run concurrently.
  3. du opens /home/alice and reads its directory entries. For each entry it calls stat() to get the entry’s type and block count; if the entry is itself a directory, du recurses into it first so it can add up the subdirectory’s total before printing a line for it.
  4. Because --max-depth=1 was given, du still walks the entire tree under the hood (it has to, to compute accurate totals) but only prints a line once it reaches depth 1 relative to the starting path, suppressing deeper lines.
  5. Each printed line is written to the pipe as du produces it. sort reads from its end of the pipe, buffering the incoming lines.
  6. Once du exits and closes its end of the pipe, sort sees end-of-file, sorts everything it has buffered (numerically, respecting the K/M/G suffixes because of -h), and writes the sorted result to its own standard output — your terminal.

df‘s internals are simpler: it reads the list of mounted filesystems (from /proc/mounts), then for each one calls the statvfs() system call, which asks the kernel directly for the filesystem’s total block count, free blocks, and free inodes. No directory is ever opened or read — the kernel already maintains these numbers as part of managing the filesystem.

Common Mistakes

Mistake 1: Running du without -s on a large tree

Without -s, du prints a line for every single directory it visits, recursively. On a home directory with thousands of files, that’s thousands of lines scrolling past before you can read any of them.

du /home/alice

Fix: summarize with -s, or limit the depth with --max-depth as in Example 3.

du -sh /home/alice

Mistake 2: Assuming du always explains what df reports as used

If df -h / shows a filesystem is nearly full but du -sh on every top-level directory only adds up to a fraction of that, don’t assume you’ve made an arithmetic error — suspect a deleted file that a running process is still holding open. Find it with lsof:

sudo lsof +L1

Output:

COMMAND   PID USER   FD   TYPE DEVICE     SIZE/OFF NLINK   NODE NAME
rsyslogd  742 root    5w   REG    8,1   2147483648      0 131081 /var/log/app.log (deleted)

+L1 tells lsof to list open files whose link count has dropped to 1 or below — effectively, files that have been unlinked (deleted) but are still open. Here rsyslogd is still writing to a 2 GB log file that was already deleted; du can never find it because it has no directory entry, but the 2 GB is still allocated and counted by df until the process closes the file (or is restarted).

Mistake 3: Forgetting -h and misreading the numbers

Without -h, du prints sizes in 1 KB blocks by default, which is easy to misread as bytes and wildly misjudge:

du -s /var/log

That command prints 131072 /var/log — 131072 KB, which is the same 128 MB shown with -h in Example 2, but easy to mistake for 131072 bytes (128 KB) if you don’t remember the default unit. Always add -h when reading output yourself; only omit it when piping the raw number into a script that needs to do exact arithmetic.

Mistake 4: Unquoted variables in a script wrapping du

A script that expands a path variable without quotes will word-split on spaces and glob on wildcards, silently breaking on any directory name that isn’t a single simple word.

#!/usr/bin/env bash
dir=$1
du -sh $dir

If someone runs this against "Client Files", $dir expands unquoted into two separate arguments, Client and Files, and du reports on two nonexistent paths instead of the one directory you meant. Quote the expansion, and add set -euo pipefail so a missing argument or a failed du call doesn’t get silently ignored:

#!/usr/bin/env bash
set -euo pipefail
dir="$1"
du -sh "$dir"

Best Practices

  • Use -h whenever you’re reading output yourself; keep raw block counts only when a script needs exact numbers for math.
  • Use du -h --max-depth=1 /path | sort -h (or du -sh /path/*) to quickly find the biggest subdirectory instead of scrolling through an unsummarized recursive dump.
  • Check inodes with df -i, not just bytes — a filesystem can report "No space left on device" while df -h still shows free bytes, if it has run out of inodes (common with directories full of tiny files, like mail spools or session caches).
  • When df and du disagree, suspect a deleted-but-open file and check with lsof +L1 before hunting through directories that no longer exist.
  • Use du -x when scanning a directory tree that might have other filesystems or network shares mounted underneath it, so you don’t wander onto (and wait on) unrelated storage.
  • Automate a disk-usage check (for example, a cron job that alerts when df -h / crosses 90%) instead of discovering a full disk when a service crashes.
  • Exclude noisy paths you don’t care about with --exclude when profiling a large tree, such as .git or node_modules, to get a faster and more relevant picture.
  • Always quote path variables ("$dir", "$1") in any script that wraps du or df, so directory names containing spaces don’t get split into multiple arguments.

Practice Exercises

  1. Run df -h on your own machine and identify which mounted filesystem is closest to full. Then use du -h --max-depth=1 on likely candidates (/var, /home, your own home directory) to find what’s actually consuming the most space there.
  2. Create a test directory and a 50 MB file inside it with dd if=/dev/zero of=~/disktest/bigfile bs=1M count=50, then compare the output of du -sh ~/disktest, du -sh --apparent-size ~/disktest, and ls -lh ~/disktest/bigfile. Explain any differences you observe.
  3. Write a Bash script disk_check.sh that takes exactly one argument (a directory path), prints its total size with du -sh, and exits with a clear error message and non-zero exit status if the argument is missing or the path isn’t a directory. Make sure every expansion of the argument is quoted.

Summary

  • df reports free and used space for entire mounted filesystems by reading kernel-tracked counters via statvfs(); it’s fast because it never scans files.
  • du reports space consumed by specific files and directories by recursively walking the tree and calling stat() on each entry; it can be slow on very large trees.
  • df and du can legitimately disagree — most often because a deleted file is still held open by a running process, which lsof +L1 can reveal.
  • Use -h for human-readable output when reading it yourself, and -s / --max-depth to control how much du prints.
  • Check inodes with df -i, not just bytes — a filesystem can run out of inodes while still showing free space.
  • Quote variables and paths in any script that wraps du or df so directory names containing spaces or wildcards don’t break the command.