Combining Redirection and Pipes

Pipes and redirection are two separate features of the Bash shell, but real command lines almost always use them together: you pipe a command’s output into a filter, then redirect the final result into a file, or you send a script’s error stream into the same pipe that’s logging its normal output. Once you understand each mechanism on its own, the next step is understanding the order the shell applies them in, because that order decides whether your error messages end up in your log file or vanish into thin air. This lesson walks through exactly how the two combine, with runnable examples and the ordering rules that trip up almost everyone at least once.

Overview: How Pipes and Redirection Combine

A pipe (|) and a redirection (>, >>, <, 2>&1) do different jobs at the kernel level, and seeing that difference explains everything about how they interact.

A pipe asks the kernel to create an anonymous pipe: a small, kernel-buffered queue with two file descriptor ends, one for reading and one for writing. The shell forks a child process for each command in the pipeline and uses the dup2() system call to make the left command’s standard output (file descriptor 1) point at the pipe’s write end, and the right command’s standard input (file descriptor 0) point at the pipe’s read end. From then on, anything the left command writes to fd 1 flows through the kernel buffer straight into the right command’s fd 0 — no temporary file is ever created.

A redirection is also implemented with dup2(), but it changes where one specific file descriptor points, usually by opening a file and pointing fd 0, 1, or 2 at it instead of the terminal. > opens a file for writing and truncates it to zero length first; >> opens it for appending instead; < opens a file for reading and points fd 0 at it; and 2>&1 makes fd 2 (standard error) point at whatever fd 1 currently points at — it does not open a file at all, it just copies a file descriptor.

When a command line mixes both, the shell wires up the pipeline connections first — connecting each command’s fd 1 to the next command’s fd 0 — and then, for each individual command, applies that command’s own redirections in the order they’re written, left to right, right before it executes the program. This is the rule that matters most: redirections attached to one command in a pipeline only affect that command, and their left-to-right order determines what 2>&1 actually copies at the moment it runs. Get the order backwards and stderr quietly ends up in the wrong place — usually the terminal instead of your log file.

By default, a pipe only carries standard output. Standard error is untouched and still goes straight to the terminal, which is why the 2>&1 idiom shows up constantly whenever a script’s errors need to travel through the same pipe as its normal output — for example into tee, so the combined stream is both shown on screen and saved to a file.

Syntax

There’s no special combined operator; you simply chain the pipe and redirection tokens you already know on one command line. The general shape looks like this:

command1 < input_file | command2 2>&1 | command3 > output_file
Token Meaning
| Connects the left command’s stdout to the right command’s stdin through a kernel pipe.
> Redirects stdout to a file, truncating (overwriting) it first.
>> Redirects stdout to a file, appending to the end instead of truncating.
< Redirects a file’s contents onto stdin.
2>&1 Duplicates stderr onto whatever stdout currently points at. Order relative to > and | matters.
&> / &>> Bash shorthand that redirects both stdout and stderr to a file (truncate / append).
tee FILE Not an operator but a command: reads stdin, writes an identical copy to both stdout and FILE.
tee -a FILE Same as tee, but appends to FILE instead of truncating it.

Examples

Example 1: Filter, count, and save in one pipeline

grep "ERROR" /var/log/app.log | sort | uniq -c | sort -rn > /var/log/error-summary.txt
cat /var/log/error-summary.txt

Output:

     42 ERROR: connection timeout
     17 ERROR: invalid token
      3 ERROR: disk full

The pipeline runs entirely in memory, process to process: grep filters the log for lines containing ERROR, sort groups identical lines together (required because uniq only collapses adjacent duplicates), uniq -c prefixes each unique line with a count, and a second sort -rn orders the results by that count, largest first. Only the very last stage’s output is redirected with > into error-summary.txt; everything upstream of it travels through pipes, not files.

Example 2: Sending both stdout and stderr through a pipe

./backup.sh 2>&1 | tee /var/log/backup.log

Output:

Starting backup of /home/alice...
tar: /home/alice/.cache: Cannot access: Permission denied
Backup finished with warnings.

Because 2>&1 is written before the pipe symbol, it duplicates stderr onto fd 1 while fd 1 is still pointing at the pipe’s write end — so both the normal progress messages and the permission warning travel into the same stream. tee receives that merged stream on its stdin and writes an identical copy to the terminal and to backup.log, so you see everything live and still have a record afterward.

Example 3: Redirecting input into a pipeline and the result into a file

sort < unsorted.txt | uniq -c | sort -rn > report.txt
cat report.txt

Output:

      3 banana
      2 apple
      1 cherry

Here a redirection supplies the pipeline’s input (< unsorted.txt feeds sort‘s stdin instead of the terminal) and a redirection captures its output (> report.txt catches what the final sort -rn would otherwise print to the terminal). Everything in between is connected by pipes. This is the classic shape of a Bash one-liner: redirect in, filter through several commands, redirect out.

How It Works Step by Step

Take the pipeline from Example 2: ./backup.sh 2>&1 | tee /var/log/backup.log. Here is what Bash actually does, in order:

  1. Bash parses the line and sees one pipe, so it treats this as a two-stage pipeline: ./backup.sh 2>&1 on the left, tee /var/log/backup.log on the right.
  2. Before forking anything, Bash calls pipe(2), which asks the kernel for a new pipe and returns a read-end file descriptor and a write-end file descriptor.
  3. Bash forks a child process for backup.sh. In that child, it duplicates the pipe’s write end onto file descriptor 1, so the script’s standard output now goes into the pipe instead of the terminal.
  4. Still inside that same child, Bash processes 2>&1. Because this comes after fd 1 was reassigned in the previous step, it duplicates fd 2 onto the pipe’s write end as well — both stdout and stderr now point at the same destination.
  5. Bash forks a second child for tee, duplicating the pipe’s read end onto that child’s file descriptor 0, so whatever arrives in the pipe becomes tee‘s standard input.
  6. Both children call execve() to become backup.sh and tee. The parent shell closes its own copies of the pipe descriptors and waits for both children to exit.
  7. tee reads the merged stdout/stderr stream from its stdin and splits it: one copy goes to its own stdout (the terminal), and one copy is written to /var/log/backup.log.
  8. The pipeline’s overall exit status is, by default, the exit status of the last command (tee) — not backup.sh. If you need to detect that an earlier stage failed, enable set -o pipefail, covered in the error-handling lesson.

Common Mistakes

Mistake 1: Putting 2>&1 after the pipe instead of before it

Wrong:

./deploy.sh | tee /var/log/deploy.log 2>&1

Here 2>&1 is attached to tee, not deploy.sh. It only redirects tee‘s own stderr (which is almost never used); deploy.sh‘s stderr still goes straight to the terminal and is never captured in the log or piped anywhere.

Correct:

./deploy.sh 2>&1 | tee /var/log/deploy.log

Mistake 2: Redirecting to a file before piping, then expecting the pipe to still see stdout

Wrong:

./deploy.sh > /var/log/deploy.log 2>&1 | mail -s "deploy output" "ops@example.com"

By the time the shell reaches the pipe, fd 1 for deploy.sh has already been redirected into the file (and fd 2 duplicated onto that same file). Nothing is left pointing at the pipe, so mail receives an empty message.

Correct:

./deploy.sh 2>&1 | tee /var/log/deploy.log | mail -s "deploy output" "ops@example.com"

Mistake 3: Leaving a variable unquoted inside a pipeline

Wrong:

grep $pattern /var/log/app.log | wc -l

If pattern holds something like connection timeout, the unquoted expansion word-splits into two separate arguments, so grep sees a pattern of connection and treats timeout as a second filename to search — not what was intended, and it can fail outright if that file doesn’t exist.

Correct:

grep "$pattern" /var/log/app.log | wc -l

Best Practices

  • Put 2>&1 immediately after the command and before the pipe symbol whenever you want stderr to flow through the same pipe as stdout.
  • Reach for tee (or tee -a to append) when you need to both watch output live and save it, instead of redirecting to a file and then running cat afterward.
  • Turn on set -o pipefail in scripts so a failing command earlier in a pipeline isn’t masked by a later command’s success.
  • Redirect noisy, expected stderr on purpose with 2>/dev/null rather than suppressing all output blindly.
  • Always quote variables and command substitutions inside a pipeline stage; the pipe itself does nothing to protect you from word-splitting or globbing.
  • Use >>, not >, whenever a command’s output should accumulate across multiple runs — a stray > in a logging script silently erases prior history.

Practice Exercises

  • Write a single pipeline that searches every .log file under /var/log for lines containing error (case-insensitive), counts how many matches came from each file, and saves the result sorted from most to fewest into ~/error-report.txt. Hint: combine grep -ril or a loop, grep -ic, and sort.
  • You have a script named sync.sh that prints progress to stdout and occasional warnings to stderr. Write one command line that shows both streams on your terminal live and also appends them to ~/logs/sync.log without erasing previous runs.
  • Given the broken command mycommand > out.txt 2>&1 | grep -i fail, explain in your own words why grep never receives any input, then rewrite it so grep actually filters mycommand‘s combined output while out.txt still keeps a full copy of everything.

Summary

  • Pipes connect one command’s stdout to the next command’s stdin through a kernel pipe; redirection points a specific file descriptor at a file instead of the terminal — they’re independent mechanisms that combine freely on one command line.
  • The shell wires up pipeline connections first, then applies each command’s own redirections left to right before executing it, so the order of tokens changes behavior.
  • A pipe only carries stdout by default; write 2>&1 before the pipe symbol to send stderr through it too.
  • tee splits a stream so you can watch it live and save it to a file at the same time — the standard way to combine a pipeline with logging.
  • set -o pipefail and disciplined quoting are essential once pipelines like these end up inside real scripts.