Redirection and Pipelines

Every process starts with standard input, standard output, and standard error. Redirections reconnect those streams to files or other sources, while a pipeline connects one command's output to the next command's input.

This lesson develops Redirection and Pipelines from its underlying shell behavior through practical, runnable examples. The goal is not merely to memorize punctuation, but to understand what Bash expands, what it executes, and how to keep data boundaries intact.

How Redirection and Pipelines Works

Redirections are applied by the shell before a command runs. File descriptor 1 is standard output and 2 is standard error. In a pipeline, commands normally run concurrently, and pipefail is needed when failure of an earlier stage should make the whole pipeline fail.

Bash processes source in stages: it parses the command, performs the relevant expansions, applies redirections, and finally runs a builtin, function, or external program. Quoting affects the expansion stage, while control structures react to exit statuses. Keeping these stages separate in your mental model makes debugging much faster.

Syntax

command >output.txt 2>errors.txt
producer | filter | consumer
command >>append.txt
command <<<"input"

Whitespace and quoting are part of the language. Do not copy a compact example and then remove quotes or spaces casually: those characters often determine argument boundaries or separate reserved words. Use bash -n script.sh to check syntax without executing the script.

Worked Examples

Example 1: Core behavior

result=$(printf '%s\n' orange apple apricot | grep '^a' | sort)
printf 'Matches:\n%s\n' "$result"

Output:

Matches:
apple
apricot

The first example keeps every expansion explicit and quotes values at the point where Bash builds the command’s arguments. Trace it one line at a time: assignments update shell state, expansions produce values, and printf receives a format plus distinct arguments.

Example 2: Practical composition

message='warning: disk usage high'
printf '%s\n' "$message" | { read -r level rest; printf 'Level=%s Message=%s\n' "${level%:}" "$rest"; }

Output:

Level=warning Message=disk usage high

The second example composes the feature with another shell capability. Its output is deterministic, so you can paste the script into a temporary file, run it, and compare every line. If your result differs, enable a trace locally and inspect the expanded arguments rather than guessing.

Execution Step by Step

1. Bash reads enough input to form a complete command or compound command.
2. Parameter, command, and arithmetic expansions are performed where the syntax requests them.
3. Quoting determines whether expanded text stays together as one argument.
4. Redirections are connected before the selected command runs.
5. The command returns an exit status, which a conditional, loop, or following command may use.

This sequence also explains why printing a variable can look correct even when passing it unquoted is unsafe: the destructive splitting happens while Bash constructs arguments, not while the value is stored.

Common Mistakes

• Writing 2>&1 before >file and misunderstanding left-to-right redirection order.
• Assuming a pipeline fails when its first command fails without enabling pipefail.

Another common mistake is testing only the happy path. Shell programs interact with files, permissions, processes, and user-controlled strings. Always try an empty value, a value containing spaces, and a failing command. Those tests expose most quoting and error-handling defects early.

Best Practices

• Keep data and diagnostics on separate streams.
• Use tee when output must be displayed and recorded.
• Enable pipefail in automation.

For production automation, start scripts with a clear interpreter line, document inputs and outputs, keep functions small, and run ShellCheck. Strict settings can help, but explicit validation and purposeful error messages remain necessary.

Practice Exercises

• Recreate the first Redirection and Pipelines example without copying it, then change one input containing a space.
• Add one validation failure that prints a useful message to standard error and returns a non-zero status.
• Wrap the central behavior in a function and call it twice with different arguments.

Summary

• Redirection and Pipelines is easiest to use correctly when you track parsing, expansion, execution, and exit status separately.
• Quoted expansions preserve data boundaries and should be the default.
• Deterministic examples, syntax checks, and failure-path tests turn shell snippets into reliable automation.