grep, sed, and awk

Published by Programming Line in Bash Shell Scripting. Last updated .

grep selects matching lines, sed performs stream edits, and awk processes field-oriented records. These tools complement Bash: the shell orchestrates work while each utility performs one focused text transformation.

This lesson develops grep, sed, and awk from its underlying shell behavior through practical, runnable examples. The goal is not merely to memorize punctuation, but to understand what Bash expands, what it executes, and how to keep data boundaries intact.

How grep, sed, and awk Works

grep's success status indicates whether it found a match. sed applies commands to each input line. awk splits records into fields and supports variables, arithmetic, and formatted output. Quoting the program protects $ and other symbols from premature shell expansion.

Bash processes source in stages: it parses the command, performs the relevant expansions, applies redirections, and finally runs a builtin, function, or external program. Quoting affects the expansion stage, while control structures react to exit statuses. Keeping these stages separate in your mental model makes debugging much faster.

Syntax

grep 'pattern' file
sed 's/old/new/g' file
awk -F, '{ print $1 }' file

Whitespace and quoting are part of the language. Do not copy a compact example and then remove quotes or spaces casually: those characters often determine argument boundaries or separate reserved words. Use bash -n script.sh to check syntax without executing the script.

Worked Examples

Example 1: Core behavior

printf '%s\n' INFO ERROR WARNING ERROR | grep -c '^ERROR$' | { read -r count; printf 'Errors: %s\n' "$count"; }

Output:

Errors: 2

The first example keeps every expansion explicit and quotes values at the point where Bash builds the command’s arguments. Trace it one line at a time: assignments update shell state, expansions produce values, and printf receives a format plus distinct arguments.

Example 2: Practical composition

printf '%s\n' 'Ada,80' 'Grace,95' | awk -F, '{ total += $2; count++ } END { printf "Average: %.1f\n", total / count }'

Output:

Average: 87.5

The second example composes the feature with another shell capability. Its output is deterministic, so you can paste the script into a temporary file, run it, and compare every line. If your result differs, enable a trace locally and inspect the expanded arguments rather than guessing.

Execution Step by Step

  1. Bash reads enough input to form a complete command or compound command.
  2. Parameter, command, and arithmetic expansions are performed where the syntax requests them.
  3. Quoting determines whether expanded text stays together as one argument.
  4. Redirections are connected before the selected command runs.
  5. The command returns an exit status, which a conditional, loop, or following command may use.

This sequence also explains why printing a variable can look correct even when passing it unquoted is unsafe: the destructive splitting happens while Bash constructs arguments, not while the value is stored.

Common Mistakes

  • Parsing structured formats with a fragile regular expression.
  • Using double quotes around an awk program and allowing Bash to expand $1 or $2.

Another common mistake is testing only the happy path. Shell programs interact with files, permissions, processes, and user-controlled strings. Always try an empty value, a value containing spaces, and a failing command. Those tests expose most quoting and error-handling defects early.

Best Practices

  • Use grep -F for literal text.
  • Keep sed and awk programs single-quoted.
  • Move complex transformations into a maintainable script or a language with a real parser.

For production automation, start scripts with a clear interpreter line, document inputs and outputs, keep functions small, and run ShellCheck. Strict settings can help, but explicit validation and purposeful error messages remain necessary.

Practice Exercises

  • Recreate the first grep, sed, and awk example without copying it, then change one input containing a space.
  • Add one validation failure that prints a useful message to standard error and returns a non-zero status.
  • Wrap the central behavior in a function and call it twice with different arguments.

Summary

  • grep, sed, and awk is easiest to use correctly when you track parsing, expansion, execution, and exit status separately.
  • Quoted expansions preserve data boundaries and should be the default.
  • Deterministic examples, syntax checks, and failure-path tests turn shell snippets into reliable automation.