sed: Stream Editing Basics

sed (short for stream editor) is a command-line tool that reads text one line at a time and applies simple, script-like editing commands to it — substituting text, deleting lines, printing only the lines you want, and more — without ever opening an interactive editor. It is one of the oldest and most heavily used tools in the Unix toolbox, and on Linux you will reach for it constantly: rewriting configuration files during deployment, cleaning up log output, and transforming text inside shell scripts. Because sed works line by line and prints as it goes, it is fast, memory-light, and composes naturally with pipes.

Overview / How it works

sed operates in a loop called the cycle. For each line of input it: (1) reads the line into an internal buffer called the pattern space, stripping the trailing newline; (2) runs your script against that pattern space, applying any commands whose address (a line number, range, or regular expression) matches; and (3) unless you told it otherwise, prints the pattern space to standard output and adds the newline back. Then it moves to the next line and repeats until the input is exhausted. This is why a script as short as s/foo/bar/ can rewrite an entire file: sed is really just running that tiny substitution once per line, automatically.

By default sed uses Basic Regular Expressions (BRE), the older regex dialect where metacharacters like (, ), +, and { are treated as literal characters unless you escape them with a backslash (\(, \)). Passing -E (or the older -r) switches to Extended Regular Expressions (ERE), where those characters are metacharacters by default, matching the flavor most people already know from other tools. This course uses GNU sed, the version installed by default on Ubuntu and Debian (check with sed --version); it has convenient extensions beyond the POSIX standard, some of which are used in this lesson.

Normally sed only ever reads standard input or the files you name and writes to standard output — it never touches the original file. The -i (in-place) option changes that: GNU sed writes the edited output to a temporary file in the same directory and then renames it over the original once the whole file has been processed successfully. That rename is why -i is reasonably safe against a script crashing partway through, but it offers no protection against a script that runs to completion and simply does the wrong substitution — there is no undo. That is why testing a script without -i first, or keeping a backup, matters (see Common Mistakes below).

Syntax

The general form of a sed command is:

sed [OPTIONS] 'SCRIPT' [FILE...]

If no file is given, sed reads from standard input, so it works naturally at the end of a pipeline. The most common options are:

Option	Meaning
-n	Suppress automatic printing of the pattern space; only print what an explicit p command (or similar) outputs.
-e SCRIPT	Add a script to the commands to run; combine multiple -e flags to run several edits in one pass.
-i[SUFFIX]	Edit files in place. With a suffix (-i.bak) the original is saved with that suffix appended before being overwritten.
-E, -r	Use Extended Regular Expressions instead of the default Basic Regular Expressions.
-f FILE	Read the sed script from a file instead of the command line.

Inside the script, the most-used commands are:

Command	Meaning
s/PATTERN/REPLACEMENT/FLAGS	Substitute text matching PATTERN with REPLACEMENT. Common flags: g (replace every match on the line, not just the first), i (case-insensitive), or a number (replace only the Nth match).
p	Print the pattern space (usually paired with -n).
d	Delete the pattern space and start the next cycle immediately (nothing is printed for that line).
q	Quit processing immediately.
=	Print the current line number.

Addresses control which lines a command applies to: a bare number (3), a range (2,4), the last line ($), a step (1~2 for every other line, a GNU extension), or a regular expression between slashes (/ERROR/).

Examples

Example 1: preview a substitution before applying it

Suppose app.conf contains:

host = localhost
port = 8080

Before touching the real file, run sed without -i to see the edited result printed to the terminal — the file itself stays untouched:

sed 's/localhost/127.0.0.1/' app.conf

Output:

host = 127.0.0.1
port = 8080

sed read the first line into the pattern space, matched localhost, replaced it with 127.0.0.1, and auto-printed the result; the second line had no match, so it was printed unchanged. Nothing was written back to app.conf.

Example 2: edit in place with a safety backup

Suppose .env contains a single line, DEBUG=false. Once you have previewed the change, apply it in place while keeping a backup of the original:

sed -i.bak 's/DEBUG=false/DEBUG=true/' .env

Check what happened:

ls .env*

Output:

.env  .env.bak

.env now contains DEBUG=true, and .env.bak holds the original content exactly as it was before the edit, because GNU sed copied it aside before overwriting the file.

Example 3: extract matching lines from a log

Suppose /var/log/app.log is a large log file and you only want the lines that mention an error:

sed -n '/ERROR/p' /var/log/app.log

Output:

2026-08-04 09:12:03 ERROR failed to connect to database
2026-08-04 09:14:51 ERROR request timed out

-n turned off the automatic print, so only lines that matched the regular expression /ERROR/ and were explicitly printed by p made it to the output — everything else in the log file was silently skipped.

How it works step by step

Take Example 3, sed -n '/ERROR/p' /var/log/app.log, and walk through what happens for each line of the file:

1. sed reads one line into the pattern space and strips its trailing newline.
2. It checks the address /ERROR/ against the pattern space using the current regex dialect (BRE by default).
3. If the line matches, the p command runs and prints the pattern space, plus a newline, to standard output.
4. If the line does not match, no command in the script runs for that cycle.
5. Because -n was given, sed does not perform its normal automatic print at the end of the cycle — only an explicit p produces output. Without -n, every line would print once automatically, and matching lines would print a second time from p.
6. The pattern space is discarded and sed reads the next line, repeating the cycle until end of file.

Deletion works from the opposite direction. sed '/^$/d' notes.txt matches every blank line (^$ means start of line immediately followed by end of line) and runs d, which deletes the pattern space and jumps straight to the next cycle, skipping the automatic print entirely for that line. Every non-blank line has no matching command, so it falls through to the normal automatic print. The net effect is a copy of the file with all blank lines removed.

Common Mistakes

1. Running -i on an important file without testing first

Because -i overwrites the file with no built-in undo, running an untested pattern directly against a file you cannot easily restore is risky:

sed -i 's/staging/production/' deploy.conf

If the pattern has a typo or matches more than intended, the file is silently rewritten and the original content is gone. Preview first without -i, then run with a backup suffix once you are confident:

sed 's/staging/production/g' deploy.conf
sed -i.bak 's/staging/production/g' deploy.conf

2. Forgetting the g flag

s/// only replaces the first match on each line unless told otherwise. On a CSV file this silently leaves most fields untouched:

sed 's/,/;/' orders.csv

Only the first comma on each line becomes a semicolon. Add the g flag to replace every match on every line:

sed 's/,/;/g' orders.csv

3. Using / as the delimiter for a pattern that contains slashes

The / character is just a conventional delimiter, not a required one. When the pattern is a file path, sticking with / forces you to escape every slash, which is easy to get wrong:

sed 's/\/usr\/local\/bin/\/opt\/bin/' setup.sh

Pick a delimiter that does not appear in the text instead, such as | or #, and the command becomes far easier to read:

sed 's|/usr/local/bin|/opt/bin|' setup.sh

4. Expecting parentheses to capture groups in Basic Regular Expressions

Without -E, sed uses BRE, where ( and ) are literal characters, not group markers — so a capture-group reference like \1 will not do what you expect:

sed 's/(foo)/[\1]/' notes.txt

Either escape the parentheses to turn them into a BRE group, or switch to extended regular expressions where plain parentheses already act as a group:

sed 's/\(foo\)/[\1]/' notes.txt
sed -E 's/(foo)/[\1]/' notes.txt

Best Practices

• Always run a sed script without -i first (or check the printed output) to confirm it does exactly what you expect before letting it modify a file in place.
• When you do use -i, pass a backup suffix (-i.bak) until you fully trust the script, then remove the backups once you have verified the result.
• Prefer -E when your pattern uses groups, alternation, or +/? — it reads closer to the regular expressions you already know from grep -E or other languages.
• Wrap the script in single quotes, not double quotes, unless you specifically need the shell to expand a variable inside it — double quotes let the shell interpret $ and backticks before sed ever sees the script.
• Pick a delimiter other than / (such as |, #, or ~) whenever the pattern or replacement itself contains slashes, such as file paths.
• Combine multiple edits into one invocation with several -e flags or semicolons instead of piping one sed into another — it is faster and easier to read.
• Remember GNU sed (Linux) and BSD sed (macOS) differ in the syntax for -i: GNU accepts -i.bak as one argument, BSD requires -i .bak as two arguments. Scripts meant to run on both need to account for this.

Practice Exercises

1. You have a file deploy.conf containing the line region = us-east-1. Write a command that previews changing it to us-west-2 without modifying the file, then a second command that applies the change in place while keeping a .bak backup.
2. You have a file access.log with many blank lines scattered through it. Write a single sed command that prints the file with all blank lines removed. Hint: the address /^$/ matches an empty line.
3. You have a file notes.txt with 40 lines. Write a command that prints only lines 10 through 20, and a second command that prints every line except those containing the word DRAFT. Hint: a ! right after an address negates it, for example /DRAFT/!p combined with -n.

Summary

• sed reads input one line at a time into a pattern space, applies your script, and auto-prints the result unless you pass -n.
• The most common command is s/PATTERN/REPLACEMENT/FLAGS; add g to replace every match on a line, not just the first.
• Addresses — line numbers, ranges, $, or /regex/ — restrict which lines a command runs on.
• -i edits files in place by writing a temp file and renaming it over the original; always test without it first, and keep a backup suffix while you build confidence.
• BRE, the default, requires escaping (, ), and + to use them as metacharacters; -E switches to ERE where they work unescaped.
• Choose a delimiter other than / when your pattern contains slashes, and always quote the script in single quotes unless you need shell variable expansion.