Stage 2 · Tools
Text Processing
sed — Stream Editing
Substitute, delete, insert, and transform text streams in place.
sed Basics
sed (stream editor) reads input line by line and applies editing commands. It is designed for stream processing — transforming text as it flows through a pipeline. sed is essential for text manipulation in shell scripts.
# Basic syntax
sed 'COMMAND' file.txt
# Multiple commands
sed -e 'COMMAND1' -e 'COMMAND2' file.txt
# Command file
sed -f commands.sed file.txt
# Print specific lines
sed -n '5p' file.txt # Print line 5
sed -n '10,20p' file.txt # Print lines 10-20sed processes input line by line. Without -n, it prints every line. With -n, it only prints lines you explicitly request with the p (print) command.
Substitution
The s (substitute) command is sed's most used feature. It replaces text that matches a pattern. The syntax is s/pattern/replacement/flags.
# Replace first occurrence per line
sed 's/foo/bar/' file.txt
# Replace all occurrences per line
sed 's/foo/bar/g' file.txt
# Replace on specific lines
sed '3s/old/new/' file.txt # Line 3 only
sed '1,5s/old/new/g' file.txt # Lines 1-5
# Case-insensitive replacement
sed 's/error/ERROR/gI' file.txt
# Use different delimiters (useful for paths)
sed 's|/old/path|/new/path|g' file.txt
sed 's#http#https#g' file.txtThe g flag replaces all occurrences on each line (not just the first). I makes it case-insensitive. You can use any character as the delimiter — | and # are common alternatives to / for paths.
When replacing file paths, use | or # as delimiters instead of / to avoid escaping: sed 's|/old/path|/new/path|g' is much cleaner than sed 's/\/old\/path/\/new\/path/g'.
Address Ranges
sed commands can target specific lines using addresses. You can use line numbers, patterns, or ranges to control which lines get modified.
# Line numbers
sed '3d' file.txt # Delete line 3
sed '1,5d' file.txt # Delete lines 1-5
# Pattern matching
sed '/^#/d' file.txt # Delete comment lines
sed '/^$/d' file.txt # Delete empty lines
sed '/ERROR/p' logfile.txt # Print lines containing ERROR
# Address ranges
sed '/BEGIN/,/END/d' file.txt # Delete from BEGIN to END
sed '10,$d' file.txt # Delete from line 10 to end
# Negate addresses
sed '/^#/!d' file.txt # Delete everything except commentsAddresses can be numbers, patterns enclosed in /, or ranges separated by commas. The $ symbol represents the last line. ! negates an address.
In-Place Editing
sed normally prints its output to stdout. The -i flag edits files in place — modifying the original file. This is dangerous without a backup but essential for configuration management.
# Edit in place (no backup)
sed -i 's/old/new/g' config.txt
# Edit in place with backup
sed -i.bak 's/old/new/g' config.txt # Creates config.txt.bak
# macOS sed requires empty string argument for backup
sed -i '' 's/old/new/g' config.txt # macOS
# GNU sed (Linux)
sed -i 's/old/new/g' config.txt # LinuxmacOS sed and GNU sed have different -i syntax. macOS requires -i '' while GNU sed uses -i alone. For portable scripts, use sed -i.bak and remove the backup afterward.
sed -i modifies files directly. If your pattern is wrong, you corrupt the file. Always use sed -i.bak to create a backup, verify the result, then remove the backup with rm file.bak.
Multi-Line Processing
sed can operate on multiple lines at once using hold space commands. This is advanced but useful for transformations that span lines.
# Join lines (remove newlines between matching lines)
sed '/^#/N;s/\n//' file.txt
# Delete blank lines (including consecutive)
sed '/^$/d' file.txt
# Insert a line after each match
sed '/pattern/a\new line here' file.txt
# Insert a line before each match
sed '/pattern/i\new line here' file.txt
# Replace newline with comma (join all lines)
sed ':a;N;$!ba;s/\n/,/g' file.txtThe N command appends the next line to the pattern space. The hold space (h, H, g, G) allows storing and retrieving text across lines. These are advanced features for complex transformations.
Common Patterns
Here are practical sed patterns you will use frequently in shell scripts.
# Remove comments and blank lines (useful for config files)
sed '/^#/d; /^$/d; /^\s*$/d' config.txt
# Extract a value from a config file
sed -n 's/^PORT=//p' config.txt
# Add a line after a pattern
sed '/^\[server\]/a\port=8080' config.txt
# Comment/uncomment a line
sed 's/^/#/' file.txt # Comment all lines
sed 's/^#//' file.txt # Uncomment all lines
# Print only the first match
sed -n '/pattern/{p;q}' file.txt
# Remove ANSI color codes
sed 's/\x1b\[[0-9;]*m//g' logfile.txtsed is powerful but can be cryptic for complex operations. For anything beyond simple substitutions, consider whether awk would be more readable.
Use sed for simple find-and-replace operations and line deletions. Use awk when you need to process columns, perform calculations, or combine conditions with actions. When in doubt, try sed first — if it gets complicated, switch to awk.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.