Stage 2 · Tools
Process & System Management
File System Operations
find, xargs, rsync, tar, and safe file manipulation patterns for automation.
The find Command
find recursively searches directories for files matching criteria. It is more powerful and reliable than globbing because it handles hidden files, special characters, and deep directory trees.
# Find by name
find /var/log -name "*.log"
# Case-insensitive
find . -iname "readme*"
# Find by type
find . -type f -name "*.sh" # Files only
find . -type d -name "node_modules" # Directories only
find . -type l # Symlinks only
# Find by size
find . -size +100M # Larger than 100MB
find . -size -1k # Smaller than 1KB
# Find by time
find . -mtime -7 # Modified in last 7 days
find . -atime +30 # Accessed more than 30 days ago
find . -newer reference.txt # Newer than reference.txt
# Find by permissions
find . -perm 755 # Exact permissions
find . -perm -u+x # Executable by ownerfind is the most reliable way to locate files in complex directory structures. Unlike globs, it handles filenames with spaces, newlines, and special characters.
find . -print0 | xargs -0 handles filenames with spaces, newlines, and special characters correctly. The null byte is the only character that cannot appear in a filename.
xargs — Pipe to Arguments
xargs reads items from stdin and passes them as arguments to a command. It bridges the gap between piped output and command arguments.
# Basic usage
echo "file1 file2 file3" | xargs rm
# With find (safe for special filenames)
find . -name "*.tmp" -print0 | xargs -0 rm
# Parallel execution
find . -name "*.gz" | xargs -P 4 -I {} gunzip {}
# Limit arguments per invocation
cat urls.txt | xargs -n 1 curl -sO
# Dry run (show what would be executed)
find . -name "*.log" | xargs -p rmxargs -0 reads null-delimited input, making it safe for any filename. -P 4 runs up to 4 processes in parallel. -n 1 passes one argument at a time. -p prompts before each command.
rsync — Syncing Files
rsync synchronizes files between locations. It only transfers differences, making it efficient for backups and mirroring. It works locally or over SSH.
# Local sync
rsync -av /source/ /dest/
# Remote sync (push)
rsync -avz ./data/ user@remote:/backup/
# Remote sync (pull)
rsync -avz user@remote:/data/ ./local/
# Delete files not in source
rsync -av --delete /source/ /dest/
# Dry run
rsync -avn /source/ /dest/
# Exclude patterns
rsync -av --exclude='*.log' --exclude='.git' ./project/ /backup/
# Show progress
rsync -avP /large/file.tar.gz remote:/backup/The trailing slash matters: /source/ copies contents, /source copies the directory itself. -a preserves permissions, timestamps, and symlinks. -v is verbose. -z compresses during transfer.
rsync /source/ /dest/ copies contents of source into dest. rsync /source /dest/ copies source as a subdirectory of dest. This is the most common rsync mistake.
tar — Archives
tar bundles files into a single archive. Combined with gzip or bzip2, it creates compressed archives. tar is the standard for distributing and backing up files.
# Create gzip archive
tar -czf archive.tar.gz /path/to/dir
# Create bzip2 archive (better compression)
tar -cjf archive.tar.bz2 /path/to/dir
# Extract archive
tar -xzf archive.tar.gz
# Extract to specific directory
tar -xzf archive.tar.gz -C /dest/
# List archive contents
tar -tzf archive.tar.gz
# Exclude files
tar -czf archive.tar.gz --exclude='*.log' /path/to/dir
# Create archive from find output
find . -name "*.log" -mtime -7 -print0 | tar -czf logs.tar.gz --null -T -c creates, x extracts, t lists. z uses gzip, j uses bzip2. f specifies the filename. --null -T - reads null-delimited filenames from stdin, making it safe with find -print0.
File Tests and Permissions
Bash provides test operators for checking file properties. These are essential for writing scripts that validate state before acting.
FILE="/etc/hosts"
# Existence and type
[[ -f "$FILE" ]] && echo "Regular file"
[[ -d "$FILE" ]] && echo "Directory"
[[ -L "$FILE" ]] && echo "Symbolic link"
[[ -e "$FILE" ]] && echo "Exists (any type)"
# Permissions
[[ -r "$FILE" ]] && echo "Readable"
[[ -w "$FILE" ]] && echo "Writable"
[[ -x "$FILE" ]] && echo "Executable"
# Size and age
[[ -s "$FILE" ]] && echo "Not empty"
[[ "$FILE" -nt "other.txt" ]] && echo "Newer than other.txt"These tests are used in if statements and while loops. They let your script check state before acting — essential for idempotent operations.
Safe File Manipulation
File operations in scripts must handle edge cases: filenames with spaces, missing files, permission errors, and race conditions.
# Safe directory creation
mkdir -p "/path/to/nested/dir" # -p creates parents, no error if exists
# Safe file removal
rm -f "file.txt" # -f ignores nonexistent files
# Safe file move (atomic on same filesystem)
mv "file.txt" "file.txt.bak" 2>/dev/null || true
# Safe file creation with temp file
TMPFILE=$(mktemp)
trap 'rm -f "$TMPFILE"' EXIT
echo "data" > "$TMPFILE"
# Process files safely with null delimiters
while IFS= read -r -d '' file; do
echo "Processing: $file"
done < <(find . -name "*.txt" -print0)Always quote variable expansions in file operations. Use -print0 and read -d '' for filenames with special characters. Use trap to clean up temp files.
For critical files, write to a temp file first, then use mv temp target to atomically replace the target. This prevents corruption if the script is interrupted during the write.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.