Stage 2 · Tools
Scripting Patterns
Idempotent Scripts
Write scripts that are safe to run multiple times without side effects.
What Is Idempotent?
An idempotent operation produces the same result whether it runs once or multiple times. Idempotent scripts are safe to re-run — they check current state before acting and only make changes when necessary.
This is critical for automation: scripts may run multiple times due to retries, manual re-execution, or scheduling. Non-idempotent scripts cause duplicate work, data corruption, or failed deployments.
# NOT idempotent — runs every time
echo "new line" >> config.txt # Adds duplicate line on re-run
mkdir /data # Fails on second run
git clone repo /project # Fails on second run
# Idempotent — safe to re-run
grep -q "new line" config.txt || echo "new line" >> config.txt
mkdir -p /data # No error if exists
git clone repo /project 2>/dev/null || true # Ignore if existsThe key difference: idempotent scripts check state before acting. grep -q checks if the line exists. mkdir -p does not fail if the directory exists. git clone with || true ignores the already-cloned error.
Check Before Acting
# Check if user exists before creating
if ! id "$USERNAME" &>/dev/null; then
sudo useradd "$USERNAME"
fi
# Check if service is installed before configuring
if systemctl is-enabled "$SERVICE" &>/dev/null; then
sudo systemctl restart "$SERVICE"
fi
# Check if package is installed
if ! dpkg -l "$PACKAGE" &>/dev/null 2>&1; then
sudo apt-get install -y "$PACKAGE"
fi
# Check if file exists before copying
if [[ ! -f "$DEST" ]] || ! diff -q "$SRC" "$DEST" &>/dev/null; then
cp "$SRC" "$DEST"
echo "Updated $DEST"
else
echo "$DEST is up to date"
fiThe pattern is: check condition, act only if needed. command &>/dev/null suppresses output. || true ignores errors. diff -q checks if files differ.
State Management
#!/usr/bin/env bash
set -euo pipefail
STATE_DIR="/var/lib/myscript"
STATE_FILE="$STATE_DIR/last_run"
ensure_state_dir() {
mkdir -p "$STATE_DIR"
}
already_run() {
[[ -f "$STATE_FILE" ]] && [[ "$(cat "$STATE_FILE")" == "$1" ]]
}
mark_complete() {
echo "$1" > "$STATE_FILE"
}
# Usage
ensure_state_dir
VERSION="1.2.3"
if already_run "$VERSION"; then
echo "Version $VERSION already deployed"
exit 0
fi
# ... deploy ...
mark_complete "$VERSION"
echo "Deploy complete"State files track what has been done. The script checks the state file before acting. If the current version is already deployed, it exits. Otherwise, it runs and updates the state.
Idempotent File Operations
#!/usr/bin/env bash
set -euo pipefail
# Ensure line exists in file (idempotent)
ensure_line() {
local file="$1"
local line="$2"
grep -qF "$line" "$file" 2>/dev/null || echo "$line" >> "$file"
}
# Ensure file content (overwrite if different)
ensure_file() {
local file="$1"
local content="$2"
local current
current=$(cat "$file" 2>/dev/null || echo "")
if [[ "$current" != "$content" ]]; then
echo "$content" > "$file"
echo "Updated $file"
fi
}
# Add to PATH if not already there
ensure_path() {
local dir="$1"
if ! echo "$PATH" | tr ':' '\n' | grep -q "^{dir}$"; then
echo "export PATH="$dir:$PATH"" >> ~/.bashrc
fi
}
# Create symlinks (idempotent)
link_file() {
local src="$1"
local dest="$2"
if [[ -L "$dest" ]]; then
local current_target
current_target=$(readlink "$dest")
if [[ "$current_target" == "$src" ]]; then
return 0 # Already linked correctly
fi
rm "$dest"
elif [[ -e "$dest" ]]; then
rm "$dest"
fi
ln -s "$src" "$dest"
echo "Linked $dest -> $src"
}ensure_line() uses grep -qF to check for exact string match. ensure_file() compares content before writing. link_file() handles existing files, symlinks, and missing targets.
Idempotent System Operations
#!/usr/bin/env bash
set -euo pipefail
# Ensure user exists
ensure_user() {
local username="$1"
local groups="{2:-}"
if ! id "$username" &>/dev/null; then
sudo useradd -m "$username"
echo "Created user $username"
fi
if [[ -n "$groups" ]]; then
for group in $(echo "$groups" | tr ',' ' '); do
if ! groups "$username" 2>/dev/null | grep -q "$group"; then
sudo usermod -aG "$group" "$username"
echo "Added $username to $group"
fi
done
fi
}
# Ensure directory exists with correct permissions
ensure_dir() {
local dir="$1"
local owner="{2:-root:root}"
local perms="{3:-755}"
if [[ ! -d "$dir" ]]; then
sudo mkdir -p "$dir"
echo "Created $dir"
fi
sudo chown "$owner" "$dir"
sudo chmod "$perms" "$dir"
}
# Ensure service is enabled and running
ensure_service() {
local service="$1"
if ! systemctl is-enabled "$service" &>/dev/null; then
sudo systemctl enable "$service"
echo "Enabled $service"
fi
if ! systemctl is-active "$service" &>/dev/null; then
sudo systemctl start "$service"
echo "Started $service"
fi
}
# Usage
ensure_user "deploy" "docker,adm"
ensure_dir "/opt/app" "deploy:deploy" "755"
ensure_service "nginx"Each ensure_ function checks state before acting. They can be run multiple times safely. The first run creates/configures. Subsequent runs are no-ops. This is the foundation of configuration management.
Ansible, Terraform, Puppet, and Chef all rely on idempotency. Your shell scripts should too. Always ask: 'If this script runs twice, will the second run produce the same result as the first?'
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.