Stage 3 · Build
Boot & Init Systems
journald Boot Logs
journalctl -b, priorities, persistent journals, rate limits, and forwarding logs to syslog.
journald Overview
systemd-journald is the systemd logging daemon. It collects logs from the kernel, initrd, services, and stdout/stderr of all processes. Logs are stored in a structured binary format with rich metadata.
# Runtime journal (lost on reboot)
ls -lh /run/log/journal/
# 9a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d/
# Persistent journal (survives reboot)
ls -lh /var/log/journal/
# 9a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d/
# system.journal
# Journal configuration
cat /etc/systemd/journald.conf
# [Journal]
# Storage=auto
# SystemMaxUse=100M
# SystemMaxFileSize=50M
# RateLimitIntervalSec=30s
# RateLimitBurst=10000journald stores logs in binary format for fast searching. By default, logs are volatile (lost on reboot). Enable persistent storage for production systems.
journalctl Basics
# View all logs
journalctl
# View logs for current boot
journalctl -b
# View previous boot logs
journalctl -b -1
# View logs for a specific unit
journalctl -u nginx.service
journalctl -u nginx.service -u php-fpm.service
# View logs with priority filtering
journalctl -p err # Errors only
journalctl -p warning # Warnings and above
# View logs since/until a time
journalctl --since "2024-01-15 10:00:00" --until "2024-01-15 12:00:00"
journalctl --since "1 hour ago"
journalctl --since today
# Follow logs in real-time
journalctl -f
# View kernel logs
journalctl -k
# View logs from a specific PID
journalctl _PID=12345journalctl is the primary tool for querying the journal. Use -u for services, -p for priority, -b for boot-specific logs.
Boot Log Analysis
# List available boots
journalctl --list-boots
# -3 1f2a3b4c... Mon 2024-01-15 10:30:00 UTC - Mon 2024-01-15 15:00:00 UTC
# -2 2a3b4c5d... Mon 2024-01-15 15:00:00 UTC - Mon 2024-01-16 09:00:00 UTC
# -1 3b4c5d6e... Mon 2024-01-16 09:00:00 UTC - Tue 2024-01-17 08:00:00 UTC
# 0 4c5d6e7f... Tue 2024-01-17 08:00:00 UTC - ...
# View current boot logs
journalctl -b
# View previous boot
journalctl -b -1
# Filter by boot and priority
journalctl -b -p err
# View boot time
journalctl -b | head -1
# -- Boot abc123def456 (Kernel: 6.1.0-15-generic) --
# Show only systemd-related boot messages
journalctl -b -u systemd-sysusers.service
journalctl -b -t systemdUse --list-boots to find the boot ID, then -b to view logs from that specific boot. Essential for diagnosing boot failures.
Persistent Journal
# Enable persistent storage
sudo mkdir -p /var/log/journal
sudo systemd-tmpfiles --create --prefix /var/log/journal
# Or edit /etc/systemd/journald.conf
# Storage=persistent
# Restart journald
sudo systemctl restart systemd-journald
# Configure retention
# /etc/systemd/journald.conf
# SystemMaxUse=500M # Maximum disk usage
# SystemMaxFileSize=100M # Maximum size per file
# MaxRetentionSec=6month # Keep logs for 6 months
# MaxFileSec=1week # Rotate files weekly
# Manually vacuum old logs
sudo journalctl --vacuum-time=30d
sudo journalctl --vacuum-size=500MPersistent journal stores logs across reboots. Set SystemMaxUse and MaxRetentionSec to control disk usage.
Rate Limiting
# Default rate limits in journald.conf
# RateLimitIntervalSec=30s
# RateLimitBurst=10000
# This means: max 10000 messages per 30 seconds per unit
# Messages beyond this are dropped
# Disable rate limiting (use with caution)
# RateLimitIntervalSec=0
# RateLimitBurst=0
# Check if messages were dropped
journalctl -b | grep -c "Suppressed"
# 45 (messages were dropped)
# Per-unit rate limits can be set in unit files
# [Service]
# LogLevelMax=info
# LogRateLimitIntervalSec=0
# LogRateLimitBurst=0Rate limiting prevents runaway services from filling the journal. Disable it only for specific debugging scenarios.
Log Forwarding
# Forward to remote syslog server
# /etc/systemd/journald.conf
# ForwardToSyslog=yes
# SyslogHost=10.0.0.5
# SyslogFacility=daemon
# Forward to /var/log/syslog
# /etc/rsyslog.d/99-journald.conf
# module(load="imjournal")
# input(type="imjournal" PersistStateFile="/var/lib/rsyslog/imjournal.state")
# Forward via remote journal (systemd-networkd)
# /etc/systemd/journal-upload.conf
# [Upload]
# URL=https://logserver.example.com:19532
# ServerKeyFile=/etc/ssl/private/journal-upload.pem
# ServerCertificateFile=/etc/ssl/certs/journal-upload.crt
# Receive remote journals
# /etc/systemd/journal-remote.conf
# [Remote]
# SplitMode=host
# Seal=noFor centralized logging, forward journal to a remote server. Use TLS certificates for secure transmission.
journalctl -f follows the journal in real-time. Combine with -u for a specific service to watch logs as you test changes.
Without persistent storage, all journal logs are lost on reboot. Enable persistent storage for any production system where you need historical logs.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.