Stage 3 · Build
Users, Permissions & Processes
Process Inspection Tools
ps, pgrep, pstree, lsof, strace, and reading /proc for live process state.
ps Deep Dive
# BSD style (no dash)
ps aux
# USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
# root 1 0.0 0.1 1234 5678 ? Ss Jan15 0:01 /sbin/init
# POSIX style (with dash)
ps -ef
# Custom output format
ps -eo pid,ppid,ni,cls,rtprio,comm --sort=-ni
# PID PPID NI CLS RTPRIO COMMAND
# 12345 1 -5 FF 50 nginx
# Filter by process
ps aux | grep nginx
ps -C nginx -o pid,comm,%cpu,%mem
# Thread view
ps -eLf | grep nginx
# Shows individual threads (LWP)
# Show process tree
ps axjf
# Show memory details
ps -eo pid,rss,vsz,comm --sort=-rss | head -10ps is the primary process inspection tool. Use -eo for custom fields and --sort for ordering. BSD style (aux) is most commonly used.
pgrep and pkill
# Find PID by name
pgrep nginx
# 1234
# 1235
# Find with full details
pgrep -a nginx
# 1234 nginx: worker process
# Find by user
pgrep -u www-data
# Find by parent
pgrep -P 1
# Find by command line pattern
pgrep -f "python3.*server.py"
# Kill by name
pkill nginx
pkill -9 nginx # Force kill
# Kill by pattern
pkill -f "python3.*server.py"
# Send specific signal
pkill -HUP nginx
# Count matching processes
pgrep -c nginx
# 4pgrep finds PIDs by pattern, pkill sends signals to matching processes. Use -f to match the full command line, not just the process name.
pstree Visualization
# Show process tree
pstree
# systemd─┬─NetworkManager───2*[{NetworkManager}]
# ├─agetty
# ├─crond
# ├─dbus-daemon───{dbus-daemon}
# ├─dockerd───docker-proxy───containerd-shim───nginx───6*[{nginx}]
# ├─sshd───sshd───bash───pstree
# └─systemd-journal
# Show PIDs
pstree -p
# Show full command lines
pstree -a
# Show specific process tree
pstree -a 1234
# Show with thread groups
pstree -H 1234
# Vertical layout
pstree -l
# Sort by PID
pstree -spstree shows process relationships as a tree. It helps understand parent-child relationships and identify orphaned processes.
lsof File Inspection
# List all open files
lsof | head -20
# COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
# systemd 1 root cwd DIR 253,1 4096 2 /
# List files opened by a process
lsof -p 12345
# List processes using a file
lsof /var/log/syslog
# List network connections
lsof -i
lsof -i :22 # Connections on port 22
lsof -i tcp # TCP connections only
# List deleted files still in use
lsof +L1
# Shows files that were deleted but still have open handles
# List files opened by user
lsof -u www-data
# Find who is using a port
lsof -i :80 -P -n
# COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
# nginx 1234 root 6u IPv4 123456 0t0 TCP *:80 (LISTEN)lsof shows open files, network connections, and deleted files. It is invaluable for diagnosing 'file in use' errors and finding processes on specific ports.
strace Syscall Tracing
# Trace all syscalls
strace ls /tmp
# Trace specific syscalls
strace -e trace=open,read,write cat /etc/hostname
# Trace with timing
strace -T curl http://localhost
# Attach to running process
strace -p 12345
# Trace child processes
strace -f -p 12345
# Count syscall frequency
strace -c ls /tmp
# % time seconds usecs/call calls errors
# ------ ----------- ----------- --------- ---------
# 45.23 0.000234 23 10 getdents64
# Output to file
strace -o trace.log -f -p 12345
# Filter by syscall type
strace -e trace=network curl http://example.comstrace shows every syscall a process makes. Use -c for summary statistics, -T for timing, and -e to filter by syscall type.
/proc Filesystem
# Process information
cat /proc/12345/status
# Name: nginx
# State: S (sleeping)
# Tgid: 12345
# Pid: 12345
# PPid: 1
# Uid: 33 33 33 33
# Memory mapping
cat /proc/12345/maps | head -10
# Open files
ls -la /proc/12345/fd/ | head -10
# Command line
cat /proc/12345/cmdline | tr '\0' ' '
# Environment
cat /proc/12345/environ | tr '\0' '\n'
# File descriptors
ls -la /proc/12345/fd/ | wc -l
# 42 open file descriptors
# Network connections
cat /proc/12345/net/tcp | head -5/proc is a virtual filesystem that exposes kernel data structures. /proc/PID/ contains all information about a specific process.
lsof +L1 shows files that were deleted but still have open file handles. The disk space is not freed until all handles are closed. Find the process and restart it to free the space.
strace intercepts every syscall, causing 10-100x slowdown. Never use strace on production systems without understanding the performance impact. Use perf or bpftrace for lower overhead.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.