Stage 3 · Build
Linux Kernel Internals
Process Management
task_struct, PID namespaces, fork/exec, and the scheduler — how Linux manages every running program.
The task_struct
Every process in Linux is represented by a task_struct — a massive C structure (roughly 6 KB on modern kernels) that holds everything the kernel needs to know about a process: its PID, memory map, open files, scheduling priority, signal handlers, and hundreds of other fields.
cat /proc/self/status | head -20
# Name: cat
# State: S (sleeping)
# Tgid: 12345
# Pid: 12345
# PPid: 6789
# Uid: 1000 1000 1000 1000
# Gid: 1000 1000 1000 1000The /proc filesystem exposes many task_struct fields as human-readable text. Tgid is the thread group ID (the main process PID), while Pid is the individual thread ID.
In Linux, there is no distinction between processes and threads at the kernel level. Both are task_structs. Threads share address space and file descriptors; independent processes do not.
Process Creation: fork/exec
Linux creates processes using a two-step mechanism: fork() creates a copy of the calling process, and exec() replaces that copy with a new program. This separation allows the child to modify its state before loading a new binary.
# strace shows the fork+exec sequence
strace -f -e trace=clone,clone3,execve bash -c "ls"
# clone() creates a new task with shared resources
# execve() loads the new program binary
# The shell forks, then the child execs lsThe clone() syscall is the modern interface for fork(). It lets the caller specify exactly which resources to share (memory, files, signals) between parent and child.
PID Namespaces
PID namespaces isolate the PID number space. A process has a PID visible within its namespace and a different PID in the parent namespace. This is the foundation of container isolation.
# Create a new PID namespace and run a shell in it
sudo unshare --pid --fork --mount-proc bash
# Inside the namespace, PID 1 is your shell
echo $$ # 1
ps aux # Only processes in this namespace
# Exit the namespace
exitunshare --pid creates a new PID namespace. --mount-proc remounts /proc so ps shows only namespace-local processes. --fork is required so the child process is PID 1 in the new namespace.
# See namespace relationships
ls -la /proc/*/ns/pid | head -5
# Check which namespace a process belongs to
readlink /proc/self/ns/pid
# pid:[4026531836]
readlink /proc/1/ns/pid
# pid:[4026531836] (same namespace),Each namespace has a unique inode number. Processes in the same namespace share the same inode value.
The Scheduler
The Linux scheduler uses the Completely Fair Scheduler (CFS) as its default. CFS uses a red-black tree to track runnable processes and assigns each a virtual runtime. The process with the smallest virtual runtime runs next, ensuring fair CPU distribution.
# Check process scheduling policy
chrt -p $(pidof sshd)
# View scheduler statistics
cat /proc/schedstat | head -5
# Check CFS bandwidth control
cat /proc/sys/kernel/sched_min_granularity_ns
cat /proc/sys/kernel/sched_latency_nsCFS parameters like sched_min_granularity_ns control how the scheduler divides CPU time. Shorter values improve responsiveness but increase overhead.
| Scheduler | Type | Use Case |
|---|---|---|
| SCHED_NORMAL (CFS) | Fair | Default for all regular processes |
| SCHED_FIFO | Real-time | Fixed-priority, preempts CFS |
| SCHED_RR | Real-time | FIFO with time slicing |
| SCHED_BATCH | Fair | Optimized for CPU-bound batch work |
Process States
A process transitions through several states during its lifetime. Understanding these states is essential for diagnosing performance issues and zombie processes.
- R (Running/Runnable) — In the run queue, waiting for or using CPU
- S (Sleeping) — Waiting for an event (I/O, signal, timer)
- D (Disk Sleep) — Uninterruptible sleep, usually waiting for disk I/O
- T (Stopped) — Suspended by signal (SIGSTOP) or job control
- Z (Zombie) — Terminated but not yet reaped by parent
- I (Idle) — Kernel idle threads, not visible to userspace
# Show state of all processes
ps -eo pid,stat,comm | head -20
# Watch for D-state (disk sleep) processes
watch -n 1 'ps -eo pid,stat,comm | grep " D "'
# Count processes in each state
ps -eo stat | tail -n +2 | sort | uniq -c | sort -rnD-state processes are often waiting on slow storage. If many processes are stuck in D-state, investigate disk I/O with iostat or iotop.
Zombie Processes
A zombie process has finished execution but its parent has not yet called wait() to read its exit status. Zombies consume a PID slot but no other resources. They are usually harmless but indicate a parent that is not reaping children.
# Find zombie processes
ps aux | awk '$8 ~ /Z/ {print $2, $11}'
# Identify the parent of a zombie
ps -o ppid= -p <zombie_pid>
# If the parent won't reap, you can kill the parent
# This reparents zombies to init/systemd which reaps them
kill -SIGCHLD <parent_pid>Killing the parent process causes init (PID 1) to adopt and reap the zombies. This is usually safe for zombie cleanup, but understand why the parent was not reaping before doing this in production.
Sending SIGKILL to a zombie has no effect — the process is already dead. Only the parent can reap it by calling wait(). The correct approach is to fix the parent or kill the parent so init reaps the zombie.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.