Stage 3 · Build
Networking Stack Deep Dive
Socket API Internals
struct socket, inet sockets, epoll, io_uring, and the internals behind high-performance network I/O.
Socket Internals
When you call socket(), the kernel creates a struct socket and a protocol-specific struct sock. The socket layer handles the API, while the sock layer handles protocol logic. Understanding this separation helps diagnose socket-level issues.
# View socket statistics
cat /proc/net/sockstat
# TCP: inuse 45 orphan 12 tw 67 alloc 89 mem 23
# UDP: inuse 5 mem 2
# RAW: inuse 1
# FRAG: inuse 0 memory 0
# Per-socket details with ss
ss -tnp
# State Recv-Q Send-Q Local:Port Peer:Port Users
# ESTAB 0 0 10.0.0.1:22 10.0.0.2:54321 users:(("sshd",pid=1234,fd=3))ss -tnp shows TCP connections with process info. The Recv-Q for established connections shows data waiting to be read by the application — persistent values here indicate the application is not keeping up.
Socket Types
Linux supports several socket types, each with different semantics. The type determines how data is transmitted and received.
| Type | Protocol | Characteristics |
|---|---|---|
| SOCK_STREAM | TCP | Reliable, ordered byte stream |
| SOCK_DGRAM | UDP | Unreliable, message-based |
| SOCK_RAW | IP/ICMP | Direct IP access, bypasses TCP/UDP |
| SOCK_SEQPACKET | SCTP | Reliable, message-based |
| SOCK_RDM | Reliable datagram | Reliable, connectionless |
epoll: Event-Driven I/O
epoll is the Linux API for monitoring multiple file descriptors. Unlike select/poll, epoll scales to millions of connections because it uses an in-kernel event table rather than scanning all fds. It is the foundation of most high-performance web servers.
# epoll modes
# EPOLLLT (Level-Triggered) — Default. Notification when fd is ready.
# EPOLLET (Edge-Triggered) — Notification on state change only.
# Monitor socket events with ss
ss -tnp state established | wc -l
# 45 (active connections)
# Check epoll file descriptors
ls -la /proc/<pid>/fd | grep eventpoll
# lrwx------ 1 root root 64 ... 7 -> anon_inode:[eventpoll]
# Watch for socket events
strace -e epoll_wait -p <pid>Level-triggered is safer — you get notified whenever the fd is ready. Edge-triggered is more efficient but requires draining the fd completely on each notification to avoid missed events.
io_uring: Async I/O
io_uring is the modern Linux async I/O interface. It uses shared ring buffers between userspace and kernel to minimize syscall overhead. A single io_uring instance can handle network I/O, file I/O, and even timers without any syscalls.
# Check io_uring support
cat /proc/version
# io_uring available in kernel 5.1+
# io_uring vs epoll performance
# io_uring: ~1M IOPS for network I/O
# epoll: ~500K IOPS for network I/O
# Check io_uring statistics (kernel 5.10+)
cat /proc/sys/kernel/io_uring_disabled
# 0 = enabledio_uring reduces syscall overhead by submitting I/O requests through a shared memory ring buffer. The kernel processes them without context switches, giving near-native performance.
Essential Socket Options
# TCP_NODELAY — Disable Nagle's algorithm (lower latency)
sysctl net.ipv4.tcp_low_latency
# 1 = prefer low latency
# SO_REUSEADDR — Allow binding to TIME_WAIT ports
sysctl net.ipv4.tcp_tw_reuse
# 1 = enable (safe for client-side)
# SO_BACKLOG — Connection queue size
sysctl net.core.somaxconn
# 65535
# SO_SNDBUF/SO_RCVBUF — Per-socket buffer sizes
sysctl net.core.rmem_default net.core.wmem_default
# 212992TCP_NODELAY is critical for interactive protocols (SSH, databases). It sends data immediately rather than waiting to fill a TCP segment.
Performance Patterns
- Use epoll or io_uring for high connection counts
- Enable TCP_NODELAY for latency-sensitive protocols
- Tune buffer sizes based on bandwidth-delay product
- Use SO_REUSEPORT for multi-threaded accept()
- Consider io_uring for new high-performance servers
- Monitor socket memory with ss -m
# SO_REUSEPORT allows multiple sockets on the same port
# The kernel distributes connections across listening sockets
# Check if application uses SO_REUSEPORT
ss -tnlp | head -5
# LISTEN 0 128 0.0.0.0:80 0.0.0.0:* users:(("nginx",...))
# This enables the kernel to balance accept() across threads
# without a single accept() bottleneckSO_REUSEPORT is especially effective for multi-process servers like Nginx. Each worker can have its own listening socket, and the kernel distributes SYN packets evenly.
When using EPOLLET, you must read/write until EAGAIN. If you leave data in the buffer, you will not get another notification until new data arrives. Level-triggered (default) is safer for most applications.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.