Load Balancing Algorithms

Why Load Balancing?

Load balancing distributes incoming traffic across multiple backend servers. Without it, all traffic hits a single server, creating a bottleneck and single point of failure. The load balancing algorithm determines which server receives each request.

Round Robin

Round robin cycles through servers in order, sending each request to the next server. It is the simplest algorithm and works well when all servers have equal capacity and all requests have equal cost.

Request 1 ──► Server A
Request 2 ──► Server B
Request 3 ──► Server C
Request 4 ──► Server A
Request 5 ──► Server B
Request 6 ──► Server C

Least Connections

Least connections routes each request to the server with the fewest active connections. This adapts to real-time server load and is ideal for workloads where request duration varies significantly.

• Best for long-lived connections (WebSockets, database connections).
• Requires tracking connection count per server.
• More overhead than round robin but much more accurate load distribution.
• Can cause hot servers if connections are short and bursty.

IP Hash

IP hash uses the client IP address to determine which server receives the request. This provides session affinity — the same client always hits the same server. Useful for applications that store session state locally.

Consistent Hashing

Consistent hashing maps both servers and request keys onto a hash ring. When a server is added or removed, only the keys that mapped to that server are redistributed. This minimizes cache invalidation during scaling events.

         Hash Ring
    ┌─────────────────┐
    │     Server A    │
    │  ●──────────────┤
    │       │         │
    │  Server B  Server C
    │  ●──────────●   │
    │                 │
    └─────────────────┘

Key "user:123" ──hash──► lands between A and B
                          → routed to Server B

Add Server D between B and C:
  Only keys between B and C move to D
  All other keys stay put

Weighted Algorithms

Weighted algorithms assign different proportions of traffic based on server capacity. A server with 2x the capacity receives 2x the traffic. This is useful when your fleet has heterogeneous hardware.