Stage 4 · Provision
Designing for High Availability
Redundancy Patterns
Active-passive, active-active, and quorum-based designs for eliminating single points of failure.
Why Redundancy Matters
Redundancy is the practice of duplicating critical components so that if one fails, another takes over. Without redundancy, every component is a single point of failure. The goal is not to prevent failures but to ensure that any single failure does not cause system-wide outage.
Active-Passive (Cold/Warm/Hot)
In active-passive setups, one instance handles traffic while the standby sits idle or partially ready. The standby takes over when the active fails. The recovery time depends on how warm the standby is.
| Standby Type | Data Sync | Failover Time | Cost |
|---|---|---|---|
| Cold | None (restore from backup) | Minutes to hours | Low |
| Warm | Async replication, pre-provisioned | 10-60 seconds | Medium |
| Hot | Sync replication, always running | Sub-second | High |
Primary (Active)
│
│── async replication ──► Replica (Passive)
│
│── health check ──► Load Balancer
│
[Active] ── traffic ──► Primary
[Standby] (idle)The load balancer or DNS record points to the active node. When health checks fail, traffic is rerouted to the standby.
A cold standby that takes hours to restore from backup provides disaster recovery, not high availability. True HA requires warm or hot standbys with sub-minute failover.
Active-Active
In active-active, multiple instances handle traffic simultaneously. This provides both redundancy and horizontal scaling. However, it introduces complexity around data consistency, conflict resolution, and traffic routing.
- All instances are live and accepting traffic.
- Failover is instant — the remaining instances absorb the load.
- Requires conflict resolution for write-heavy workloads.
- More expensive but provides both HA and scale.
redis_cluster:
nodes:
- host: redis-us-east-1a
role: primary
region: us-east-1
- host: redis-us-west-2a
role: primary
region: us-west-2
replication:
type: crdt # Conflict-free replicated data types
sync_interval: 100ms
routing: anycast # Nearest node handles the requestCRDTs allow both regions to accept writes and merge them without conflicts. This is essential for active-active setups where both sides can mutate the same key.
Quorum-Based Replication
Quorum-based systems require a minimum number of nodes to agree before committing a write or responding to a read. This balances consistency with availability. The classic formula: W + R > N, where W is write quorum, R is read quorum, and N is replicas.
N = 3 # Total replicas
W = 2 # Write quorum
R = 2 # Read quorum
# W + R > N → 2 + 2 > 3 → True
# This guarantees reading the latest write
# For stronger consistency:
# W = N (all replicas must acknowledge)
# R = 1 (any replica can serve reads)
# Tradeoff: higher write latency, lower read latencyAdjusting W and R lets you tune the consistency-availability tradeoff. Higher W means more consistent writes but slower. Higher R means more consistent reads but slower.
Leader Election
Leader election determines which node in a cluster is the primary. Tools like ZooKeeper, etcd, and Consul use consensus protocols (Raft, Zab) to elect leaders. When the leader fails, a new one is elected automatically.
- Raft — Used by etcd, Consul, CockroachDB. Understandable and widely implemented.
- Zab — Used by ZooKeeper. Optimized for atomic broadcasts.
- Paxos — Used by Google Spanner. Theoretically foundational but complex.
- Leader leases — Prevent split-brain by making leaders renew their lease periodically.
Choosing the Right Pattern
There is no universal best pattern. The choice depends on your consistency requirements, latency budget, cost constraints, and operational complexity tolerance. Most systems use a combination.
Redundancy without health checks, automatic failover, and testing is just wasted resources. The redundancy pattern is only as good as your ability to detect and recover from failure.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.