Stage 3 · Build
NoSQL & Distributed Databases
CAP Theorem & Consistency Models
Consistency, availability, partition tolerance — real tradeoffs, not just theory.
CAP Theorem Defined
The CAP theorem states that a distributed system can provide at most two of three guarantees: Consistency (every read returns the most recent write), Availability (every request receives a response), and Partition Tolerance (the system continues operating during network failures).
Since network partitions are inevitable in distributed systems, the real choice is between Consistency and Availability during a partition. You must pick one — you cannot have both.
| System Type | Guarantee | Tradeoff | Examples |
|---|---|---|---|
| CP | Consistency + Partition tolerance | Unavailable during partitions | PostgreSQL, MongoDB, HBase |
| AP | Availability + Partition tolerance | Stale reads during partitions | Cassandra, DynamoDB, Riak |
| CA | Consistency + Availability | No partition handling | Single-node PostgreSQL |
A CA system is one that does not handle partitions — it assumes a reliable network. In practice, this means a single-node database or a system on a single machine. The moment you distribute across machines, you must handle partitions.
Real Tradeoffs in Practice
-- PostgreSQL defaults to consistency
-- Under network partition, replicas may become unavailable
-- Synchronous replication: strong consistency
ALTER SYSTEM SET synchronous_commit = on;
-- Replicas lag by 0ms but primary blocks if no replica acknowledges
-- Asynchronous replication: availability
ALTER SYSTEM SET synchronous_commit = off;
-- Primary continues even if replicas are unreachable
-- Tradeoff: committed writes may be lost on primary failureEven within a single database, you choose between consistency and availability. Synchronous replication ensures consistency but reduces availability. Asynchronous replication ensures availability but risks data loss.
Consistency Models
| Model | Guarantee | Performance | Use Case |
|---|---|---|---|
| Strong | Always最新数据 | Slowest | Financial transactions |
| Eventual | Eventually consistent | Fastest | Social media feeds |
| Causal | Causally related ops consistent | Moderate | Collaborative editing |
| Linearizable | Total ordering of operations | Very slow | Distributed locks |
-- PostgreSQL: READ COMMITTED (eventual for concurrent transactions)
BEGIN;
SELECT balance FROM accounts WHERE id = 1; -- reads 1000
-- Another transaction updates to 800 and commits
SELECT balance FROM accounts WHERE id = 1; -- reads 800 (new read)
COMMIT;
-- PostgreSQL: REPEATABLE READ (consistent snapshot)
BEGIN ISOLATION LEVEL REPEATABLE READ;
SELECT balance FROM accounts WHERE id = 1; -- reads 1000
-- Another transaction updates to 800 and commits
SELECT balance FROM accounts WHERE id = 1; -- still reads 1000
COMMIT;
-- PostgreSQL: SERIALIZABLE (strongest)
BEGIN ISOLATION LEVEL SERIALIZABLE;
-- Detects read/write conflicts and rolls back one transaction
-- Prevents all anomalies but reduces concurrency
COMMIT;PostgreSQL offers multiple consistency levels within a single database. Choose based on your consistency requirements per transaction, not globally. Use the weakest level that satisfies your correctness requirements.
PACELC Extension
PACELC extends CAP by considering normal operation (not just partitions). It asks: during a partition, choose between Availability and Consistency (CAP). During normal operation, choose between Latency and Consistency (ELC).
| System | Partition: A or C | Normal: L or C | Classification |
|---|---|---|---|
| PostgreSQL | C (synchronous) | C (ACID) | PC/EC |
| Cassandra | A (always writable) | L (fast writes) | PA/EL |
| MongoDB | C (majority writes) | C (majority reads) | PC/EC |
| DynamoDB | A (always writable) | L (low latency) | PA/EL |
Choosing Your Position
# Choose CP (strong consistency) when:
cp_requirements:
- Financial transactions (money cannot be duplicated)
- Inventory management (overselling is unacceptable)
- Configuration management (stale config causes errors)
- Leader election (split-brain is catastrophic)
# Choose AP (high availability) when:
ap_requirements:
- Social media feeds (slightly stale is acceptable)
- Shopping cart (conflict resolution on merge)
- IoT sensor data (latest reading matters, not history)
- DNS (availability is more important than consistency)
# Choose CA (single node) when:
ca_requirements:
- Small datasets (< 1TB)
- Single-server deployment
- No distribution requirements
- Simple operationsMost applications start as CA (single PostgreSQL). As they grow, they add read replicas (still mostly CP) and caching (AP for freshness). The choice evolves with your scale and requirements.
Consensus Protocols
Consensus protocols solve the coordination problem in distributed systems. They ensure all nodes agree on a single value even during failures. Raft and Paxos are the two dominant protocols.
| Protocol | Used By | Key Feature |
|---|---|---|
| Raft | etcd, CockroachDB | Understandable, leader-based |
| Paxos | Google Spanner, ZooKeeper | Complex, flexible |
| Paxos Made Moderately Complex | Self-hosted systems | Educational |
Consensus protocols require majority quorum (3 of 5, 5 of 7 nodes). Every write must be replicated to a majority before committing. This adds latency. Choose the minimum cluster size that meets your availability requirements.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.