Stage 4 · Provision
Designing for High Availability
Availability Fundamentals
SLAs, SLOs, SLIs — the language of reliability that every SRE must speak fluently.
Availability Math
Availability is the fraction of time a system is operational and reachable. It is expressed as a percentage and calculated by dividing uptime by total time. The difference between 99.9% and 99.99% availability is the difference between 8.76 hours and 52.6 minutes of downtime per year.
| Availability | Downtime/Year | Downtime/Month | Use Case |
|---|---|---|---|
| 99% | 3.65 days | 7.31 hours | Internal tools |
| 99.9% | 8.76 hours | 43.8 minutes | Standard SaaS |
| 99.99% | 52.6 minutes | 4.38 minutes | Financial systems |
| 99.999% | 5.26 minutes | 26.3 seconds | Telecom/medical |
A system can be available but unreliable — returning incorrect data or timing out. Reliability measures correctness; availability measures reachability. You need both.
SLIs, SLOs, and SLAs
Service Level Indicators (SLIs) are the raw measurements of your system's behavior. Service Level Objectives (SLOs) are the target values for those indicators. Service Level Agreements (SLAs) are the contractual commitments backed by financial penalties.
service: payment-gateway
slo:
- indicator: availability
target: 99.95%
measurement_window: 30_days
error_budget: 21.6_minutes
- indicator: latency_p99
target: 200ms
measurement_window: 30_days
error_budget: 5%_of_requestsSLIs are measurable, SLOs are aspirational targets, and SLAs are legally binding. An SLO should be stricter than your SLA to create a buffer.
Error Budgets
An error budget is the inverse of your SLO. If your SLO is 99.95% availability, your error budget is 0.05% — the amount of allowable downtime. Error budgets turn reliability into a shared resource between engineering and product teams.
- When the error budget is healthy, product teams can ship faster with less polish.
- When the error budget is consumed, feature freezes are enforced to prioritize reliability.
- Error budgets create alignment — everyone shares the same reliability incentive.
- Burn rate measures how fast you are consuming the error budget.
Some teams try to inflate SLIs or relax measurement windows to preserve their budget. This defeats the purpose. Error budgets exist to drive honest conversations about reliability tradeoffs.
The Nines
Each additional nine of availability is exponentially harder and more expensive to achieve. Going from 99.9% to 99.99% typically costs 10x more in redundancy, monitoring, and operational overhead. The decision of how many nines to target should be driven by business requirements, not engineering pride.
Failure Domains
A failure domain is a portion of your infrastructure that can fail independently. The goal of HA design is to minimize the blast radius of any single failure. Common failure domain boundaries include: racks, availability zones, regions, and clouds.
Cloud Provider (external)
└── Region A
├── Availability Zone 1 (AZ)
│ ├── Rack 1
│ │ ├── Server 1
│ │ └── Server 2
│ └── Rack 2
└── Availability Zone 2 (AZ)
├── Rack 3
└── Rack 4Design so that a failure in one AZ does not affect services in another AZ. Cross-region failover protects against regional disasters.
Designing for Availability
High availability is not a feature you add at the end. It is a property that emerges from deliberate architectural decisions made at the beginning. Eliminate single points of failure, use redundancy across failure domains, and design for graceful degradation.
- No single point of failure — every critical path has at least one redundant component.
- No correlated failures — redundancy across independent failure domains.
- Fast detection — monitoring and alerting that catch issues in seconds.
- Fast recovery — automated failover and self-healing mechanisms.
Define your SLOs first. They dictate your architecture. A 99% system and a 99.99% system look completely different in terms of redundancy, deployment strategy, and operational investment.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.