Stage 4 · Provision
Scalability & Capacity
Cost Performance Tradeoffs
Rightsizing, reserved capacity, spot instances, and latency SLOs in cloud capacity plans.
Cost-Aware Architecture
Cost is a first-class architectural constraint. Every design decision — instance type, database choice, caching strategy — has a cost implication. Cost-aware architecture optimizes for the right balance of performance, reliability, and spend.
Rightsizing
Rightsizing matches instance types to actual workload requirements. Most over-provisioned instances run at 10-30% utilization. Rightsizing to the correct instance type can save 40-60% with no performance impact.
Current: m5.4xlarge (16 vCPU, 64GB RAM) - $0.768/hr
CPU utilization: 15%
Memory utilization: 20%
Rightsizing options:
m5.xlarge (4 vCPU, 16GB RAM) - $0.192/hr
CPU: 60% (still has headroom)
Memory: 80% (comfortable)
Savings: 75%
m5.2xlarge (8 vCPU, 32GB RAM) - $0.384/hr
CPU: 30% (more headroom)
Memory: 40% (comfortable)
Savings: 50%
Recommendation: m5.xlarge
Saves $488/month per instance
No performance impactUse cloud provider tools (AWS Compute Optimizer, GCP Recommender) to identify over-provisioned instances. Rightsize conservatively — leave headroom for spikes.
Reserved Capacity
| Commitment | Discount | Flexibility |
|---|---|---|
| On-demand | None | Full |
| 1-year reserved | 30-40% | Moderate |
| 3-year reserved | 50-60% | Low |
| 1-year savings plan | 25-35% | High |
| Spot instances | 60-90% | Low (can be interrupted) |
Spot Instances
Spot instances are unused cloud capacity sold at 60-90% discount. The cloud provider can reclaim them with 2 minutes notice. Use spot instances for fault-tolerant workloads: batch processing, CI/CD, containerized services with graceful shutdown.
apiVersion: apps/v1
kind: Deployment
metadata:
name: batch-worker
spec:
replicas: 10
template:
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: node.kubernetes.io/capacity-type
operator: In
values: ["spot"]
tolerations:
- key: "spot"
operator: "Equal"
value: "true"
effect: "NoSchedule"
terminationGracePeriodSeconds: 30Batch workers are ideal for spot instances. They are fault-tolerant, can be restarted, and benefit from the 60-90% discount. Use Pod Disruption Budgets to handle spot interruptions.
Latency SLOs and Cost
Every 10ms reduction in latency SLO costs more. Going from 100ms to 50ms may double the required instances. Going from 50ms to 10ms may quadruple them. Choose latency SLOs that match user expectations, not engineering ambition.
Target: p99 latency < X ms
p99 < 200ms: 10 instances ($1,000/month)
p99 < 100ms: 20 instances ($2,000/month)
p99 < 50ms: 40 instances ($4,000/month)
p99 < 20ms: 100 instances ($10,000/month)
The last 20ms costs 10x more than the first 180ms.
Is that worth it? Depends on the business.Latency improvements have diminishing returns and increasing costs. Choose the latency SLO that provides adequate user experience at reasonable cost.
Cost Optimization Strategies
- Reserved capacity for baseline — Cover 60-70% of capacity with reserved.
- On-demand for variability — Use on-demand for remaining 30-40%.
- Spot for batch — Use spot instances for non-critical workloads.
- Auto-scaling — Scale down during quiet periods.
- Graviton/ARM — 20% cheaper with similar performance.
Schedule monthly cost reviews with each team. Show per-service costs. Identify waste. Track cost per request. Make cost optimization a regular practice, not a one-time project.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.