Stage 4 · Provision
Observability-Driven Design
Dashboard Design
RED and USE methods, golden signals, and building dashboards that tell you what's wrong.
Purpose of Dashboards
Dashboards provide visual context for understanding system behavior. They are not for looking pretty — they are for diagnosing problems quickly. A good dashboard answers the question: Is the system healthy, and if not, what is wrong?
The Four Golden Signals
Google's SRE book defines four golden signals that every service dashboard should monitor. These signals capture the essential health of any service regardless of its specific function.
| Signal | What It Measures | Example Metric |
|---|---|---|
| Latency | Time to serve a request | request_duration_seconds p99 |
| Traffic | Demand on the system | requests_per_second |
| Errors | Rate of failed requests | error_rate percentage |
| Saturation | How full the system is | cpu_usage, memory_usage |
RED Method
RED is designed for request-driven services. It focuses on the three metrics that matter for any service handling requests: Rate, Errors, and Duration. This is the most common dashboard pattern for microservices.
# Rate: Requests per second
rate(http_requests_total[5m])
# Errors: Error percentage
rate(http_requests_total{status=~"5.."}[5m])
/ rate(http_requests_total[5m]) * 100
# Duration: 99th percentile latency
histogram_quantile(0.99,
rate(http_request_duration_seconds_bucket[5m]))These three metrics tell you almost everything about service health. If all three are normal, the service is healthy.
USE Method
USE is designed for infrastructure and resource-centric dashboards. It focuses on Utilization, Saturation, and Errors for each resource. Use this for databases, servers, and network equipment.
| Resource | Utilization | Saturation | Errors |
|---|---|---|---|
| CPU | cpu_usage_percent | run_queue_length | hardware_errors |
| Memory | memory_used / total | swap_usage | oom_kills |
| Disk | disk_used / total | io_queue_length | io_errors |
| Network | bandwidth_used / max | packet_queue | dropped_packets |
Dashboard Anti-Patterns
- Too many panels — If everything is highlighted, nothing is highlighted.
- Vanity metrics — CPU usage looks busy but does not indicate problems.
- Missing context — Graphs without thresholds or SLO lines are hard to interpret.
- Static dashboards — Dashboards that are never updated become stale and misleading.
- One dashboard for everything — Different audiences need different views.
Building Effective Dashboards
- Start with the four golden signals or RED method.
- Add SLO threshold lines for context.
- Group related metrics together.
- Use consistent time ranges across panels.
- Version control your dashboards as code (Grafana JSON, Terraform).
The best dashboards are built during incidents when you know exactly what information you needed. After every incident, ask: What dashboard would have helped us diagnose this faster?
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.