Multi-Window Burn-Rate Alerts

Implementing fast and slow burn alerts with Prometheus rules from Google SRE workbook patterns.

8 min readObservability & Monitoring StackOperate

Burn Rate Concept

Burn rate measures how fast you are consuming your error budget. A burn rate of 1x means you are consuming budget at exactly the rate your SLO allows. A burn rate of 10x means you will exhaust your budget 10x faster than planned.

Burn rate calculation


# Burn rate = actual error rate / allowed error rate
# For 99.9% SLO (0.1% allowed errors)
sum(rate(http_requests_total{status=~"5.."}[1h]))
  / (0.001 * sum(rate(http_requests_total[1h])))

Burn Rate Calculation

The burn rate is calculated over a time window. A 1-hour window detects fast burns. A 6-hour window detects slow burns. The Google SRE workbook recommends using two windows: a fast window (short) and a slow window (long) to catch both types of burns.

Burn rate over different windows


# Fast burn (1h window)
sum(rate(http_requests_total{status=~"5.."}[1h]))
  / (0.001 * sum(rate(http_requests_total[1h])))

# Slow burn (6h window)
sum(rate(http_requests_total{status=~"5.."}[6h]))
  / (0.001 * sum(rate(http_requests_total[6h])))

Multi-Window Alerts

Multi-window burn-rate alerts combine a short burn window with a long burn window. The short window provides fast detection. The long window reduces false positives by confirming the burn rate is sustained.

Multi-window alert structure


# Fast burn alert: 1h burn rate > 14.4x for 5m
# Slow burn alert: 6h burn rate > 1x for 30m
# Together these catch errors that would exhaust the
# 30-day error budget in ~2 days

Burn Rate Table

The burn rate table maps burn rates to time windows. Each row tells you which windows to use for a given burn rate and error budget consumption target.

Burn Rate	Budget Consumed In	Short Window	Long Window	Query Interval
14.4x	2 days	1h	5m	1 minute
6x	5 days	6h	30m	5 minutes
3x	10 days	1d	2h	15 minutes
1x	30 days	3d	6h	1 hour

Prometheus Alert Rules

Multi-window burn-rate alerts


groups:
  - name: slo_burn_rate
    rules:
      # Fast burn: 14.4x for 1h window, 5m alert window
      - alert: SLOBurnRateHigh
        expr: |
          (
            sum(rate(http_requests_total{status=~"5.."}[1h]))
            / (0.001 * sum(rate(http_requests_total[1h])))
          ) > 14.4
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "High burn rate detected"
          description: "Error budget will be exhausted in less than 2 days"

      # Slow burn: 1x for 6h window, 30m alert window
      - alert: SLOBurnRateLow
        expr: |
          (
            sum(rate(http_requests_total{status=~"5.."}[6h]))
            / (0.001 * sum(rate(http_requests_total[6h])))
          ) > 1
        for: 30m
        labels:
          severity: warning
        annotations:
          summary: "Sustained burn rate above target"
          description: "Error budget is being consumed faster than planned"

Tuning Burn Rate Alerts

Start with the SRE workbook defaults and adjust based on false positive rates.
Increase the burn rate threshold to reduce false positives.
Decrease the alert window to detect faster.
Use different burn rates for different severity levels.
Test with historical data before deploying to production.

Use recording rules for burn rate queries

Burn rate calculations are complex and expensive. Precompute them with recording rules and alert on the precomputed values. This improves alert latency and reduces Prometheus load.

Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.

Stage 6 · Operate

SLOs & Production Observability