Defining SLIs

What Are SLIs?

Service Level Indicators are quantitative measures of service behavior from the user's perspective. They answer one question: how well is the service performing for actual users? SLIs are not infrastructure metrics — they are user-centric measurements.

A well-chosen SLI captures something your users care about. Response time for a web page, error rate for an API, or data freshness for a streaming pipeline. If users would notice a degradation in the SLI, it is the right indicator.

Latency SLI

Latency measures how long it takes to serve a successful request. It is the most common SLI for web services. The key is to measure at the user-facing endpoint and only for successful responses — errors should be counted separately under availability.

# Record successful requests with their duration
- record: sli:http_request_duration_seconds:success
  expr: |
    histogram_quantile(0.99,
      sum(rate(http_request_duration_seconds_bucket{
        code!~"5..",
        handler="/api/users"
      }[5m])) by (le)
    )

Always use percentiles, not averages. The 99th percentile captures tail latency — the experience of your unluckiest 1% of users. Averaging hides slow requests behind a sea of fast ones.

Availability SLI

Availability measures the proportion of requests that succeed. A request is successful if it returns the expected response within the latency SLI threshold. Availability is not uptime — a service can be up but returning errors.

# Availability = successful requests / total requests
- record: sli:http_availability:ratio
  expr: |
    sum(rate(http_request_duration_seconds_count{
      code!~"5..",
      handler="/api/users"
    }[5m]))
    /
    sum(rate(http_request_duration_seconds_count{
      handler="/api/users"
    }[5m]))

Quality SLI

Quality measures correctness of responses. Does the search result actually match the query? Does the recommendation engine return relevant suggestions? Quality SLIs require domain-specific definitions.

# Quality = queries returning non-empty results
- record: sli:search_quality:ratio
  expr: |
    sum(rate(search_queries_total{
      results_count>0
    }[5m]))
    /
    sum(rate(search_queries_total[5m]))

Throughput SLI

Throughput measures the rate at which the service successfully processes requests. It is critical for batch processing, data pipelines, and streaming systems where volume matters more than individual request latency.

# Events processed per second
- record: sli:pipeline_throughput:rate
  expr: |
    sum(rate(events_processed_total[5m]))

Choosing the Right SLIs

1. Identify user journeys — what paths do users take through your service?
2. Select 2-3 SLIs per service — latency, availability, and quality cover most cases.
3. Measure from the user's perspective — instrument at the edge, not inside the service.
4. Keep SLIs simple — if an SLI is too complex to understand, it is too complex to act on.
5. Align SLIs with business value — the SLI should reflect something the business cares about.

SLIs in Prometheus

The standard pattern for SLI implementation in Prometheus is a recording rule that produces a ratio. The numerator is good events, the denominator is total events. This ratio feeds directly into SLO calculations.

groups:
  - name: sli_recording_rules
    interval: 30s
    rules:
      # Latency SLI - p99 for successful requests
      - record: sli:http_latency_seconds:p99
        expr: |
          histogram_quantile(0.99,
            sum(rate(http_request_duration_seconds_bucket{
              code!~"5.."
            }[5m])) by (le)
          )

      # Availability SLI - ratio of non-5xx requests
      - record: sli:http_availability:ratio
        expr: |
          sum(rate(http_request_duration_seconds_count{
            code!~"5.."
          }[5m]))
          /
          clamp_min(sum(rate(http_request_duration_seconds_count[5m])), 1)

      # Quality SLI - success rate for downstream calls
      - record: sli:downstream_success:ratio
        expr: |
          sum(rate(downstream_call_total{
            code="200"
          }[5m]))
          /
          clamp_min(sum(rate(downstream_call_total[5m])), 1)