Grouping and Deduplication

Alert Grouping

Alertmanager groups related alerts into a single notification. Grouping prevents alert storms from flooding channels. When multiple alerts match the same group_by labels, Alertmanager batches them into one message.

route:
  receiver: "slack-default"
  group_by: ["alertname", "job"]
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 4h

Group Timing Parameters

Three parameters control group timing. group_wait is the initial wait before sending the first notification. group_interval is the wait between notifications for the same group. repeat_interval is how often to resend a notification for a still-firing group.

group_by Best Practices

Choose group_by labels that group related alerts together. Grouping by alertname and job creates one group per service. Grouping by alertname alone groups all instances of the same alert type. The choice affects notification granularity.

# One notification per alert type across all jobs
group_by: ["alertname"]

# One notification per job for each alert
group_by: ["alertname", "job"]

# One notification per service and severity
group_by: ["service", "severity"]

# Do not group — each alert gets its own notification
group_by: []

HA Deduplication

When running Alertmanager in high-availability mode with multiple replicas, deduplication prevents duplicate notifications. Alertmanager uses the fagnerng protocol to synchronize state and deduplicate alerts that fire on multiple replicas.

# Prometheus sends to all Alertmanager replicas
alerting:
  alertmanagers:
    - static_configs:
        - targets:
            - "alertmanager-0:9093"
            - "alertmanager-1:9093"

# Alertmanager uses gossip for HA
alertmanager:
  cluster:
    peers:
      - "alertmanager-0:9094"
      - "alertmanager-1:9094"

Repeat Interval

The repeat interval controls how often Alertmanager re-sends notifications for alerts that remain firing. After the initial group_wait and subsequent group_interval notifications, repeat_interval determines the reminder frequency.

route:
  receiver: "default"
  repeat_interval: 4h

  routes:
    - match:
        severity: critical
      receiver: "pagerduty-critical"
      repeat_interval: 1h    # Page every hour if still firing

    - match:
        severity: warning
      receiver: "slack-warnings"
      repeat_interval: 12h   # Remind once per half day