Long-Term Storage

Why Long-Term Storage?

Prometheus's local storage is not designed for long retention. It stores data on local disk and has no built-in replication. For retention beyond weeks or months, you need remote storage. Long-term storage enables capacity planning, historical analysis, and compliance.

Remote Write

Prometheus remote_write sends samples to an external storage system in real time. The remote system stores the data and handles long-term retention. This is the foundation for all long-term storage solutions.

remote_write:
  - url: "http://mimir:9009/api/v1/push"
    queue_config:
      capacity: 10000
      max_shards: 200
      batch_send_deadline: 5s
    write_relabel_configs:
      - source_labels: [__name__]
        regex: "debug_.*"
        action: drop

Thanos

Thanos extends Prometheus with long-term storage, global query, and downsampling. It runs as a sidecar alongside Prometheus, uploading blocks to object storage. A Store Gateway serves historical data. Query Frontend provides caching and query splitting.

thanos:
  sidecar:
    object_storage:
      type: S3
      config:
        bucket: thanos-metrics
        endpoint: s3.us-east-1.amazonaws.com
    prometheus_url: http://prometheus:9090
    tsdb_path: /prometheus

Mimir

Grafana Mimir is a horizontally scalable, highly available Prometheus long-term storage solution. It accepts remote_write requests, stores data in object storage, and provides a PromQL-compatible query API. Mimir is the recommended solution for Grafana Cloud.

# Mimir components
distributor:    # Receives remote_write, distributes to ingesters
ingester:       # Writes to long-term storage blocks
compactor:      # Merges and deduplicates blocks
store_gateway:  # Serves historical data from object storage
querier:        # Executes PromQL queries
query_frontend: # Caches and splits queries

Downsampling

Downsampling reduces data resolution over time. Raw data at full resolution is kept for recent periods. Older data is aggregated to 5-minute or 1-hour resolution. This dramatically reduces storage costs while preserving trend visibility.

Object Storage

All long-term storage solutions use object storage (S3, GCS, Azure Blob) as the backing store. Object storage is durable, cheap, and scales infinitely. The tradeoff is higher latency for individual reads, which is mitigated by caching and compaction.