Cardinality: The #1 Reason Prometheus Bills Explode
Every unique combination of label values on a metric creates a separate time series. Multiply the cardinalities of every label, then multiply by the number of replicas emitting that metric — that's your total series count for the metric. Get this wrong and a single metric can produce millions of series, crashing Prometheus or racking up thousands per month on managed observability platforms.
Labels That Should NEVER Exist on Metrics
user_id— unbounded, grows with your user countrequest_id/trace_id— unique per request, infinite cardinalitysession_id,order_id,email— same problemip/client_ip— millions of possible values- Raw URL
pathwith IDs — use route templates instead (e.g./users/:id) - Timestamps as labels — never
Safe Cardinality Targets
| Per metric | Status | Action |
|---|---|---|
| < 1,000 | Healthy | None |
| 1K-10K | Watch | Note in dashboards |
| 10K-100K | Risk | Audit labels |
| 100K-1M | High | Drop high-card labels via relabel_config |
| > 1M | Critical | Drop the metric or relabel immediately |
Finding Your Worst Offenders
Run this PromQL in your environment to find the highest-cardinality metrics:
topk(20, count by(__name__) ({__name__=~".+"})) To find the worst label per metric:
topk(20, count by(__name__, label_name) ({__name__=~".+"})) See the Metrics & Observability Guide for the full optimization playbook.