PromQL Cheat Sheet

Selecting series

Select latest sample for series with a given metric name:

my_metric_name

Select 5-minute range of samples for series with a given metric name:

my_metric_name[5m]

Only series with given label values:

my_metric_name{label1="value1", label2="value2"}

Complex label matchers:

my_metric_name{label1!="value1", label2=~"regex1", label3!~"regex2"}
  • =: Equality
  • !=: Non-equality
  • =~: Regex match
  • !~: Negative regex match

Rates of increase for counters

Per-second rate of increase, averaged over last 5 minutes:

rate(http_requests_total[5m])

Per-second rate of increase, calculated over last two samples in a 1-minute time window:

irate(http_requests_total[1m]

Absolute increase over last hour:

increase(http_requests_total[1h])

Aggregating over multiple series

Sum over all series:

sum(my_metric_name)

Preserve the method and path label dimensions:

sum by(method, path) (my_metric_name)

Aggregate away the method and path label dimensions:

sum without(method, path) (my_metric_name)

Available aggregation operators: sum(), min(), max(), avg(), stddev(), stdvar(), count(), count_values(),bottomk(), topk(), quantile()

Math between series

Add all equally-labelled series from both sides:

left_metric + right_metric

Add series, matching only on the instance and job labels:

left_metric + on(instance, job) right_metric

Add series, ignoring the instance and job labels for matching:

left_metric + ignoring(instance, job) right_metric

Explicitly allow many-to-one matching:

left_metric + on(instance, job) group_left right_metric

Include the version label from "one" (right) side in the result:

left_metric + on(instance, job) group_left(version) right_metric

Available arithmetic operators: +, -, *, /, %, ^

Set operations

Include any label sets that are either on the left or right side:

left_metric or right_metric

Include any label sets that are present both on the left and right side:

left_metric and right_metric

Include any label sets from the left side that are not present in the right side:

left_metric unless right_metric

Match only on some labels:

left_metric and on(job, instance) right_metric

Filtering series by value

Only keep series with a sample value greater than 1024:

left_metric > 1024

Only keep series from the left-hand side whose sample values are larger than their right-hand-side matches:

left_metric > right_metric

Instead of filtering, return 0 or 1 for each compared series:

left_metric > bool right_metric

Match only on some labels:

left_metric > on(job, instance) right_metric

Available comparison operators: ==, !=, >, <, >=,<=

Quantiles from histograms

90th percentile request latency over last 5 minutes, for every label dimension:

histogram_quantile(0.9, rate(my_request_durations_seconds_bucket[5m]))

...for only the path and method dimensions:

histogram_quantile(
  0.9,
  sum by(le, path, method) (
    rate(my_request_durations_seconds_bucket[5m])
  )
)

Changes in gauges

Per-second derivative using linear regression:

deriv(my_gauge[5m])

Absolute change in value over last hour:

delta(my_gauge[1h])

Predict value in 1 hour, based on last 4 hours:

predict_linear(my_gauge[4h], 3600)

Time

Get the Unix time in seconds at each resolution step:

time()

Get the age of the last successful batch job run:

time() - my_batch_job_last_success_timestamp_seconds

Find batch jobs which haven't succeeded in an hour:

time() - my_batch_job_last_success_timestamp_seconds > 3600

Dealing with missing data

Create one output series when the input vector is empty:

absent(my_metric_name)

Create one output series when the input range vector is empty for 5 minutes:

absent_over_time(my_metric_name[5m])

Manipulating labels

Join the values of two labels with a - separator:

label_join(my_metric_name, "-", "label1", "label2")

Extract part of a label and store it in a new label:

label_replace(my_metric, "dest-label", "$1", "source-label", "(.*):.*")

More...

For more details about PromQL, see the official PromQL documentation: