* Add anchored and smoothed to vector selectors.
This adds "anchored" and "smoothed" keywords that can be used following a matrix selector.
"Anchored" selects the last point before the range (or the first one after the range) and adds it at the boundary of the matrix selector.
"Smoothed" applies linear interpolation at the edges using the points around the edges. In the absence of a point before or after the edge, the first or the last point is added to the edge, without interpolation.
*Exemple usage*
* `increase(caddy_http_requests_total[5m] anchored)` (equivalent of *caddy_http_requests_total - caddy_http_requests_total offset 5m* but takes counter reset into consideration)
* `rate(caddy_http_requests_total[step()] smoothed)`
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* Update docs/feature_flags.md
Co-authored-by: Charles Korn <charleskorn@users.noreply.github.com>
Signed-off-by: Julien <291750+roidelapluie@users.noreply.github.com>
* Smoothed/Anchored rate: Add more tests
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
* Anchored/Smoothed modifier: error out with histograms
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
---------
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
Signed-off-by: Julien <291750+roidelapluie@users.noreply.github.com>
Co-authored-by: Charles Korn <charleskorn@users.noreply.github.com>
So far, we emitted a `HistogramCounterResetCollisionWarning` when
encountering conflicting counter resets in the calculation of (i)rate
and friends. We even tested for that. However, in the rate
calculation, we are not interested in those collisions. They are
actually expected.
On the other hand, we did not warn about those collisions when doing a
`sum` aggregation, where such a warning would be appropriate.
This commit removes the warning in the former case and adds it in the
latter. Sadly, we cannot really test this as we still remove the
counter reset hint for the first sample in a chunk. (And that's the
only sample where we could get a `NotCounterReset` hint.)
Signed-off-by: beorn7 <beorn@grafana.com>
Add a first_over_time function, and corresponding ts_of_first_over_time
function. Both are behind the experimental functions feature flag.
Signed-off-by: Craig Ringer <craig.ringer@enterprisedb.com>
See
https://pkg.go.dev/golang.org/x/tools/gopls/internal/analysis/modernize
for details.
This ran into a few issues (arguably bugs in the modernize tool),
which I will fix in the next commit, so that we have transparency what
was done automatically.
Beyond those hiccups, I believe all the changes applied are
legitimate. Even where there might be no tangible direct gain, I would
argue it's still better to use the "modern" way to avoid micro
discussions in tiny style PRs later.
Signed-off-by: beorn7 <beorn@grafana.com>
This means we only do it once, rather than on every step of a range
query. Also the code gets a bit shorter.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Currently, the promql functions take the interface slice []parser.Value as an argument,
which is being implemented by the conrete types Vector, Matrix etc. This PR replaces
the interface with the concrete types, resulting in improved performance.
The inspiration for this PR came from #16698 which does this for binops.
I extended the idea to all promql functions
Signed-off-by: darshanime <deathbullet@gmail.com>
* pass single Matrix
Signed-off-by: darshanime <deathbullet@gmail.com>
---------
Signed-off-by: darshanime <deathbullet@gmail.com>
Observed on go1.24.4 darwin/arm64, the compensation variable is not correctly calculated.
Note you may have to run the tests several times to see it, to get the right ordering of series.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
CI / Report status of build Prometheus for all architectures (push) Has been cancelledDetails
CI / Publish main branch artifacts (push) Has been cancelledDetails
CI / Publish release artefacts (push) Has been cancelledDetails
CI / Publish UI on npm Registry (push) Has been cancelledDetails
This deals with the count field of native histograms in the same way
as with simple float counters. It then scale the whole histogram with
the same factor as it has scaled the count. This will still allow
individual buckets to get extrapolated below zero, but maybe that is
fine.
This implements approach (2) as described in
https://github.com/prometheus/prometheus/issues/15976#issuecomment-3032095158
Signed-off-by: beorn7 <beorn@grafana.com>
This commit brings back direct mean calculation (for `avg` and
`avg_over_time`) but isn't an outright revert of #16569. It keeps the
improved incremental mean calculation and features generally a bit
cleaner code than before.
Also, this commit...
- ...updates the lengthy comment explaining the whole situation and
trade-offs.
- ...divides the running sum and the Kahan compensation term
separately (in direct mean calculation) to avoid the (unlikely)
possibility that sum and Kahan compensation together ovorflow
float64.
- ...uncomments the tests that should now work again on darwin/arm64.
- ...uncomments the test that should now reliably yield the
(inaccurate) value 0 on all hardware platforms. Also, the test
description has been updated accordingly.
- ...adds avg_over_time tests for zero and one sample in the range.
Signed-off-by: beorn7 <beorn@grafana.com>
CI / Publish main branch artifacts (push) Blocked by required conditionsDetails
CI / Publish release artefacts (push) Blocked by required conditionsDetails
CI / Publish UI on npm Registry (push) Blocked by required conditionsDetails
Scorecards supply-chain security / Scorecards analysis (push) Waiting to runDetails
* fix(promql): histogram_quantile NaN observed in native histogram
Fixes: #16578
See the issue for detailed explanation.
When a histogram had only NaN observations and no normal observations,
we returned 0 from the quantile, which is completely wrong. If there were
normal observations but we went over them, we returned the upper bound of
the existing buckets, however that contradicts expectations on
histogram_fraction. Now we return NaN if the quantile is calculated to be
over all normal observations, falling into NaNs (in a virtual +Inf bucket).
We also return info level annotations if we see any NaN observations.
The annotation calls out if we returned NaN or even if we took the
virtual +Inf bucket into account.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
* fix(promql): histogram_fraction NaN observed in native histogram
Fixes: #16580
According to the specification we should not take NaN observations
into account when calculating the fraction. This commit fixes that
and adds an info level annotation to let the user know about this.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
This PR fixes a bug in ts_of_last_over_time where the float samples
where used when computing the last timestamp of the histogram samples.
Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com>
This commit adds the ts_of_(max,min,last)_over_time functions behind the experimental feature flag.
Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com>
As it turns out, if we combine Kahan summation and incremental mean
calculation properly, it works quite well and we do not need to switch
between simple mean calculation and incremental calculation based on
overflow.
This simplifies the code quite a bit.
Signed-off-by: beorn7 <beorn@grafana.com>
* promql: histogram_fraction for bucket histograms
This PR extends the histogram_fraction function to also work with classic bucket histograms. This is beneficial because it allows expressions like sum(increase(my_bucket{le="0.5"}[10m]))/sum(increase(my_total[10m])) to be written without knowing the actual values of the "le" label, easing the transition to native histograms later on.
It also feels natural since histogram_quantile also can deal with classic histograms.
Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com>
* promql: histogram_fraction for bucket histograms
* Add documentation and reduce code duplication
* Fix a bug in linear interpolation between bucket boundaries
* Add more PromQL tests
Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com>
* Update docs/querying/functions.md
Co-authored-by: Björn Rabenstein <github@rabenste.in>
Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
---------
Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com>
Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
Co-authored-by: Björn Rabenstein <github@rabenste.in>
promql: return NaN from `irate()` if either of last two samples is NaN
Signed-off-by: Charles Korn <charles.korn@grafana.com>
---------
Signed-off-by: Charles Korn <charles.korn@grafana.com>
Fix duplicate output vector if delayed name removal is disabled
This error is emitted in cleanupMetricLabels, but is skipped if
enableDelayedNameRemoval is false.
This makes it consistent with label_replace
Signed-off-by: Joshua Hesketh <josh@nitrotech.org>
---------
Signed-off-by: Joshua Hesketh <josh@nitrotech.org>
Signed-off-by: Björn Rabenstein <github@rabenste.in>
Co-authored-by: Björn Rabenstein <github@rabenste.in>
If a rate (or increase) is calculated on native histograms, and there
is a counter reset between the 1st and 2nd histogram, we never have to
touch the 1st histogram, so it doesn't even matter if it has an
incompatible bucket layout. So we should not error out in that case.
This simply nulls out the 1st histogram in that case.
Signed-off-by: beorn7 <beorn@grafana.com>
Ensure metric name is present on histogram_quantile annotations
Previously the series __name__ label was dropped before the annotation
was emitted causing the annotation message to have a blank value.
This fix also allows delayed name removal for classic histograms.
This also adds a test for malformed le label
This annotation could be split into two to be clearer for the user
(one for missing, and one for malformed). This is outside of the scope
of this change.
Fixes: #15411
---------
Signed-off-by: Joshua Hesketh <josh@nitrotech.org>
Signed-off-by: Björn Rabenstein <github@rabenste.in>
Co-authored-by: Björn Rabenstein <github@rabenste.in>
`funcDoubleExponentialSmoothing` did not get its doc comment updated
when we renamed it from the confusing `funcHoltWinters`.
Signed-off-by: beorn7 <beorn@grafana.com>
Julien
Björn Rabenstein
Craig Ringer
Julius Hinze
Arve Knudsen
Neeraj Gartia
Bryan Boreham
Matthieu MOREL
Darshan Chaudhary
George Krajcsovits
🌲 Harry 🌊 John 🏔
Michael Hoffmann
Bartlomiej Plotka
Charles Korn
Aman
Michael Hoffmann
Charles Korn
Joshua Hesketh
Neeraj Gartia