546 Commits

Author	SHA1	Message	Date
NamanParlecha	594f9d63a5	refactor(textparse): Introduce Variadic options in textParse.New (#17155) ... * refactor(textparse): introduce ParserOptions struct for cleaner parser initialization Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(fuzz): update fuzzParseMetricWithContentType to use ParserOptions Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): simplify ParserOptions usage in tests and implementations Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parse): using variadic options Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): add fallbackType & SymbolTable to variadic options Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): private fields Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(scrape): compose parser options Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): add comments Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): update to use ParserOptions struct for configuration Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(scrape): remove unused parserOptions field from scrapeLoop Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): update ParserOptions field names and add comments for clarity Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> --------- Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com>	2025-09-11 10:49:42 +01:00
George Krajcsovits	acd9aa0afb	fix(textparse/protobuf): metric family name corrupted by NHCB parser (#17156) ... * fix(textparse): implement NHCB parsing in ProtoBuf parser directly The NHCB conversion does some validation, but we can only return error from Parser.Next() not Parser.Histogram(). So the conversion needs to happen in Next(). There are 2 cases: 1. "always_scrape_classic_histograms" is enabled, in which case we convert after returning the classic series. This is to be consistent with the PromParser text parser, which collects NHCB while spitting out classic series; then returns the NHCB. 2. "always_scrape_classic_histograms" is disabled. In which case we never return the classic series. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * refactor(textparse): skip classic series instead of adding NHCB around Do not return the first classic series from the EntryType state, switch to EntrySeries. This means we need to start the histogram field state from -3 , not -2. In EntrySeries, skip classic series if needed. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * reuse nhcb converter Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * test(textparse/nhcb): test corrupting metric family name NHCB parse doesn't always copy the metric name from the underlying parser. When called via HELP, UNIT, the string is directly referenced which means that the read-ahead of NHCB can corrupt it. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2025-09-08 17:26:41 +02:00
Arve Knudsen	913cc8f72b	Replace gopkg.in/yaml.v2 with go.yaml.in/yaml/v2 (#17151) ... * Replace gopkg.in/yaml.v2 with go.yaml.in/yaml/v2 * Upgrade to client_golang@v1.23.2 --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2025-09-06 13:04:24 +02:00
George Krajcsovits	31e4d84edd	refactor(textparse): allow for parsers with direct NHCB support (#17153) ... Hide adding NHCB parser on top another parser in New() function so we can easily add direct NHCB capable parsers. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2025-09-06 11:45:44 +02:00
bwplotka	172cde8af1	Revert "feat(storage): add new CombinedAppender interface and compatibility layer" ... This reverts commit 2fb680a229.	2025-08-29 08:16:39 +01:00
bwplotka	2fb680a229	feat(storage): add new CombinedAppender interface and compatibility layer ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2025-08-29 08:14:34 +01:00
beorn7	747c5ee2b1	Apply analyzer "modernize" to the whole codebase ... See https://pkg.go.dev/golang.org/x/tools/gopls/internal/analysis/modernize for details. This ran into a few issues (arguably bugs in the modernize tool), which I will fix in the next commit, so that we have transparency what was done automatically. Beyond those hiccups, I believe all the changes applied are legitimate. Even where there might be no tangible direct gain, I would argue it's still better to use the "modern" way to avoid micro discussions in tiny style PRs later. Signed-off-by: beorn7 <beorn@grafana.com>	2025-08-27 14:48:41 +02:00
Julius Hinze	77b5c3f217	Histograms: set annotation when adding or subtracting histograms that have `not_reset` and `reset` hints. ... Signed-off-by: Julius Hinze <julius.hinze@grafana.com>	2025-08-20 15:00:45 +02:00
Arve Knudsen	0a40df33fb	Make metric/label name validation scheme explicit (#16928) ... * Parameterize metric/label name validation scheme Parameterized metric/label name validation scheme --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Julius Hinze <julius.hinze@grafana.com>	2025-08-18 08:09:00 +00:00
Julius Hinze	5855d973b0	model: set native histogram GaugeType hint when subtracting or multiplying/dividing with negative factors ... Signed-off-by: Julius Hinze <julius.hinze@grafana.com>	2025-08-12 18:16:39 +02:00
George Krajcsovits	929bd787ec	fix(ci): run linter with all build tags (#17027) ... Fix up lint errors that were not previously checked. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2025-08-08 09:43:41 +00:00
Matthieu MOREL	cef219c31c	chore: enable unused-receiver rule from revive ... Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2025-08-04 09:43:33 +00:00
Bryan Boreham	d9e5748a27	[REFACTOR] Scraping: Remove unnecessary yolostring calls (#16927) ... Go will not allocate when reading from a map with a key cast from []byte to string. Also remove some yoloString calls in package `textparse` - call a more suitable library function. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2025-07-28 14:54:32 +01:00
Bryan Boreham	6b37b42f6f	Merge pull request #16616 from prymitive/stringLabels ... Rename labels.go -> labels_slicelabels.go	2025-07-17 09:28:29 +01:00
Vlad Shulcz	19fa1ed008	test(rulefmt): fix description annotation index in TestParseFileSuccessWithAliases (#16839) ... Signed-off-by: shulcz <vshulcz@gmail.com>	2025-07-08 18:38:34 +02:00
Lukasz Mierzwa	559fd44be6	Rename labels.go -> labels_slicelabels.go ... labels.go is now holding slicelabels code, so let's rename it. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-07-07 12:37:42 +01:00
Jon Kartago Lamida	819500bdbc	Add ByteSize method for Labels (#16717) ... Add `ByteSize()` method to different labels implementations. One of the use case so that we can track the memory used by Labels. Signed-off-by: Jon Kartago Lamida <me@lamida.net>	2025-07-04 15:09:01 +01:00
Bryan Boreham	4eafbcae93	lint ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2025-07-02 09:56:28 +01:00
Bryan Boreham	e7ac3f440d	[TESTS] Labels: Add a test for SizeOfLabels ... This requires a bit of repetition to cover all the different builds, but it seems worth checking that the function does what is expected. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2025-07-02 09:31:27 +01:00
Bryan Boreham	507227781b	[REFACTOR] Labels: Extract test case data from TestLabels_String ... So we can use them in other tests. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2025-07-02 09:31:25 +01:00
Carrie Edwards	7289d41940	Add tests for relabeling of type and unit labels (#16743) ... Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>	2025-06-23 16:42:38 +01:00
György Krajcsovits	85d63ba861	refactor(nhcb): get rid of extra lastHistogramExponential state ... and make it a regular state of inhibiting NHCB conversion. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2025-06-11 20:06:38 +02:00
György Krajcsovits	3844a21738	fix(nhcb): do not loose NHCB after exponential with different labels ... The problem happens when we parse a standalone native histogram, which sets the p.lastHistogramExponential state flag. We never unset it. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2025-06-11 19:48:56 +02:00
machine424	50a6efd5ec	fix(model/textparse): Labels(): copy the input to avoid dangling references ... Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2025-05-23 11:03:48 +02:00
György Krajcsovits	772d5ab433	Merge branch 'main' into krajo/intern-custom-values	2025-05-20 08:23:15 +02:00
Bartlomiej Plotka	8e6b008608	feature: type-and-unit-labels (PROM-39 implementation) (#16228) ... * feature: type-and-unit-labels (extended MetricIdentity) Experimental implementation of https://github.com/prometheus/proposals/pull/39 Previous (unmerged) experiments: * https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels * https://github.com/prometheus/prometheus/pull/16025 Signed-off-by: bwplotka <bwplotka@gmail.com> feature: type-and-unit-labels (extended MetricIdentity) Experimental implementation of https://github.com/prometheus/proposals/pull/39 Previous (unmerged) experiments: * https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels * https://github.com/prometheus/prometheus/pull/16025 Signed-off-by: bwplotka <bwplotka@gmail.com> * Fix compilation errors Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Lint Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Revert change made to protobuf 'Accept' header Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Fix compilation errors for 'dedupelabels' tag Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Rectored into schema.Metadata Signed-off-by: bwplotka <bwplotka@gmail.com> * texparse: Added tests for PromParse Signed-off-by: bwplotka <bwplotka@gmail.com> * add OM tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * add proto tests Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * add schema label tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * fix tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * add promql tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * lint Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>	2025-05-17 09:37:25 +00:00
hardlydearly	ba4b058b7a	refactor: use slices.Contains to simplify code ... Signed-off-by: hardlydearly <799511800@qq.com>	2025-05-09 08:27:10 +02:00
György Krajcsovits	6c646657d5	perf(chunkenc): intern the custom values for native histograms ... The custom values are the "le" bucket boundaries of native histograms with custom buckets. They are never modified. It is ok to not copy them when iterating a chunk, just reference them. If we will ever have a function that modifies the custom values, like 'trim' for example. That function will have to make a copy on write. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2025-05-07 14:40:45 +02:00
Arve Knudsen	e7e3ab2824	Fix linting issues found by golangci-lint v2.0.2 (#16368) ... * Fix linting issues found by golangci-lint v2.0.2 --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2025-05-03 19:05:13 +02:00
Bryan Boreham	ca416c580c	Merge branch 'main' into slicelabels ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2025-05-02 10:31:57 +01:00
Bryan Boreham	b2c2146d7c	Labels: simpler/faster stringlabels encoding (#16069) ... Instead of using varint to encode the size of each label, use a single byte for size 0-254, or a flag value of 255 followed by the size in 3 bytes little-endian. This reduces the amount of code, and also the number of branches in commonly-executed code, so it runs faster. The maximum allowed label name or value length is now 2^24 or 16MB. Memory used by labels changes as follows: * Labels from 0 to 127 bytes length: same * From 128 to 254: 1 byte less * From 255 to 16383: 2 bytes more * From 16384 to 2MB: 1 byte more * From 2MB to 16MB: same Labels: panic on string too long. Slightly more user-friendly than encoding bad data and finding out when we decode. Clarify that Labels.Bytes() encoding can change --------- Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2025-04-30 10:53:48 +01:00
Lukasz Mierzwa	05088aaa12	Fix linter errors ... Mostly comment issues and unused variables. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-04-15 18:04:41 +01:00
Lukasz Mierzwa	bb76966992	Use stringlabels by default ... This removes the stringlabels build tag, makes that implementation the default one, and moves the old labels implementation under the slicelabels build tag. Fixes #16064. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-04-15 17:52:24 +01:00
wellweek	4e91f13db2	refactor: use slices.Equal to simplify code ... Signed-off-by: wellweek <xiezitai@outlook.com>	2025-03-27 12:17:35 +01:00
Rafal Rosa	c7ea28c36b	parser: remove expired link from comment (#16189) ... * Replace OpenMetrics link with the current specification --------- Signed-off-by: pomyslowynick <dark0mlet@gmail.com> Signed-off-by: Rafal Rosa <dark0mlet@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>	2025-03-23 12:41:48 +01:00
Matthieu MOREL	5fa1146e21	chore: enable gci linter (#16245) ... Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2025-03-22 15:46:13 +00:00
Alexander Weaver	3d603d1957	chore: Export rulefmt.ruleNode (#16252) ... * Export rulefmt.ruleNode Signed-off-by: Alex Weaver <weaver.alex.d@gmail.com> * Export RuleGroupNode for consistency Signed-off-by: Alex Weaver <weaver.alex.d@gmail.com> --------- Signed-off-by: Alex Weaver <weaver.alex.d@gmail.com>	2025-03-20 22:58:32 +00:00
Owen Williams	94b43c5d4c	utf8: Remove support for legacy global validation setting ... Global and Data Source configurations can specify legacy mode, but Prometheus now requires that the overall validation mode be set to UTF-8 Signed-off-by: Owen Williams <owen.williams@grafana.com>	2025-03-13 10:47:24 -04:00
Bartlomiej Plotka	dc85d677d8	textparse: Optimize CreatedTimestamp; It returns int64 value now. (#16072) ... ..instead of *int64. This is as an optimization and ease of use. We already accepted in many places (proto histograms, PRW) that CT (or any timestamp really) 0 means not set. Signed-off-by: bwplotka <bwplotka@gmail.com>	2025-03-07 12:43:13 +00:00
Matthieu MOREL	c7d4b53ec1	chore: enable unused-parameter from revive ... Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2025-02-19 19:50:28 +01:00
Bartlomiej Plotka	733a5e9eb4	textparse: Optimized protobuf parser with custom streaming unmarshal. (#15731) ... * textparse: Optimized protobuf parser with custom streaming decoder. Signed-off-by: bwplotka <bwplotka@gmail.com> Update model/textparse/protobufparse.go Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Addressing comments. Signed-off-by: bwplotka <bwplotka@gmail.com> decoder: reuse histograms and summaries. Signed-off-by: bwplotka <bwplotka@gmail.com> optimize help returning (5% of mem utilization). Signed-off-by: bwplotka <bwplotka@gmail.com> Apply suggestions from code review Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Update prompb/io/prometheus/client/decoder.go Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Fix build. Signed-off-by: bwplotka <bwplotka@gmail.com> * Update model/textparse/protobufparse.go Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	2025-02-13 10:38:35 +00:00
frazou	9b4c8f6be2	rulefmt: support YAML aliases for Alert/Record/Expr (#14957) ... * rulefmt: add tests with YAML aliases for Alert/Record/Expr Altough somewhat discouraged in favour of using proper configuration management tools to generate full YAML, it can still be useful in some situations to use YAML anchors/aliases in rules. The current implementation is however confusing: aliases will work everywhere except on the alert/record name and expr This first commit adds (failing) tests to illustrate the issue, the next one fixes it. The YAML test file is intentionally filled with anchors and aliases. Although this is probably not representative of a real-world use case (which would have less of them), it errs on the safer side. Signed-off-by: François HORTA <fhorta@scaleway.com> * rulefmt: support YAML aliases for Alert/Record/Expr This fixes the use of YAML aliases in alert/recording rule names and expressions. A side effect of this change is that the RuleNode YAML type is no longer propagated deeper in the codebase, instead the generic Rule type can now be used. Signed-off-by: François HORTA <fhorta@scaleway.com> * rulefmt: Add test for YAML merge combined with aliases Currently this does work, but adding a test for the related functionally here makes sense. Signed-off-by: David Leadbeater <dgl@dgl.cx> * rulefmt: Rebase to latest changes Signed-off-by: David Leadbeater <dgl@dgl.cx> --------- Signed-off-by: François HORTA <fhorta@scaleway.com> Signed-off-by: David Leadbeater <dgl@dgl.cx> Co-authored-by: David Leadbeater <dgl@dgl.cx>	2025-02-13 20:48:33 +11:00
Bartlomiej Plotka	00b69efabb	model/textparse: Change parser interface Metric(...) string to Labels(...) (#16012) ... * model/textparse: Change parser interface Metric(...) string to Labels(...) Simplified the interface given no one is using the return argument. Renamed for clarity too. Found and discussed https://github.com/prometheus/prometheus/pull/15731#discussion_r1950916842 Signed-off-by: bwplotka <bwplotka@gmail.com> * Fixed comments; optimized not needed copy for om and text. Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com>	2025-02-12 15:47:56 +00:00
Bartlomiej Plotka	8cd9069cf1	textparse: Refactor benchmark testdata for all types. (#15998) ... Also: * split benchmark functions to make sure no one compares across parsers. * testdata file have meaningful names reflecting the type representation * promtestdata.txt now has all types, taken directly from long running Prometheus (https://demo.do.prometheus.io/) Needed for https://github.com/prometheus/prometheus/pull/15731 Signed-off-by: bwplotka <bwplotka@gmail.com>	2025-02-10 12:26:18 +00:00
Bartlomiej Plotka	7427753922	scrape: Add realistic data case for scrape loop append bench. (#15966) ... * scrape: Add realistic data case for scrape loop append bench. Signed-off-by: bwplotka <bwplotka@gmail.com> * Update scrape/scrape_test.go Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	2025-02-03 14:46:39 +00:00
bwplotka	36cf85fc1e	Addressed comments. ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2025-01-27 09:54:13 +00:00
Bartlomiej Plotka	bd0d9e7a0a	Update model/rulefmt/rulefmt.go ... Co-authored-by: Owen Williams <owen.williams@grafana.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	2025-01-24 15:49:10 +00:00
bwplotka	80d702afdc	Fixed rulefmt UTF-8 expectations. ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2025-01-23 09:23:58 +00:00
bwplotka	7263dfe50e	Fixed relabelling; allowing UTF-8 in targetLabel. ... Signed-off-by: bwplotka <bwplotka@gmail.com>	2025-01-23 08:53:59 +00:00
Giedrius Statkevičius	92218ecb9b	promtool: add --ignore-unknown-fields ... Add --ignore-unknown-fields that ignores unknown fields in rule group files. There are lots of tools in the ecosystem that "like" to extend the rule group file structure but they are currently unreadable by promtool if there's anything extra. The purpose of this flag is so that we could use the "vanilla" promtool instead of rolling our own. Some examples of tools/code: https://github.com/grafana/mimir/blob/main/pkg/mimirtool/rules/rwrulefmt/rulefmt.go 8898eb3cc5/pkg/rules/rules.go (L18-L25) Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	2025-01-15 11:34:28 +02:00