516 Commits

Author	SHA1	Message	Date
Ben Kochie	204249fcb5	Update golangci-lint (#17478) ... * Update golangci-lint to v2.6.0 * Fixup various linting issues. * Fixup deprecations. * Add exception for `labels.MetricName` deprecation. Signed-off-by: SuperQ <superq@gmail.com>	2025-11-05 13:47:34 +01:00
George Krajcsovits	d7bfc89f7a	Merge pull request #17431 from grafana/thampiotr/upstream-staleness-disabling ... scrape: Allow disabling end-of-run staleness markers for targets	2025-11-04 13:24:05 +01:00
Piotr	d6848c9f40	scrape: Allow disabling end-of-run staleness markers for targets ... Signed-off-by: Piotr <17101802+thampiotr@users.noreply.github.com>	2025-11-04 11:59:23 +00:00
Ben Kochie	48956f60d7	Update modernize (#17471) ... Apply additional Go modernize tool improvements. Signed-off-by: SuperQ <superq@gmail.com>	2025-11-04 05:13:49 +00:00
Julius Volz	0093e2159e	Merge pull request #17337 from prometheus/ui/visualize-relabel-steps ... ui: Allow viewing detailed relabeling steps for each discovered target	2025-11-02 13:51:55 +01:00
Lukasz Mierzwa	aac472df5b	Fix TestScrapeLoop_HistogramBucketLimit ... TestScrapeLoop_HistogramBucketLimit tests the bucket limiter but it also sets sample_limit to the same value, which seems incorrect. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-10-24 10:04:18 +01:00
Lukasz Mierzwa	ce520b9256	Include histograms in sample_limit logic ... Currently histograms bypass sample_limit logic as the limitAppender only implements the Append() method, while histograms are appended using AppendHistogram. This means that they are effectively ignored during sample_limit checks and a scrape with sample_limit=100 and 500 histograms will accept all samples. Add AppendHistogram method to the limitAppender so histograms are also counted towards sample_limit. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-10-24 10:04:18 +01:00
Julius Volz	8b1bd7d6c3	ui: Allow viewing detailed relabeling steps for each discovered target ... This adds: * A `ScrapePoolConfig()` method to the scrape manager that allows getting the scrape config for a given pool. * An API endpoint at `/api/v1/targets/relabel_steps` that takes a pool name and a label set of a target and returns a detailed list of applied relabeling rules and their output for each step. * A "show relabeling" link/button for each target on the discovery page that shows the detailed flow of all relabeling rules (based on the API response) for that target. Note that this changes the JSON encoding of the relabeling rule config struct to output the original snake_case (instead of camelCase) field names, and before merging, we need to be sure that's ok :) See my comment about that at https://github.com/prometheus/prometheus/pull/15383#issuecomment-3405591487 Fixes https://github.com/prometheus/prometheus/issues/17283 Signed-off-by: Julius Volz <julius.volz@gmail.com>	2025-10-15 15:33:27 +02:00
beorn7	ad7d1aed99	Phase out native histogram feature flag ... The detailed plan for this is laid out in https://github.com/prometheus/prometheus/issues/16572 . This commit adds a global and local scrape config option `scrape_native_histograms`, which has to be set to true to ingest native histograms. To ease the transition, the feature flag is changed to simply set the default of `scrape_native_histograms` to true. Further implications: - The default scrape protocols now depend on the `scrape_native_histograms` setting. - Everywhere else, histograms are now "on by default". Documentation beyond the one for the feature flag and the scrape config are deliberately left out. See https://github.com/prometheus/prometheus/pull/17232 for that. Signed-off-by: beorn7 <beorn@grafana.com>	2025-10-15 14:50:52 +02:00
beorn7	72960c076d	scrape: "Modernize" test files ... Applied the analyzer "modernize" to the test files. $ go run golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize@latest -fix -test ./... Signed-off-by: beorn7 <beorn@grafana.com>	2025-10-14 19:30:22 +02:00
Harsh	4d7d8ebcfa	lint fixes ... Signed-off-by: Harsh <harshmastic@gmail.com>	2025-10-13 13:59:10 +05:30
harsh kumar	a612cd8954	Update scrape/manager_test.go ... Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: harsh kumar <135993950+hxrshxz@users.noreply.github.com>	2025-10-13 12:41:46 +05:30
harsh kumar	5244b0e3f7	Update scrape/manager_test.go ... Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: harsh kumar <135993950+hxrshxz@users.noreply.github.com>	2025-10-13 00:50:58 +05:30
harsh kumar	a1e163a462	Update scrape/manager_test.go ... Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: harsh kumar <135993950+hxrshxz@users.noreply.github.com>	2025-10-13 00:50:42 +05:30
Harsh	a63414b8e8	suggested changes added ... Signed-off-by: Harsh <harshmastic@gmail.com>	2025-10-12 21:29:18 +05:30
Harsh	3a7a8d7547	promql: Enhance TestNHCBAndCTZeroIngestion to validate exemplar parsing with created timestamps ... Signed-off-by: Harsh <harshmastic@gmail.com>	2025-10-11 17:27:47 +05:30
harsh kumar	2b2cbec8cd	Merge branch 'prometheus:main' into fix-17224-remove-nhcb-check	2025-10-11 17:13:22 +05:30
Ali Nazari	f64ee61312	Use rlock in read methods of scrape target ... Signed-off-by: Ali Nazari <ali4nazari4@gmail.com>	2025-10-08 09:06:09 +03:30
Harsh	febd734137	test: Add TestNHCBAndCTZeroIngestion to verify simultaneous feature usage ... Signed-off-by: Harsh <harshmastic@gmail.com>	2025-10-08 09:07:24 +05:30
Harsh	c6793e717a	Remove obsolete check ... Signed-off-by: Harsh <harshmastic@gmail.com>	2025-10-08 08:41:43 +05:30
György Krajcsovits	bdf547ae9c	fix(nativehistograms): validation should fail on unsupported schemas ... Histogram.Validate and FloatHistogram.Validate now return error on unsupported schemas. Scrape and remote-write handler reduces the schema to the maximum allowed if it is above the maximum, but below theoretical maximum of 52. For scrape the maximum is a configuration option, for remote-write it is 8. Note: OTLP endpont already does the reduction, without checking that it is below 52 as the spec does not specify a maximum. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2025-09-13 16:54:44 +02:00
NamanParlecha	594f9d63a5	refactor(textparse): Introduce Variadic options in textParse.New (#17155) ... * refactor(textparse): introduce ParserOptions struct for cleaner parser initialization Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(fuzz): update fuzzParseMetricWithContentType to use ParserOptions Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): simplify ParserOptions usage in tests and implementations Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parse): using variadic options Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): add fallbackType & SymbolTable to variadic options Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): private fields Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(scrape): compose parser options Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): add comments Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): update to use ParserOptions struct for configuration Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(scrape): remove unused parserOptions field from scrapeLoop Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> * refactor(parser): update ParserOptions field names and add comments for clarity Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com> --------- Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com>	2025-09-11 10:49:42 +01:00
Arve Knudsen	913cc8f72b	Replace gopkg.in/yaml.v2 with go.yaml.in/yaml/v2 (#17151) ... * Replace gopkg.in/yaml.v2 with go.yaml.in/yaml/v2 * Upgrade to client_golang@v1.23.2 --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2025-09-06 13:04:24 +02:00
George Krajcsovits	31e4d84edd	refactor(textparse): allow for parsers with direct NHCB support (#17153) ... Hide adding NHCB parser on top another parser in New() function so we can easily add direct NHCB capable parsers. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2025-09-06 11:45:44 +02:00
Owen Williams	6ee965c255	common: Update to prom/common v0.66.0, fix TextParser creation (#17139) ... TextParser as of prom/common v0.66.0 requires an explicit validation scheme. Signed-off-by: Owen Williams <owen.williams@grafana.com>	2025-09-03 11:20:04 -04:00
George Krajcsovits	d09db02854	fix(nhcb): flaky test TestConvertClassicHistogramsToNHCB (#17112) ... * fix(nhcb): flaky test TestConvertClassicHistogramsToNHCB The test was e2e, including actually scraping an HTTP endpoint and running the scrape loop. This led to some timing issues. I've simplified it to call the scrape loop append directly. I think that this isn't nice as that is a private interface, but should gets rid of the flakiness and there's already a bunch of test doing this. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2025-09-02 13:37:14 +02:00
Bryan Boreham	70bf09cb2b	Merge pull request #16429 from prymitive/scrapeCacheStaleNaN ... Append staleness markers only for known series	2025-09-02 10:41:07 +01:00
beorn7	747c5ee2b1	Apply analyzer "modernize" to the whole codebase ... See https://pkg.go.dev/golang.org/x/tools/gopls/internal/analysis/modernize for details. This ran into a few issues (arguably bugs in the modernize tool), which I will fix in the next commit, so that we have transparency what was done automatically. Beyond those hiccups, I believe all the changes applied are legitimate. Even where there might be no tangible direct gain, I would argue it's still better to use the "modern" way to avoid micro discussions in tiny style PRs later. Signed-off-by: beorn7 <beorn@grafana.com>	2025-08-27 14:48:41 +02:00
Arve Knudsen	0a40df33fb	Make metric/label name validation scheme explicit (#16928) ... * Parameterize metric/label name validation scheme Parameterized metric/label name validation scheme --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Julius Hinze <julius.hinze@grafana.com>	2025-08-18 08:09:00 +00:00
pipiland2612	1607b5c8cc	parallelize ./scrape test ... Signed-off-by: pipiland2612 <nguyen.t.dang.minh@gmail.com>	2025-08-12 14:12:48 +02:00
Matthieu MOREL	cef219c31c	chore: enable unused-receiver rule from revive ... Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2025-08-04 09:43:33 +00:00
Bryan Boreham	d9e5748a27	[REFACTOR] Scraping: Remove unnecessary yolostring calls (#16927) ... Go will not allocate when reading from a map with a key cast from []byte to string. Also remove some yoloString calls in package `textparse` - call a more suitable library function. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2025-07-28 14:54:32 +01:00
socialsister	869c946370	chore: fix some minor issues in comments ... Signed-off-by: socialsister <seekseat@qq.com>	2025-07-16 11:24:42 +01:00
vanshika	0fc5e75504	scraping: Create a span and send the traceparent header during scrape requests (#16425) ... * Traceparent header Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> (cherry picked from commit 44a620dd73) Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> * changes Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> (cherry picked from commit 6e98a77b2d) Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> * adding test Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> (cherry picked from commit 97f288ad87) Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> * more changes Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> (cherry picked from commit d5dd861544) Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> * extract http client creation to newScrapeClient Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> (cherry picked from commit 3cd8092b155df069d02d9409b6327fe60c788bec) Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> * rebase Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> * rebase Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> * reverting Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> * ctx Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> --------- Signed-off-by: Vanshikav123 <vanshikav928@gmail.com>	2025-07-15 09:37:24 +02:00
Yandi Lee	8eb445b8a4	Discovery.Manager: close sync ch after sender() is stopped (#14465) ... * close sync ch after sender() is stopped * break if chan is closed Signed-off-by: liyandi <littlepangdi@163.com> Co-authored-by: liyandi <liyandi@xiaomi.com>	2025-07-11 17:15:01 +01:00
George Krajcsovits	dae80e66a5	test(scrape): make nhcb scrape test more tolerant (#16862) ... Fixes #16689 well, maybe not 100%, but should improve it. Increase the scrape timeout to be more tolerant of slow test and also use eventually when checking for targets. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2025-07-11 12:09:53 +02:00
Arve Knudsen	5a5424cbc1	Consolidate around prometheus/common/model.ValidationScheme (#16806) ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2025-07-03 15:37:46 +02:00
Lukasz Mierzwa	bb690a23b9	Make sure we never call trackStaleness with nil cache entry ... Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-07-01 14:22:01 +01:00
Lukasz Mierzwa	6687bf5653	Only add series to scrape cache if they were appended to TSDB ... Scrape cache is used to emit StaleNaN markers after a series disappears so it should only hold entries for series that did end up in TSDB, which is not always the case due to sample_limit. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-07-01 14:22:01 +01:00
Lukasz Mierzwa	c75768739a	Sort series by labels in requireEqual() ... Tests that look at samples with StaleNaN values will fail because these samples are generated from map iteration and so the order can be unstable. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-07-01 14:22:01 +01:00
Lukasz Mierzwa	e2193f634f	Add a test for StaleNaNs after hitting sample_limit ... I was confused why there are no StaleNaN markers appended when a scrape hits sample_limit, but reading the code I see that's expected, so add a test for it. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-07-01 14:22:01 +01:00
Lukasz Mierzwa	0eedc046f4	Check ref value when appending staleness markers ... Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-07-01 14:22:01 +01:00
Lukasz Mierzwa	872f03766c	Pass last know ref ID when injecting staleness markers ... Currently all staleness markers are appended for any sample that disappears from scrape cache, even if that sample was never appended to TSDB. When staleness markers are appended they always use ref=0 as the SeriesRef, so the downstream appender doesn't know if the sample is for a know series or not. This changes the scrape cache so the map used for staleness tracking stores the cache entry instead of only the label set. Having the cache entry means: - we can ignore stale samples that didn't end up in TSDB (not in the scrape cache) - we can append them to TSDB using correct ref value, so the appender knows if they are for know or unknown series Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-07-01 14:22:01 +01:00
Lukasz Mierzwa	1f7a23cced	Add tests for staleness markers appended to TSDB when sample_limit is set ... Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-07-01 14:22:01 +01:00
Łukasz Mierzwa	748fe6d825	Limit concurrency of scrape pool reloads (#16783) ... To avoid possible overload. As per https://github.com/prometheus/prometheus/pull/16595#issuecomment-3005027067 this changes scrape pool manager to limit the number of scrape pools that can reload at the same time. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-06-27 12:34:07 +01:00
Owen Williams	aa1d46a9da	scrape: set validation and escaping defaults in default config vars (#16751) ... Fixes https://github.com/prometheus/prometheus/issues/16750 Signed-off-by: Owen Williams <owen.williams@grafana.com>	2025-06-25 11:14:13 -04:00
Carrie Edwards	7289d41940	Add tests for relabeling of type and unit labels (#16743) ... Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>	2025-06-23 16:42:38 +01:00
Łukasz Mierzwa	c528293376	Enhancement: Reload all scrape pools concurrently (#16595) ... * Reload all scrape pools concurrently At the moment all scrape pools that need to be reloaded are reloaded one by one. While reloads are ongoing mtxScrape is locked. For each pool that's being reloaded we need to wait until all targets are updated. This whole process can take a while and the more scrape pools to reload the longer. At the same time all pools are independent and there's no real reason to do them one-by-one. Reload each pool in a seperate goroutine so we finish config reload as ASAP as possible and unlock the mtxScrape. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * Address PR review feedback Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> --------- Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2025-06-09 16:01:35 +02:00
Bartlomiej Plotka	8e6b008608	feature: type-and-unit-labels (PROM-39 implementation) (#16228) ... * feature: type-and-unit-labels (extended MetricIdentity) Experimental implementation of https://github.com/prometheus/proposals/pull/39 Previous (unmerged) experiments: * https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels * https://github.com/prometheus/prometheus/pull/16025 Signed-off-by: bwplotka <bwplotka@gmail.com> feature: type-and-unit-labels (extended MetricIdentity) Experimental implementation of https://github.com/prometheus/proposals/pull/39 Previous (unmerged) experiments: * https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels * https://github.com/prometheus/prometheus/pull/16025 Signed-off-by: bwplotka <bwplotka@gmail.com> * Fix compilation errors Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Lint Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Revert change made to protobuf 'Accept' header Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Fix compilation errors for 'dedupelabels' tag Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Rectored into schema.Metadata Signed-off-by: bwplotka <bwplotka@gmail.com> * texparse: Added tests for PromParse Signed-off-by: bwplotka <bwplotka@gmail.com> * add OM tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * add proto tests Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * add schema label tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * fix tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * add promql tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * lint Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>	2025-05-17 09:37:25 +00:00
George Krajcsovits	2726a18ae3	Merge pull request #16452 from chardch/global-always-scrape-classic-hist ... Add global config option for always_scrape_classic_histograms	2025-05-06 09:29:30 +02:00