b8d19543b8
ReduceResolution is currently called before validation during ingestion. This will cause a panic if there are not enough buckets in the histogram. If there are too many buckets, the spurious buckets are ignored, and therefore the error in the input histogram is masked. Furthermore, invalid negative offsets might cause problems, too. Therefore, we need to do some minimal validation in reduceResolution. Fortunately, it is easy and shouldn't slow things down. Sadly, it requires to return errors, which triggers a bunch of code changes. Even here is a bright side, we can get rud of a few panics. (Remember: Don't panic!) In different news, we haven't done a full validation of histograms read via remote-read. This is not so much a security concern (as you can throw off Prometheus easily by feeding it bogus data via remote-read) but more that remote-read sources might be makeshift and could accidentally create invalid histograms. We really don't want to panic in that case. So this commit does not only add a check of the spans and buckets as needed for resolution reduction but also a full validation during remote-read. Signed-off-by: beorn7 <beorn@grafana.com> |
||
|---|---|---|
| .. | ||
| agent | ||
| chunkenc | ||
| chunks | ||
| docs | ||
| encoding | ||
| errors | ||
| fileutil | ||
| goversion | ||
| index | ||
| record | ||
| testdata | ||
| tombstones | ||
| tsdbutil | ||
| wlog | ||
| .gitignore | ||
| CHANGELOG.md | ||
| README.md | ||
| block.go | ||
| block_test.go | ||
| blockwriter.go | ||
| blockwriter_test.go | ||
| compact.go | ||
| compact_test.go | ||
| db.go | ||
| db_test.go | ||
| example_test.go | ||
| exemplar.go | ||
| exemplar_test.go | ||
| head.go | ||
| head_append.go | ||
| head_bench_test.go | ||
| head_dedupelabels.go | ||
| head_other.go | ||
| head_read.go | ||
| head_read_test.go | ||
| head_test.go | ||
| head_wal.go | ||
| isolation.go | ||
| isolation_test.go | ||
| mocks_test.go | ||
| ooo_head.go | ||
| ooo_head_read.go | ||
| ooo_head_read_test.go | ||
| ooo_head_test.go | ||
| ooo_isolation.go | ||
| ooo_isolation_test.go | ||
| querier.go | ||
| querier_bench_test.go | ||
| querier_test.go | ||
| repair.go | ||
| repair_test.go | ||
| testutil.go | ||
| tsdbblockutil.go | ||
README.md
TSDB
This directory contains the Prometheus TSDB (Time Series DataBase) library, which handles storage and querying of all Prometheus v2 data.
Documentation
External resources
- A writeup of the original design can be found here.
- Video: Storing 16 Bytes at Scale from PromCon 2017.
- Compression is based on the Gorilla TSDB white paper.
A series of blog posts explaining different components of TSDB: