7be1fd953a
fix(unified-storage): enable in-process events for single-instance ( #100807 )
2025-02-21 12:25:35 +01:00
b4c4b9abbd
Remove embedded WriteEvent from WrittenEvent struct, to avoid having unnecessary fields. ( #101081 )
2025-02-20 17:34:25 +01:00
b1222be02e
unistore: add small buffer of watched events ( #100431 )
...
* change log level
* Add a small buffer when watching events
2025-02-14 12:18:59 +02:00
a5355fd66c
Storage: Add command line tool to migrate legacy dashboards (and folders) to unified storage ( #99199 )
2025-02-11 19:57:46 +02:00
3ba0d8d4b5
Unified Storage: Improve observability for indexing latency ( #99700 )
...
* adds extra debug logs and a new metric for poller query latency to help us better understand watch and index latency for write events
* adds trace span to the index for handling index write events
2025-01-28 12:30:20 -06:00
0cef2b9ae7
Dashboard Versions: Make compatible with app platform ( #99327 )
2025-01-28 08:17:52 -06:00
c4c934e0bd
SQL/Storage: Remove SkipDataMigration flag ( #99404 )
2025-01-23 12:34:48 +02:00
9d30911107
SQL/Storage: Use xorm code migration for DeletionMarker ( #99353 )
2025-01-22 14:07:08 +03:00
356b32008b
Storage: Show history+trash using the list command ( #99009 )
...
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-01-17 15:54:25 +03:00
8f6e9f8ed0
Restores in app platform ( #97582 )
2024-12-13 16:55:43 -06:00
d762a96436
Unified Storage: Init at startup, fix traces, and speed up indexing ( #97529 )
...
* dont lazy init unified storage
* Inits index when creating new resource server. Fixes trace propagation by passing span ctx. Update some logging.
* Use finer grained cache locking when building indexes to speed things up. Locking the whole function was slowing things down.
* formatting
* linter fix
* go mod
* make update-workspace
* fix workspaces check error
* update dependency owner in mod file
* wait 1 second before querying metrics
* try with big timeout, see if fixes CI. Wont fail locally.
* skips postgres integration test. Only fails in drone. Will fix later.
* put delay back to 500 ms
2024-12-09 22:32:19 -06:00
8165258a2d
Storage: Query stats within a namespace ( #97403 )
2024-12-05 13:58:13 +03:00
9d89d8757f
Search: Build index from resource stats ( #97320 )
2024-12-03 20:20:27 +03:00
f2b96593ea
SQL: close rows to release connection ( #97147 )
...
* SQL: close rows to release connection
* dont return err from rows.Close()
2024-11-28 14:28:55 +01:00
c3571752b6
Unitstore: Fix the Folder migration ( #96078 )
...
* UniStore: Add folder to the SQL backend
This reverts commit aaa4f4a23e
2024-11-12 12:52:04 +01:00
b6596db75e
Unified Storage Indexer: Build tenant indexes concurrently ( #95795 )
...
* WIP - build tenant indexes concurrently
* adds local dev seeders and readme for indexer
* update logging and adds locking in getShard()
* update logs
* Adds Namespaces func. Initializes index after ResourceServer is created.
* fixes Count() and updates test lint issues
* Initialize index separately. Don't do it when creating the resource server. Makes testing really awkward.
* fix lint error
* handles error when getting namespaces
* updates test and index helper funcs
2024-11-06 12:58:07 -06:00
aaa4f4a23e
Revert "UniStore: Add folder to the SQL backend" ( #95966 )
...
Revert "UniStore: Add folder to the SQL backend (#95671 )"
This reverts commit f075662696
2024-11-06 17:04:57 +02:00
f075662696
UniStore: Add folder to the SQL backend ( #95671 )
...
* Add folder to the SQL backend
* remove unused key
* fix tests
* Return folders from watch
2024-11-05 10:37:23 +01:00
ecebaf1bdf
Unified Storage: Add traces to poller ( #95369 )
...
add traces to poller
2024-10-24 22:49:38 +03:00
257aca5340
Unistore Chore: Fix span name ( #94965 )
...
fix span name
2024-10-24 12:12:20 -03:00
3e1f5559a6
Unistore Chore: Add database-level observability ( #92266 )
...
* add testing harness
* fix mockery and linters
* WIP
* wip
* fix transactions
* fix transaction tracing; add tracing by default
* rename package
* move WithTx to simplify logic of DB implementations
* fix potential issue with context deadline
* add db instrumentation to dbutil
* add otel tests
* improve naming
* minor fix in semantics and add comprehensive OTel testing
* fix naming
* instrument resourceVersionAtomicInc
* provide a default testing tracer
* fix docs
* fix typo in docs
* add semconv for k8s
2024-10-18 00:32:08 -03:00
cf08f6762d
Unistore Chore: Add OTEL testing harness ( #94835 )
...
* add testing harness
* fix mockery and linters
* fix data race in tests
* fix data race in tests
* reduce cardinality of data
2024-10-17 08:41:06 -03:00
d999b415df
UniStore: Use epoch with microsecond resolution as RV ( #92638 )
...
* Use epoch with microsecond resolution as RV
* fix backend tests
* Add solution for when the clock goes back
* Add solution for when the clock goes back
* generate mocks
* go lint
* remove comment
* Use Greatest instead of max in msyql and postgres
* update tests
* Update pkg/storage/unified/sql/sqltemplate/dialect_sqlite.go
Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>
* cast to bigint
* add additional round trip
* increment the RV using 2 sql round trips instead of 3
* cleanup comments
* cast unix timestamp to integer
* fix postgres query
* remove old increment test data
* remove greatest
* cast unix_timestamp to signed
* Use statement_timestamp instead of clock_timestamp
---------
Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>
2024-10-11 12:11:33 +03:00
4a800eda9f
Unistore Chore: add inocuous preparative changes for otel-based db observability ( #94473 )
...
add inocuous preparative changes for otel tracing
2024-10-09 20:32:09 +00:00
03f55e5288
Unistore : Ensure Watch works in HA mode ( #94120 )
...
* Revert "Revert "Unistore : Ensure Watch works in HA mode." (#94097 )"
This reverts commit 7c3fc2f261
2024-10-07 10:01:53 +02:00
7c3fc2f261
Revert "Unistore : Ensure Watch works in HA mode." ( #94097 )
...
Revert "Unistore : Ensure Watch works in HA mode. (#93428 )"
This reverts commit 0a26c9e9ae
2024-10-01 18:45:47 +00:00
0a26c9e9ae
Unistore : Ensure Watch works in HA mode. ( #93428 )
...
* Replace Watch with WatchNext
* remove watchset
* fix previous page and closing the channel
* Remove the broadcaster cache to prevent dupplicated events
* add watch bookmark
* add watch bookmark
* cleanup comments
* disable the tests for bookmarks for now
* Ensure we send previosu events
* lint
* re-introduce the cache
* load from cache
* disabling legacy test
* disabling legacy test
* Update pkg/storage/unified/resource/server.go
Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>
* Could not read previous events
* add proper migration
* Add previous_resource_version to both history and resource
* First event should have an RV of 2 and not 1
* Test both storage backends
* fix the inital RV for the sql backend
* ensure graceful stop of the stream decoder
* gocyclo
---------
Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>
2024-09-30 13:14:07 +02:00
216b63549d
unistore: fix watch events forever looping ( #93517 )
2024-09-20 15:51:09 +02:00
e788df921c
Storage: always use transactions and make them readonly when possible ( #92110 )
...
* always use transactions and make them readonly when possible
* fix linters
* fix reference
2024-08-20 09:29:06 -03:00
f432a1713b
SQLTemplate: Swap the IFace with the concrete struct ( #92008 )
2024-08-16 15:12:37 +03:00
d9cabe5e14
SQL: Add sql template test helper ( #91953 )
2024-08-16 14:36:56 +03:00
f804b0baa3
Storage: Return an iterator from backend List ( #91185 )
2024-07-31 12:05:59 +03:00
da65825c5f
Storage: Consolidate error handling ( #91167 )
2024-07-30 10:16:16 +00:00
e4ed000d05
ResourceStore: Fix sql List ( #91043 )
...
* fix resource list
* ensure we get the resource version without FOR SHARE
2024-07-26 13:56:24 +02:00
4baca6947d
ResourceServer: make the resource store the default unified storage backend ( #90899 )
...
* make the resource store the default unified storage backend
* add integration tests
* fix test non passing
* Update pkg/storage/unified/sql/test/integration_test.go
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* lint
* fix tests
* fix no rows
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2024-07-25 18:17:39 +02:00
399d77a0fd
Resource server improvements and fixes ( #90715 )
...
* cleanup dependencies and improve list method
* Improve Resource Server API, remove unnecessary dependencies
* Reduce the API footprint of ResourceDBInterface and its implementation
* Improve LifecycleHooks to use context
* Improve testing
* reduce API size and improve code
* sqltemplate: add DialectForDriver func and improve naming
* improve lifecycle API
* many small fixes after adding more tests
2024-07-22 20:08:30 +03:00
b2766ece5e
Resource store sql backend watch ( #90480 )
...
* Squashed commit of the following:
commit 3ae9aea0fdc5eef70e16e0ba26f817ca1883ebc9
Author: Georges Chaudy <chaudyg@gmail.com>
Date: Wed Jul 17 23:21:07 2024 +0200
Add group and resource to ListOptions
commit 638441a656d4098343fd72f1942c5845fb50f536
Author: Georges Chaudy <chaudyg@gmail.com>
Date: Wed Jul 17 22:25:35 2024 +0200
poll for resources independently
commit 4c4c88e2d6b69a589a03fc4ea3b287ef557d67a0
Author: Georges Chaudy <chaudyg@gmail.com>
Date: Wed Jul 17 17:39:10 2024 +0200
Update pkg/storage/unified/sql/data/resource_history_poll.sql
Co-authored-by: Dan Cech <dcech@grafana.com>
commit 67f42dbfcaa6b92056c7e6e25f21e870b7fa08e6
Author: Georges Chaudy <chaudyg@gmail.com>
Date: Wed Jul 17 16:01:13 2024 +0200
Update pkg/storage/unified/sql/backend.go
Co-authored-by: Dan Cech <dcech@grafana.com>
commit e869e829d07d6fce6f79f18679680d32fa9491f5
Author: Georges Chaudy <chaudyg@gmail.com>
Date: Wed Jul 17 16:01:06 2024 +0200
Update pkg/storage/unified/sql/backend.go
Co-authored-by: Dan Cech <dcech@grafana.com>
commit 6e498c0cd7434007649fd8de30a7d17e1c425c42
Author: Georges Chaudy <chaudyg@gmail.com>
Date: Tue Jul 16 16:47:35 2024 +0200
Add watch working for multiple resources
commit 0da775e521e15d1d060ca64f88b0eadd383150bfccf12406a62b4b9f66d3df72a61a52079f0715aadf72a61a5212257d9dfe07b804457e1fc1dd09142207ef10de36d9099d1a37e0745d7aa6538fa8dfe8214869def6b32c94bdd305f1baf4c8f516fc55f3c554e66868a0cf5d81d98da81a4280bf285216396979bcf626a2f947e84275a01bc275c25a527d774ae238cb95ffa3486bab591cf4d1aa4901e20636f761009740cb4ac30f40cb4ac30f8f4b3062d6fe201b6bb28f4b3062d6428b90c14b9e5b88c6ddedc1f50c4e9e5b88c6dd8dd8bc85a659a142095fde0676285259a142095f53f16521a7f70f60efd053f16521a79fa906ab8022d3f163dfd09979b332411bab6d44d09979b332ddcbc753d32a4fedb6eee728b12ec0dc163d10cc36f761009726ef342b7eddb5f125f06a0822472504d57a99e55b6edc96d904d57a99e533917141f0924c42f7072a1b620c38924c42f7070db0e13248a3d9f8aab28c1f60aa8babacd9aef2ddae2b4969a984a4e4f9b25e735c021fb97cf417fe8a05c648c3b5cabb14fe8a05c6489f2aae1c48ece478e12d86466aec61ece478e12df472e95f9255ba32bda7f472e95f92#89832 )
commit 8674d705ea#77338 )
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
commit 52d476db05#89867 )
commit cc5b4d6a94#89857 )
Add missing deps
commit 7c86705b9b#89864 )
commit aa9299c4d7#89863 )
Co-authored-by: Adela Almasan <88068998+adela-almasan@users.noreply.github.com>
commit f90168f500#89861 )
commit 0f65c06cfe#89852 )
commit 058538287f#87082 )
* Adding new export button
* Create Export as JSON drawer
* update scene drawer and add css
* update css
* Update ExportAsJson to be regular react component
* add tests to export menu and button
* add tests
* prettier and lint
* fix translations
* update translation
* Apply suggestions from code review
Co-authored-by: Juan Cabanas <juan.cabanas@grafana.com>
* delete extra file
* Update to use SceneObject
* add spinner
* Rename ExportAsJSON.tsx to ExportAsJson.tsx
* update i18n
* Upate texts
* small fixes from code review
* add space
* i18n
* fix build issues
* changes from review feedback
* update test
* update test
---------
Co-authored-by: Juan Cabanas <juan.cabanas@grafana.com>
commit 79092ebc6a#89802 )
commit d842178c6a#89767 )
* Add Azure AD configuration for SAML integration
commit 920756f797#89779 )
* add very basic tests to verify storybook builds correctly
* add storybook step to drone
* reorder steps
* drone tweaks
* don't need host since it's set in env
* don't need to wait
* format build.star and readd wait
* install netcat in CI
* do a yarn install here to get correct bindings
* refactoring to hopefully work better in CI
* add wait-on
* add verbose logging
* localhost?
* more logging
* specify storybook host
* ...
* back to grafana-server
* does this work? 🤔
* run storybook e2e test after rgm-package so the backround process is running for less time
* split into separate step
* format
commit a760dad1e3#89790 )
* VizPanelManager: Adapt color palete after plugin change
* Upgrade scenes
* Remove broken code
* Upgrade scenes version
* linter
commit c8ce20a807#89341 )
* Add cloudWatchRoundUpEndTime feature toggle
commit 17ebc9614b#88258 )
* Add basic button for adding a query template
* Add hook to create a template
* Handle notifications
* Add tags to invalidate cache
* Generate translations
* Updates types
* Add tests
* Simplify code
* Add user to type
* Add a better default title
* bring in piotrs pr and try to add user data
* Move out of metadata (reserved in k8s) and make new values exportable
* Show user data
* Fix bad merge
* WIP
* Add annotation data to FE
* add (failing) test
* Fix types and test
* Cleanup
* Enhance user data and send to component for display
* Fix type
* Fix expected values
* fix betterer
* Fix test
* Remove user lookup
* testing slug usage for api
* Revert "testing slug usage for api"
This reverts commit cc4556c3b76220cabd1791895e4835#88588 )
Bumps [github.com/alicebob/miniredis/v2](https://github.com/alicebob/miniredis ) from 2.30.1 to 2.33.0.
- [Release notes](https://github.com/alicebob/miniredis/releases )
- [Changelog](https://github.com/alicebob/miniredis/blob/master/CHANGELOG.md )
- [Commits](https://github.com/alicebob/miniredis/compare/v2.30.1...v2.33.0 )
---
updated-dependencies:
- dependency-name: github.com/alicebob/miniredis/v2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
commit ed6c948db0#88182 )
* Dashboards: Change datemath parse logic so parsed dates always take timezone into account
commit 523d999414#89307 )
* Use runtime data source for getting events from alert state history in the bar chart
* extract translations
* refactor
* More refactor
* Update events limit
* Add info icon with tooltip info for label querying filter
* Add translations
* Create new useRuleHistoryRecords hook skipping extraction of common labels as they are not used
* Fix test
* update limit value for the events in the api to 5000
* Use state for rows key
* remove React import
* Address review comments
* Address review comments
* run prettier
* Remove duplicated handlers
commit c3337c39b1#89692 )
* feat: add search
* feat: run i18n extraction
* refactor: styling issue for smaller screen sizes
commit 06084f0ed1#89492 )
* Zanzana: Dummy schema loading
* Load authorzation model for client
---------
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
commit ddea4ba8b2#89564 )
* Remove CancelablePromise.ts as it wasn't used in anywhere
* Move cancelable-promise to the place where it is used
commit 177747ecda#89556 )
* chore(grafana-data): remove dataframe barrel file and update all imports and exports
* chore(grafana-data): remove field barrel file and update imports n exports
* chore(grafana-data): remove utils barrel file and update imports and exports
* chore(grafana-data): remove tranformations barrel file and update imports n exports
* chore(grafana-data): remove imports from root index.ts
* chore(grafana-data): remove valueFormats barrel file and update imports n exports
* chore(grafana-data): don't import from root barrel file
commit c3b16639bc#89819 )
simplify keybinds e2e even further
commit 8b7a891db4#89522 )
Update LDAP doc
LDAP debug view does not work with single bind since there is no way to bind to the LDAP server and perform the search.
commit cb31a1b6ff#89630 )
commit 3ef048cdd7#89817 )
disable flaky new-datasource-variable e2e test
commit e09c1c7515#89176 )
* Alerting docs: adds silence RBAC 11.1
* ran prettier
* Improve docs with new rule-specific silence RBAC information
* Apply suggestions from code review
Co-authored-by: Jack Baldry <jack.baldry@grafana.com>
* Apply suggestions from code review
Co-authored-by: Jack Baldry <jack.baldry@grafana.com>
* prettier
---------
Co-authored-by: Matt Jacobson <matthew.jacobson@grafana.com>
Co-authored-by: Jack Baldry <jack.baldry@grafana.com>
commit 1aeceb9d9a#89468 )
* Alerting: Add setting for maximum allowed rule evaluation results
Added a new configuration setting `quota.alerting_rule_evaluation_results` to set the maximum number of alert rule evaluation results per rule. If the limit is exceeded, the evaluation will result in an error.
commit e96b1c0b42#89891 )
commit afe0848d162fab168fb11043efe3dbd452e53ce319496ceed5b71329a8c3be1414b36bb71329a8c337c6b56e4e80fa46b2259056ff73f680fa46b2254d49cc3118999cd506c0ed13959e33999cd506c07345ece8ef86a7064334f68390bebfba4f2713c5f68390bebfcfc192a2b5197a837e1e44a134f72b465150631944a134f72bbac63d0edbf5d9c247d9bac63d0edb37edc450a7af38567ea5b62f8c0f19af38567ea5ec925bab80b771d38d7837c2dade15bb7d6d44cdc2c9449629d67f83a1f3b07c4e421061a102274db07c4e421009741a3f49b07592620209741a3f498a6d2e9b6e63ebf7b12e75d5ad465e5e95c1bdf875d5ad465e83df3bdec8d988f5c3b083df3bdec8a115bb621427e800768ea115bb6214c17140f26313e069b5afc17140f2633a6e7dc3f05d328983a13a6e7dc3f0040f392018#89408 )
commit 3e85f87db6#89359 )
commit 0b29ca5eac0717e39ec2b3907ca5ec0717e39ec2#89337 )
commit 4b4afa1fa79f6709c1677e171702f4b0c043de5f7e171702f4760848d203890482052a760848d203#89331 )
commit 5e4fa5d7354cde5bd59fea7f6ea32f4cde5bd59f549ecfe0535f83fdef2c549ecfe0531a5de75c3431a05d56662f64556cf1c1b786c718d8abf8d851d5f3038421df9baddf538491e02cafdf9baddf53617dd1b40e8108e90fe2ef3a60110915c81b2a8bef3a60110955d1e8ad1da01f8dd099e78a5122ce504d84b8b290c3d74955504d84b8b24fa60b6546d97d59ab38f66768c67d2dd44e2f58f66768c67d4a41f7d0dd68003738fdfc820899879478befbcddd3c3b58579478befbcd19481990d12173429ce219481990d129714d87fe382d5d4e0115b958b2d1a03a978eb4e2f65dda0ca03a978eb479aed54c2c1540c98fe1f54af3d3719d3a4e236df54af3d371a0f682b3152197a0e5e8eb535e163d2197a0e5e8b696b6a32a8f44e1a349c1798320d2b44333730fafcb5a855cb44333730ff481f11d3ae7c15b0e9609e8605f1440829d645c99d802582940829d645c1857690bd0c9c18717f5ed400f0bbfc9c18717f5ad79d44db83ae95a6eb1ad79d44db8f8a2a83d597b6e4d5da416471fa05758f7032b3916471fa057d83aa7f865df148ca9ea0118c7666a5c91e52922af86293d874dc3d0ee7f465efbf8024dc3d0ee7f835879aa7aac39953d0e
2024-07-22 16:07:12 +03:00
08c611c68b
ResourceServer: Resource store sql backend ( #90170 )
2024-07-18 08:03:18 -07:00
Jean-Philippe Quéméner
Peter Štibraný
Georges Chaudy
Ryan McKinley
owensmallwood
Stephanie Hingtgen
Matheus Macabu
Leonor Oliveira
Diego Augusto Molina