4a779c4ccb
fix: unified resource server list queries order column ( #108094 )
2025-07-16 08:09:31 +02:00
0982cfd9a0
Unified Storage/Search: Add max count config for indexing ( #107255 )
...
* Add max count config for indexing
* Build empty index when max count is exceeded
* Address linting
* Refactor buildIndexes
* Add test for max count threshold
* Update test doc comments
* Refactor TestBuildIndexes_MaxCountThreshold to not use mock framework
* Rename mocks used in TestBuildIndexes_MaxCountThreshold
* Refactor mockResourceIndex
* Test setting of indexing threshold configs
* Tweak comments, log
* Fix logging in buildEmptyIndex
* Export and reuse TestDocumentBuilderSupplier
* Reuse MockResourceIndex
2025-06-27 14:00:39 +02:00
7864c1660f
K8s: Get trash fixes ( #106411 )
2025-06-11 13:54:38 -05:00
1196d66c5e
feat(unified-storage): log continue token on error ( #106475 )
2025-06-10 16:41:39 +02:00
04d39cbbc6
unistore: split ListIterator and ListHistory in StorageBackend ( #105654 )
...
split listIterator from ListHistory
2025-05-23 15:00:18 +02:00
a13fe75de8
refactor(unified-storage): set the GUID in the resource server ( #105683 )
2025-05-21 09:49:49 +02:00
a7922912fe
SecretsManager: Introduce secrets database wrapper ( #105472 )
...
SecretsManager: Introduce secret database wrapper
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-05-20 11:48:47 +01:00
91882cf50e
chore(unified-storage): add tracing for unified storage folder implementation ( #105601 )
2025-05-19 23:25:08 +03:00
5d8f2e0ec3
refactor(unified-storage): extract list iterator and add tests ( #105600 )
2025-05-19 13:05:49 +02:00
002f46736a
refactor(unified-storage): move generated protos to own pkg ( #105356 )
Actionlint / Lint GitHub Actions files (push) Waiting to run
Details
Backend Code Checks / Validate Backend Configs (push) Waiting to run
Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (1/8) (push) Waiting to run
Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (2/8) (push) Waiting to run
Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (3/8) (push) Waiting to run
Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (4/8) (push) Waiting to run
Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (5/8) (push) Waiting to run
Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (6/8) (push) Waiting to run
Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (7/8) (push) Waiting to run
Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (8/8) (push) Waiting to run
Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (1/8) (push) Waiting to run
Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (2/8) (push) Waiting to run
Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (3/8) (push) Waiting to run
Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (4/8) (push) Waiting to run
Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (5/8) (push) Waiting to run
Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (6/8) (push) Waiting to run
Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (7/8) (push) Waiting to run
Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (8/8) (push) Waiting to run
Details
CodeQL checks / Analyze (actions) (push) Waiting to run
Details
CodeQL checks / Analyze (go) (push) Waiting to run
Details
CodeQL checks / Analyze (javascript) (push) Waiting to run
Details
CodeQL checks / Analyze (python) (push) Waiting to run
Details
Lint Frontend / Verify i18n (push) Waiting to run
Details
Lint Frontend / Lint (push) Waiting to run
Details
Lint Frontend / Typecheck (push) Waiting to run
Details
Lint Frontend / Betterer (push) Waiting to run
Details
golangci-lint / lint-go (push) Waiting to run
Details
End-to-end tests / Build & Package Grafana (push) Waiting to run
Details
End-to-end tests / ${{ matrix.suite }} (dashboards-suite) (push) Blocked by required conditions
Details
End-to-end tests / ${{ matrix.suite }} (panels-suite) (push) Blocked by required conditions
Details
End-to-end tests / ${{ matrix.suite }} (smoke-tests-suite) (push) Blocked by required conditions
Details
End-to-end tests / ${{ matrix.suite }} (various-suite) (push) Blocked by required conditions
Details
End-to-end tests / ${{ matrix.suite }} (old arch) (old-arch/dashboards-suite) (push) Blocked by required conditions
Details
End-to-end tests / ${{ matrix.suite }} (old arch) (old-arch/panels-suite) (push) Blocked by required conditions
Details
End-to-end tests / ${{ matrix.suite }} (old arch) (old-arch/smoke-tests-suite) (push) Blocked by required conditions
Details
End-to-end tests / ${{ matrix.suite }} (old arch) (old-arch/various-suite) (push) Blocked by required conditions
Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (1) (push) Waiting to run
Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (2) (push) Waiting to run
Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (3) (push) Waiting to run
Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (4) (push) Waiting to run
Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (5) (push) Waiting to run
Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (6) (push) Waiting to run
Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (7) (push) Waiting to run
Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (8) (push) Waiting to run
Details
Integration Tests / Sqlite (${{ matrix.shard }}) (1/8) (push) Waiting to run
Details
Integration Tests / Sqlite (${{ matrix.shard }}) (2/8) (push) Waiting to run
Details
Integration Tests / Sqlite (${{ matrix.shard }}) (3/8) (push) Waiting to run
Details
Integration Tests / Sqlite (${{ matrix.shard }}) (4/8) (push) Waiting to run
Details
Integration Tests / Sqlite (${{ matrix.shard }}) (5/8) (push) Waiting to run
Details
Integration Tests / Sqlite (${{ matrix.shard }}) (6/8) (push) Waiting to run
Details
Integration Tests / Sqlite (${{ matrix.shard }}) (7/8) (push) Waiting to run
Details
Integration Tests / Sqlite (${{ matrix.shard }}) (8/8) (push) Waiting to run
Details
Integration Tests / MySQL (${{ matrix.shard }}) (1/8) (push) Waiting to run
Details
Integration Tests / MySQL (${{ matrix.shard }}) (2/8) (push) Waiting to run
Details
Integration Tests / MySQL (${{ matrix.shard }}) (3/8) (push) Waiting to run
Details
Integration Tests / MySQL (${{ matrix.shard }}) (4/8) (push) Waiting to run
Details
Integration Tests / MySQL (${{ matrix.shard }}) (5/8) (push) Waiting to run
Details
Integration Tests / MySQL (${{ matrix.shard }}) (6/8) (push) Waiting to run
Details
Integration Tests / MySQL (${{ matrix.shard }}) (7/8) (push) Waiting to run
Details
Integration Tests / MySQL (${{ matrix.shard }}) (8/8) (push) Waiting to run
Details
Integration Tests / Postgres (${{ matrix.shard }}) (1/8) (push) Waiting to run
Details
Integration Tests / Postgres (${{ matrix.shard }}) (2/8) (push) Waiting to run
Details
Integration Tests / Postgres (${{ matrix.shard }}) (3/8) (push) Waiting to run
Details
Integration Tests / Postgres (${{ matrix.shard }}) (4/8) (push) Waiting to run
Details
Integration Tests / Postgres (${{ matrix.shard }}) (5/8) (push) Waiting to run
Details
Integration Tests / Postgres (${{ matrix.shard }}) (6/8) (push) Waiting to run
Details
Integration Tests / Postgres (${{ matrix.shard }}) (7/8) (push) Waiting to run
Details
Integration Tests / Postgres (${{ matrix.shard }}) (8/8) (push) Waiting to run
Details
Reject GitHub secrets / reject-gh-secrets (push) Waiting to run
Details
Run dashboard schema v2 e2e / dashboard-schema-v2-e2e (push) Waiting to run
Details
Dispatch sync to mirror / dispatch-job (push) Waiting to run
Details
trigger-dashboard-search-e2e / trigger-search-e2e (push) Waiting to run
Details
Trivy Scan / trivy-scan (push) Waiting to run
Details
2025-05-15 21:36:52 +02:00
3380ea441a
feat(unified-storage): return guid, group and resource on read/list ( #104121 )
2025-04-17 12:58:58 +02:00
a9ef8bcced
Use logging from app sdk in the resource mod ( #103281 )
...
* Use logging from app sdk in the resource mod
* make update-workspace
* Use app-sdk logging in SQL backend
* Use grafana-app logging in tests
* make update-workspace
* make update-workspace
* make update-workspace
* Use default logging
* Remove dependency on grafana/grafana
* Fix imports
2025-04-08 15:35:11 +02:00
9c49c601f1
Unistore: Add generation column ( #102222 )
2025-04-01 20:38:23 +03:00
f0a6327edc
Unified Storage: Don't read before create ( #102906 )
...
* Unified Storage: Don't read before create
* test: use the existing test infra
* fix: support pq
We use pgx, but it seems to be wrapped in a pq driver shim, causing the errors to be remapped to pq's type. Weird
situation.
* feat: support CDK backend
* revert: there is a postgres_tests block
* fix(CDK): only check existence on ADDED updates
* fix(CDK): use ReadResource to deal with deleted files
2025-03-31 15:06:31 +02:00
fe1a72e41b
fix(unified-storage): unlist the deleted resource's versions ( #102588 )
2025-03-27 18:34:37 +01:00
f7b9f1ce69
Unified Storage: Return an already exists error ( #102857 )
...
* Unified Storage: Return an already exists error
When inserting a resource that already exists (i.e. race condition), we can safely catch UNIQUE constraint violations
and transform them into a `k8s.io/apimachinery/pkg/api/errors` error that stands the test of `errors.IsAlreadyExists`.
* feat: clarify existing conflict error
* chore: make update-workspace
* feat: make new package for backend error
* fix: assign dependency owner
* feat: use dialect for checking error type
* chore: go generate
* revert: to 5af369166d
2025-03-26 14:44:44 +01:00
cc8b40585b
feat(unified-storage): add traces for all the backend sub-paths ( #102876 )
2025-03-26 12:50:29 +01:00
65d474bfa4
chore(unified-storage): update default pruner history limit ( #102799 )
2025-03-25 17:00:12 +02:00
d3468105d7
Storage: Fix resource history sorting and pagination in Spanner backend ( #102777 )
...
* Align spanner ListHistory implementation with sql one
* Move migration logic
* Move migration code
---------
Co-authored-by: Marco de Abreu <18629099+marcoabreu@users.noreply.github.com>
2025-03-25 15:28:24 +01:00
64e939fd5d
Unified Storage: Adds span and debug log for paginated List calls ( #101912 )
...
Adds span and debug log for paginated List calls
2025-03-21 14:20:27 -06:00
6d570db312
Storage: Add ascending order support for NotOlderThan queries and introduce ResourceVersionMatch_Unset as default ( #102505 )
...
* Add support for ASC ordering and introduce ResourceVersionMatch_Unset as default
Add SortAscending to continue token and add integration test for pagination.
* Change protobuf order
* Make backwards compatible
* Update pkg/storage/unified/sql/backend.go
Co-authored-by: Jean-Philippe Quéméner <JohnnyQQQQ@users.noreply.github.com>
---------
Co-authored-by: Marco de Abreu <18629099+marcoabreu@users.noreply.github.com>
Co-authored-by: Jean-Philippe Quéméner <JohnnyQQQQ@users.noreply.github.com>
2025-03-21 18:35:32 +02:00
c33a53a47a
K8s: Remove restore functionality; can be done with list ( #102560 )
2025-03-20 16:38:32 -05:00
ce350df79b
Storage: Add resource version matching in unified storage API ( #102417 )
...
Add NotOlderThan support to getHistory
Add support for Exact
Add tests
Refactor tests
Add error test
Co-authored-by: Marco de Abreu <18629099+marcoabreu@users.noreply.github.com>
2025-03-19 16:16:48 +01:00
3e3ab8e064
fix(unified-storage): reduce lock contention when pruning ( #102425 )
2025-03-19 12:15:04 +01:00
1700a8aa9f
feat(unified-storage): prune history table based on limits ( #101970 )
2025-03-17 11:36:38 +01:00
3589baac68
Unistore: Batch write events ( #101381 )
...
* Batch write events
* Improve instrumentation
* Measure batch phases
* Detect lock contention
* remove the execBatch goroutine
* removing tracing prefix
* detect context cancel
* batch channel map
2025-03-13 10:24:12 +02:00
f5e5824bab
fix (unified-storage): stop registering unified storage metrics in global state ( #101322 )
...
* move prometheus.register for unified storage metrics into metrics.go and do most of the plumbing to get it to work
* convert StorageApiMetrics to pointer and check for nil before using it
* rename type and variables to something more sensible
---------
Co-authored-by: Jean-Philippe Quéméner <jeanphilippe.quemener@grafana.com>
2025-02-28 07:39:39 -05:00
806c043e45
UnifiedStorage: Rename Batch processing to Bulk ( #101413 )
2025-02-28 08:41:08 +03:00
8c935c8f4a
[unistore] Add benchmark for write throughput ( #101345 )
...
* Add generic benchmark
* address comments
2025-02-26 17:17:35 +02:00
e13bd52da6
unistore: move continue to the resource package ( #101206 )
...
* minor improvements to tests
* move continue token
* update sql backend
2025-02-24 18:02:30 +02:00
7be1fd953a
fix(unified-storage): enable in-process events for single-instance ( #100807 )
2025-02-21 12:25:35 +01:00
b4c4b9abbd
Remove embedded WriteEvent from WrittenEvent struct, to avoid having unnecessary fields. ( #101081 )
2025-02-20 17:34:25 +01:00
b1222be02e
unistore: add small buffer of watched events ( #100431 )
...
* change log level
* Add a small buffer when watching events
2025-02-14 12:18:59 +02:00
a5355fd66c
Storage: Add command line tool to migrate legacy dashboards (and folders) to unified storage ( #99199 )
2025-02-11 19:57:46 +02:00
3ba0d8d4b5
Unified Storage: Improve observability for indexing latency ( #99700 )
...
* adds extra debug logs and a new metric for poller query latency to help us better understand watch and index latency for write events
* adds trace span to the index for handling index write events
2025-01-28 12:30:20 -06:00
0cef2b9ae7
Dashboard Versions: Make compatible with app platform ( #99327 )
2025-01-28 08:17:52 -06:00
c4c934e0bd
SQL/Storage: Remove SkipDataMigration flag ( #99404 )
2025-01-23 12:34:48 +02:00
9d30911107
SQL/Storage: Use xorm code migration for DeletionMarker ( #99353 )
2025-01-22 14:07:08 +03:00
356b32008b
Storage: Show history+trash using the list command ( #99009 )
...
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-01-17 15:54:25 +03:00
8f6e9f8ed0
Restores in app platform ( #97582 )
2024-12-13 16:55:43 -06:00
d762a96436
Unified Storage: Init at startup, fix traces, and speed up indexing ( #97529 )
...
* dont lazy init unified storage
* Inits index when creating new resource server. Fixes trace propagation by passing span ctx. Update some logging.
* Use finer grained cache locking when building indexes to speed things up. Locking the whole function was slowing things down.
* formatting
* linter fix
* go mod
* make update-workspace
* fix workspaces check error
* update dependency owner in mod file
* wait 1 second before querying metrics
* try with big timeout, see if fixes CI. Wont fail locally.
* skips postgres integration test. Only fails in drone. Will fix later.
* put delay back to 500 ms
2024-12-09 22:32:19 -06:00
8165258a2d
Storage: Query stats within a namespace ( #97403 )
2024-12-05 13:58:13 +03:00
9d89d8757f
Search: Build index from resource stats ( #97320 )
2024-12-03 20:20:27 +03:00
f2b96593ea
SQL: close rows to release connection ( #97147 )
...
* SQL: close rows to release connection
* dont return err from rows.Close()
2024-11-28 14:28:55 +01:00
c3571752b6
Unitstore: Fix the Folder migration ( #96078 )
...
* UniStore: Add folder to the SQL backend
This reverts commit aaa4f4a23e
2024-11-12 12:52:04 +01:00
b6596db75e
Unified Storage Indexer: Build tenant indexes concurrently ( #95795 )
...
* WIP - build tenant indexes concurrently
* adds local dev seeders and readme for indexer
* update logging and adds locking in getShard()
* update logs
* Adds Namespaces func. Initializes index after ResourceServer is created.
* fixes Count() and updates test lint issues
* Initialize index separately. Don't do it when creating the resource server. Makes testing really awkward.
* fix lint error
* handles error when getting namespaces
* updates test and index helper funcs
2024-11-06 12:58:07 -06:00
aaa4f4a23e
Revert "UniStore: Add folder to the SQL backend" ( #95966 )
...
Revert "UniStore: Add folder to the SQL backend (#95671 )"
This reverts commit f075662696
2024-11-06 17:04:57 +02:00
f075662696
UniStore: Add folder to the SQL backend ( #95671 )
...
* Add folder to the SQL backend
* remove unused key
* fix tests
* Return folders from watch
2024-11-05 10:37:23 +01:00
ecebaf1bdf
Unified Storage: Add traces to poller ( #95369 )
...
add traces to poller
2024-10-24 22:49:38 +03:00
257aca5340
Unistore Chore: Fix span name ( #94965 )
...
fix span name
2024-10-24 12:12:20 -03:00
Mustafa Sencer Özcan
Arati R.
Stephanie Hingtgen
Jean-Philippe Quéméner
Georges Chaudy
Dana Axinte
Leonor Oliveira
Ryan McKinley
Mariell Hoversholm
Marco de Abreu
owensmallwood
Will Assis
Peter Štibraný
Matheus Macabu
Diego Augusto Molina