Commit Graph

29 Commits

Author SHA1 Message Date
Ryan McKinley c530cacb1c
Storage: Propagate RV to server on update+delete (#111866) 2025-10-01 18:00:02 +03:00
Ryan McKinley cfbf64c3fd
Validation: Move validation into apimachinery package (#111736) 2025-09-30 12:59:33 +00:00
Alexander Zobnin 294fd943c0
Chore: Update authlib (#110880)
* Chore: Update authlib

* exclude incompatible version of github.com/grafana/gomemcache

* Update go-jose to v4

* fix jose imports

* remove jose v3 from go.mod

* fix tests

* fix serialize

* fix failing live tests

* add v1 of ES256 testkeys. Port tests to use ES256 instead of HS256

* accept more signature algs for okta and azuread

* azure social graph token sig

* accept more signature algs for oauth refresh and jwt auth

* update workspace

* add a static signer for inproc

* rebase and fix ext_jwt

* fix jwt tests

* apply alex patch on gomemcache

* update linting

* fix ext_jwt panic

* update workspaces

---------

Co-authored-by: Jo Garnier <git@jguer.space>
2025-09-15 12:45:15 +02:00
Peter Štibraný 238f121e10
LastModifiedSince: return latestRV even when it hasn't changed. (#110391)
Return latestRV.
2025-09-01 08:45:40 +00:00
Peter Štibraný 3a3ba483b1
unified-storage: Skip query when ListModifiedSince cannot return anything. (#110338)
* Skip query when ListModifiedSince cannot return anything.

* Only save listRV if it's different than sinceRV. This saves a disk access if not needed.

* Add test for ListModifiedSince with same RV.
2025-08-29 14:49:20 +02:00
owensmallwood 699bffc9e4
Unified Storage: Dont need to test casing with ordering (#110245)
Dont need to test casing with ordering.
2025-08-27 14:06:43 -06:00
owensmallwood b4ed217656
Unified Storage: Removes check for name ordering and adds test (#110183)
removes check for name ordering. Adds test to assert expected name and rv ordering.
2025-08-26 12:39:30 -06:00
Matheus Macabu 78ac555c4d
CI: Run gofmt on Linting action (#110159)
* CI: Run gofmt on Linting action

* Chore: Run gofmt on all packages
2025-08-26 16:14:03 +02:00
owensmallwood cace999671
Unified Storage: Add ListSinceModified to StorageBackend (#109697)
* WIP added ListSinceModified to StorageBackend interface

* fix compile time check

* Fix method name

* Fix naming

* fix the rest of the ListSinceModified names

* Uses resource key without name field

* get latest rv from resource_version. Update test.

* adds moar tests

* adds method stub for ListModifiedSince to other StorageBackend implementations

* adds dummy impl to noop storage backend for ListModifiedSince

* skip tests for badger kv backend for now

* fixes tests and adds badgerkv impl for ListModifiedSince

* add badger kv impl

* adds test for new query

* adds test data for new query

* adds ListModifiedSince stub to mockStorageBackend

* uncomment tests

* refactors ListModifiedSince to return an iter.seq2 and handles deduplication. Updates tests. Updates query result sorting.

* remove comments

* remove folder from query (dont need it, yet?)

* regen test queries

* updates test

* updates function comment

* use resourcepb.ResourceKey instead of ModifiedResourceKey

* wrap seq in single transaction. Rollback transaction after 30s if iterator never used. Only track last seen event. Formatting.

* skip TestListModifiedSince for kv backend

* use WatchEvent_Type for action type

* remove redundant fields from order by clause and regen test data for query

* remove redundant fields from order by clause and regen test data for query
2025-08-20 11:54:31 -06:00
Mustafa Sencer Özcan 7b4fcd486a
Revert "fix: unified resource server list queries order column" (#109529) 2025-08-12 17:29:50 +02:00
Mustafa Sencer Özcan 4a779c4ccb
fix: unified resource server list queries order column (#108094) 2025-07-16 08:09:31 +02:00
Georges Chaudy b6dd08da2f
unistore: fix delete and db closed in kv store (#107918)
CodeQL checks / Analyze (actions) (push) Has been cancelled Details
CodeQL checks / Analyze (go) (push) Has been cancelled Details
CodeQL checks / Analyze (javascript) (push) Has been cancelled Details
* fix delete and db closed

* fix tests
2025-07-10 11:34:36 +02:00
Georges Chaudy d55541735a
unistore: use different ns for each test (#107509)
use different ns for each test
2025-07-02 13:50:30 +02:00
Georges Chaudy 696657bdd1
unistore: add kv based storage backend (#107305)
* Add datastore

* too many slashes

* lint

* add metadata store

* simplify meta

* Add eventstore

* golint

* lint

* Add datastore

* too many slashes

* lint

* pr comments

* extract ParseKey

* readcloser

* remove get prefix

* use dedicated keys

* parsekey

* sameresource

* unrelated

* name

* renmae tests

* add key validation

* fix tests

* refactor a bit

* lint

* allow empty ns

* get keys instead of list

* rename the functions

* refactor yield candidate

* update test

* unistore: add LastResourceVersion to datastore

* lint

* use map string

* missing err check

* fix

* Add storage backend

* remove hasmore

* fix tests

* small refactor

* pre-alloc

* extract the folder

* lint

* refactor

* handle context canceled in ListHistory to pass the tests

* fix the resource test

* unistore: provide generic tests for the kv interface (#107443)

unistore: move the kv tests to the testing package

* Update pkg/storage/unified/resource/storage_backend_test.go

Co-authored-by: Peter Štibraný <pstibrany@gmail.com>

* address comments

* comments

* comments

* comments

* normalise the names and add helper method

* events comments

* rename function

---------

Co-authored-by: Peter Štibraný <pstibrany@gmail.com>
2025-07-02 10:57:37 +00:00
Stephanie Hingtgen 7864c1660f
K8s: Get trash fixes (#106411) 2025-06-11 13:54:38 -05:00
Jean-Philippe Quéméner a13fe75de8
refactor(unified-storage): set the GUID in the resource server (#105683) 2025-05-21 09:49:49 +02:00
Jean-Philippe Quéméner 002f46736a
refactor(unified-storage): move generated protos to own pkg (#105356)
Actionlint / Lint GitHub Actions files (push) Waiting to run Details
Backend Code Checks / Validate Backend Configs (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (1/8) (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (2/8) (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (3/8) (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (4/8) (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (5/8) (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (6/8) (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (7/8) (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (8/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (1/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (2/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (3/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (4/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (5/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (6/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (7/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (8/8) (push) Waiting to run Details
CodeQL checks / Analyze (actions) (push) Waiting to run Details
CodeQL checks / Analyze (go) (push) Waiting to run Details
CodeQL checks / Analyze (javascript) (push) Waiting to run Details
CodeQL checks / Analyze (python) (push) Waiting to run Details
Lint Frontend / Verify i18n (push) Waiting to run Details
Lint Frontend / Lint (push) Waiting to run Details
Lint Frontend / Typecheck (push) Waiting to run Details
Lint Frontend / Betterer (push) Waiting to run Details
golangci-lint / lint-go (push) Waiting to run Details
End-to-end tests / Build & Package Grafana (push) Waiting to run Details
End-to-end tests / ${{ matrix.suite }} (dashboards-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (panels-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (smoke-tests-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (various-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (old arch) (old-arch/dashboards-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (old arch) (old-arch/panels-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (old arch) (old-arch/smoke-tests-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (old arch) (old-arch/various-suite) (push) Blocked by required conditions Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (1) (push) Waiting to run Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (2) (push) Waiting to run Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (3) (push) Waiting to run Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (4) (push) Waiting to run Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (5) (push) Waiting to run Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (6) (push) Waiting to run Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (7) (push) Waiting to run Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (1/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (2/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (3/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (4/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (5/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (6/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (7/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (8/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (1/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (2/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (3/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (4/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (5/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (6/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (7/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (8/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (1/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (2/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (3/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (4/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (5/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (6/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (7/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (8/8) (push) Waiting to run Details
Reject GitHub secrets / reject-gh-secrets (push) Waiting to run Details
Run dashboard schema v2 e2e / dashboard-schema-v2-e2e (push) Waiting to run Details
Dispatch sync to mirror / dispatch-job (push) Waiting to run Details
trigger-dashboard-search-e2e / trigger-search-e2e (push) Waiting to run Details
Trivy Scan / trivy-scan (push) Waiting to run Details
2025-05-15 21:36:52 +02:00
Georges Chaudy e162c69c34
search: fix document missing at startup (#105198)
* fix document missing at startup

* go-lint

* fix tests

* fix tests

* fix integration tests now that we are storing real values
2025-05-12 07:56:25 +02:00
Marco de Abreu b3b92e2471
Storage: Change timeout for flaky test (#104065)
Change timeout for flaky test

Co-authored-by: Marco de Abreu <18629099+marcoabreu@users.noreply.github.com>
2025-04-15 19:28:27 +01:00
Peter Štibraný be696dd70c
unified storage: check for iterator errors after each call to iter.Next. (#102804)
* unified storage: check for iterator errors after each call to iter.Next.
* Extracted test to separate method, add 500 events before listing.
2025-04-11 16:25:40 +02:00
Mariell Hoversholm f0a6327edc
Unified Storage: Don't read before create (#102906)
* Unified Storage: Don't read before create

* test: use the existing test infra

* fix: support pq

We use pgx, but it seems to be wrapped in a pq driver shim, causing the errors to be remapped to pq's type. Weird
situation.

* feat: support CDK backend

* revert: there is a postgres_tests block

* fix(CDK): only check existence on ADDED updates

* fix(CDK): use ReadResource to deal with deleted files
2025-03-31 15:06:31 +02:00
Mustafa Sencer Özcan fe1a72e41b
fix(unified-storage): unlist the deleted resource's versions (#102588) 2025-03-27 18:34:37 +01:00
Marco de Abreu 6d570db312
Storage: Add ascending order support for NotOlderThan queries and introduce ResourceVersionMatch_Unset as default (#102505)
* Add support for ASC ordering and introduce ResourceVersionMatch_Unset as default
Add SortAscending to continue token and add integration test for pagination.

* Change protobuf order

* Make backwards compatible

* Update pkg/storage/unified/sql/backend.go

Co-authored-by: Jean-Philippe Quéméner <JohnnyQQQQ@users.noreply.github.com>

---------

Co-authored-by: Marco de Abreu <18629099+marcoabreu@users.noreply.github.com>
Co-authored-by: Jean-Philippe Quéméner <JohnnyQQQQ@users.noreply.github.com>
2025-03-21 18:35:32 +02:00
Georges Chaudy 0bafd4e99d
unistore: update spanner ddl to include unistore tables (#102198)
* provision unistore tables

* update the tests to use generated namespace

* update ddl

* update ddl

* clean streams

* missing space

* use engine.db.query
2025-03-18 14:37:11 +01:00
Ryan McKinley 8f37822050
Unified/Blob: Allow direct access to UIDs (#101339) 2025-02-26 16:18:59 +02:00
Georges Chaudy 53e91fd5e8
unistore: close event stream on context cancelation (#101293)
* add tests for broacaster

* fix sql notifier not closing the stream

* fix sql notifier not closing the stream

* close sub

* fix broadcaster test

* fix broadcaster test

* suggestion
2025-02-25 08:28:31 -08:00
Georges Chaudy e13bd52da6
unistore: move continue to the resource package (#101206)
* minor improvements to tests

* move continue token

* update sql backend
2025-02-24 18:02:30 +02:00
Georges Chaudy 40d1677265
unistore: improve storage backend integration tests (#101134)
* Add Stats tests

* Add integration tests for getstats

* fix signature
2025-02-21 15:06:23 +01:00
Georges Chaudy 6e5de80ef2
unistore: refactor the storage backend test suite (#101061)
* refactor the storage backend test suite

* move testing

* pass a newBackendFunc
2025-02-20 16:43:26 +01:00