Commit Graph

91 Commits

Author SHA1 Message Date
Mustafa Sencer Özcan 3056924d10
fix: enforce timeout for requests run in qos (#110162) 2025-08-26 18:40:22 +02:00
owensmallwood cace999671
Unified Storage: Add ListSinceModified to StorageBackend (#109697)
* WIP added ListSinceModified to StorageBackend interface

* fix compile time check

* Fix method name

* Fix naming

* fix the rest of the ListSinceModified names

* Uses resource key without name field

* get latest rv from resource_version. Update test.

* adds moar tests

* adds method stub for ListModifiedSince to other StorageBackend implementations

* adds dummy impl to noop storage backend for ListModifiedSince

* skip tests for badger kv backend for now

* fixes tests and adds badgerkv impl for ListModifiedSince

* add badger kv impl

* adds test for new query

* adds test data for new query

* adds ListModifiedSince stub to mockStorageBackend

* uncomment tests

* refactors ListModifiedSince to return an iter.seq2 and handles deduplication. Updates tests. Updates query result sorting.

* remove comments

* remove folder from query (dont need it, yet?)

* regen test queries

* updates test

* updates function comment

* use resourcepb.ResourceKey instead of ModifiedResourceKey

* wrap seq in single transaction. Rollback transaction after 30s if iterator never used. Only track last seen event. Formatting.

* skip TestListModifiedSince for kv backend

* use WatchEvent_Type for action type

* remove redundant fields from order by clause and regen test data for query

* remove redundant fields from order by clause and regen test data for query
2025-08-20 11:54:31 -06:00
Will Assis 43cda1f1a4
unified-storage: disable watcher when search-after-write feature flag is enabled (#109634)
* disable indexer watcher when search-after-write feature flag is enabled
2025-08-14 10:18:09 -04:00
Ryan McKinley d3df5b8ddd
Secrets: Manage secure values inside any resource (#107803) 2025-08-14 12:31:24 +00:00
Ryan McKinley e0404f924c
K8s/SecureValues: Wire InlineSecureValueSupport to apistore (#109449)
* inline wire

* extra fields

* add variable

* wire
2025-08-11 15:22:56 +03:00
Will Assis 981fdb29d4
update storage-api to only build index if it owns the namespace (#108418)
* update storage-api to only build index if it owns the namespace

---------

Co-authored-by: Mustafa Sencer Özcan <mustafasencer.ozcan@grafana.com>
2025-07-23 15:59:24 -04:00
Ryan McKinley 9786389ae8
SecureValues: Support inline secure values in GrafanaMetaAccessor (#107996) 2025-07-11 16:47:54 +00:00
Georges Chaudy b6dd08da2f
unistore: fix delete and db closed in kv store (#107918)
CodeQL checks / Analyze (actions) (push) Has been cancelled Details
CodeQL checks / Analyze (go) (push) Has been cancelled Details
CodeQL checks / Analyze (javascript) (push) Has been cancelled Details
* fix delete and db closed

* fix tests
2025-07-10 11:34:36 +02:00
Mustafa Sencer Özcan b7153d4d20
fix: grpc resource delete error when qos enabled (#107560) 2025-07-03 12:02:05 +02:00
Mustafa Sencer Özcan 974a2c47f9
feat(unified-storage): add qos support for the resource server (#105939)
Actionlint / Lint GitHub Actions files (push) Waiting to run Details
Backend Code Checks / Validate Backend Configs (push) Waiting to run Details
Backend Unit Tests / Detect whether code changed (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (1/8) (push) Blocked by required conditions Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (2/8) (push) Blocked by required conditions Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (3/8) (push) Blocked by required conditions Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (4/8) (push) Blocked by required conditions Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (5/8) (push) Blocked by required conditions Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (6/8) (push) Blocked by required conditions Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (7/8) (push) Blocked by required conditions Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (8/8) (push) Blocked by required conditions Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (1/8) (push) Blocked by required conditions Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (2/8) (push) Blocked by required conditions Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (3/8) (push) Blocked by required conditions Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (4/8) (push) Blocked by required conditions Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (5/8) (push) Blocked by required conditions Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (6/8) (push) Blocked by required conditions Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (7/8) (push) Blocked by required conditions Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (8/8) (push) Blocked by required conditions Details
Backend Unit Tests / All backend unit tests complete (push) Blocked by required conditions Details
CodeQL checks / Analyze (actions) (push) Waiting to run Details
CodeQL checks / Analyze (go) (push) Waiting to run Details
CodeQL checks / Analyze (javascript) (push) Waiting to run Details
Lint Frontend / Detect whether code changed (push) Waiting to run Details
Lint Frontend / Lint (push) Blocked by required conditions Details
Lint Frontend / Typecheck (push) Blocked by required conditions Details
Lint Frontend / Betterer (push) Blocked by required conditions Details
golangci-lint / lint-go (push) Waiting to run Details
Verify i18n / verify-i18n (push) Waiting to run Details
Documentation / Build & Verify Docs (push) Waiting to run Details
End-to-end tests / Detect whether code changed (push) Waiting to run Details
End-to-end tests / Build & Package Grafana (push) Blocked by required conditions Details
End-to-end tests / Build E2E test runner (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (--flags="--env dashboardScene=false", e2e/old-arch/dashboards-suite, dashboards-suite (old arch)) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (--flags="--env dashboardScene=false", e2e/old-arch/panels-suite, panels-suite (old arch)) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (--flags="--env dashboardScene=false", e2e/old-arch/smoke-tests-suite, smoke-tests-suite (old arch)) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (--flags="--env dashboardScene=false", e2e/old-arch/various-suite, various-suite (old arch)) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (e2e/dashboards-suite, dashboards-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (e2e/panels-suite, panels-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (e2e/smoke-tests-suite, smoke-tests-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (e2e/various-suite, various-suite) (push) Blocked by required conditions Details
End-to-end tests / A11y test (push) Blocked by required conditions Details
End-to-end tests / All E2E tests complete (push) Blocked by required conditions Details
Frontend tests / Detect whether code changed (push) Waiting to run Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (1) (push) Blocked by required conditions Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (2) (push) Blocked by required conditions Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (3) (push) Blocked by required conditions Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (4) (push) Blocked by required conditions Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (5) (push) Blocked by required conditions Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (6) (push) Blocked by required conditions Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (7) (push) Blocked by required conditions Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (8) (push) Blocked by required conditions Details
Frontend tests / All frontend unit tests complete (push) Blocked by required conditions Details
Integration Tests / Sqlite (${{ matrix.shard }}) (1/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (2/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (3/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (4/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (5/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (6/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (7/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (8/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (1/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (2/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (3/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (4/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (5/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (6/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (7/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (8/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (1/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (2/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (3/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (4/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (5/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (6/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (7/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (8/8) (push) Waiting to run Details
Integration Tests / All backend integration tests complete (push) Blocked by required conditions Details
publish-technical-documentation-next / sync (push) Waiting to run Details
Reject GitHub secrets / reject-gh-secrets (push) Waiting to run Details
Build Release Packages / setup (push) Waiting to run Details
Build Release Packages / Dispatch grafana-enterprise build (push) Blocked by required conditions Details
Build Release Packages / ${{ needs.setup.outputs.version }} / ${{ matrix.name }} (targz:grafana:darwin/amd64, darwin-amd64) (push) Blocked by required conditions Details
Build Release Packages / ${{ needs.setup.outputs.version }} / ${{ matrix.name }} (targz:grafana:darwin/arm64, darwin-arm64) (push) Blocked by required conditions Details
Build Release Packages / ${{ needs.setup.outputs.version }} / ${{ matrix.name }} (targz:grafana:linux/amd64,deb:grafana:linux/amd64,rpm:grafana:linux/amd64,docker:grafana:linux/amd64,docker:grafana:linux/amd64:ubuntu,npm:grafana,storybook, linux-amd64) (push) Blocked by required conditions Details
Build Release Packages / ${{ needs.setup.outputs.version }} / ${{ matrix.name }} (targz:grafana:linux/arm/v6,deb:grafana:linux/arm/v6, linux-armv6) (push) Blocked by required conditions Details
Build Release Packages / ${{ needs.setup.outputs.version }} / ${{ matrix.name }} (targz:grafana:linux/arm/v7,deb:grafana:linux/arm/v7,docker:grafana:linux/arm/v7,docker:grafana:linux/arm/v7:ubuntu, linux-armv7) (push) Blocked by required conditions Details
Build Release Packages / ${{ needs.setup.outputs.version }} / ${{ matrix.name }} (targz:grafana:linux/arm64,deb:grafana:linux/arm64,rpm:grafana:linux/arm64,docker:grafana:linux/arm64,docker:grafana:linux/arm64:ubuntu, linux-arm64) (push) Blocked by required conditions Details
Build Release Packages / ${{ needs.setup.outputs.version }} / ${{ matrix.name }} (targz:grafana:linux/s390x,deb:grafana:linux/s390x,rpm:grafana:linux/s390x,docker:grafana:linux/s390x,docker:grafana:linux/s390x:ubuntu, linux-s390x) (push) Blocked by required conditions Details
Build Release Packages / ${{ needs.setup.outputs.version }} / ${{ matrix.name }} (targz:grafana:windows/amd64,zip:grafana:windows/amd64,msi:grafana:windows/amd64, windows-amd64) (push) Blocked by required conditions Details
Build Release Packages / ${{ needs.setup.outputs.version }} / ${{ matrix.name }} (targz:grafana:windows/arm64,zip:grafana:windows/arm64, windows-arm64) (push) Blocked by required conditions Details
Build Release Packages / Upload artifacts (push) Blocked by required conditions Details
Run dashboard schema v2 e2e / dashboard-schema-v2-e2e (push) Waiting to run Details
Shellcheck / Shellcheck scripts (push) Waiting to run Details
Verify Storybook / Verify Storybook (push) Waiting to run Details
Swagger generated code / Verify committed API specs match (push) Waiting to run Details
Dispatch sync to mirror / dispatch-job (push) Waiting to run Details
trigger-dashboard-search-e2e / trigger-search-e2e (push) Waiting to run Details
publish-kinds-next / main (push) Has been cancelled Details
2025-07-01 09:22:55 +00:00
Arati R. 0982cfd9a0
Unified Storage/Search: Add max count config for indexing (#107255)
* Add max count config for indexing
* Build empty index when max count is exceeded
* Address linting
* Refactor buildIndexes
* Add test for max count threshold
* Update test doc comments
* Refactor TestBuildIndexes_MaxCountThreshold to not use mock framework
* Rename mocks used in TestBuildIndexes_MaxCountThreshold

* Refactor mockResourceIndex

* Test setting of indexing threshold configs

* Tweak comments, log

* Fix logging in buildEmptyIndex

* Export and reuse TestDocumentBuilderSupplier

* Reuse MockResourceIndex
2025-06-27 14:00:39 +02:00
Georges Chaudy 9062d88ea0
unistore: filter trash requests (#106767)
* deleted by user

* use the correct checker

* add tests

* refactor
2025-06-24 10:17:34 +02:00
Jean-Philippe Quéméner 5f21f320f7
fix(unified-storage): use continue token containing both formats for dualwriter (#106525) 2025-06-13 15:59:46 +02:00
Stephanie Hingtgen 5135d5c87d
Unified storage: Reconstruct index in the background every 24h (#106422) 2025-06-12 14:34:48 -05:00
Georges Chaudy 04d39cbbc6
unistore: split ListIterator and ListHistory in StorageBackend (#105654)
split listIterator from ListHistory
2025-05-23 15:00:18 +02:00
Jean-Philippe Quéméner e57be36936
fix(unified-storage): race in resource server watch (#105786) 2025-05-23 09:19:20 +02:00
Jean-Philippe Quéméner a13fe75de8
refactor(unified-storage): set the GUID in the resource server (#105683) 2025-05-21 09:49:49 +02:00
Will Assis bdae4424e8
unified-storage: setup distributor module option A (#104737)
* setup distributor module

* move lifecycler into resource server provider

* remove ring/client pool setup from distributor module and use the same ring/client pool between storage server module and distributor module

* implement resourcestore server methods

* make healthcheck fail if ring is not running
2025-05-19 07:46:03 -04:00
Jean-Philippe Quéméner 002f46736a
refactor(unified-storage): move generated protos to own pkg (#105356)
Actionlint / Lint GitHub Actions files (push) Waiting to run Details
Backend Code Checks / Validate Backend Configs (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (1/8) (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (2/8) (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (3/8) (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (4/8) (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (5/8) (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (6/8) (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (7/8) (push) Waiting to run Details
Backend Unit Tests / Grafana (${{ matrix.shard }}) (8/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (1/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (2/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (3/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (4/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (5/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (6/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (7/8) (push) Waiting to run Details
Backend Unit Tests / Grafana Enterprise (${{ matrix.shard }}) (8/8) (push) Waiting to run Details
CodeQL checks / Analyze (actions) (push) Waiting to run Details
CodeQL checks / Analyze (go) (push) Waiting to run Details
CodeQL checks / Analyze (javascript) (push) Waiting to run Details
CodeQL checks / Analyze (python) (push) Waiting to run Details
Lint Frontend / Verify i18n (push) Waiting to run Details
Lint Frontend / Lint (push) Waiting to run Details
Lint Frontend / Typecheck (push) Waiting to run Details
Lint Frontend / Betterer (push) Waiting to run Details
golangci-lint / lint-go (push) Waiting to run Details
End-to-end tests / Build & Package Grafana (push) Waiting to run Details
End-to-end tests / ${{ matrix.suite }} (dashboards-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (panels-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (smoke-tests-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (various-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (old arch) (old-arch/dashboards-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (old arch) (old-arch/panels-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (old arch) (old-arch/smoke-tests-suite) (push) Blocked by required conditions Details
End-to-end tests / ${{ matrix.suite }} (old arch) (old-arch/various-suite) (push) Blocked by required conditions Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (1) (push) Waiting to run Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (2) (push) Waiting to run Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (3) (push) Waiting to run Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (4) (push) Waiting to run Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (5) (push) Waiting to run Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (6) (push) Waiting to run Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (7) (push) Waiting to run Details
Frontend tests / Unit tests (${{ matrix.chunk }} / 8) (8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (1/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (2/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (3/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (4/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (5/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (6/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (7/8) (push) Waiting to run Details
Integration Tests / Sqlite (${{ matrix.shard }}) (8/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (1/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (2/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (3/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (4/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (5/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (6/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (7/8) (push) Waiting to run Details
Integration Tests / MySQL (${{ matrix.shard }}) (8/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (1/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (2/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (3/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (4/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (5/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (6/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (7/8) (push) Waiting to run Details
Integration Tests / Postgres (${{ matrix.shard }}) (8/8) (push) Waiting to run Details
Reject GitHub secrets / reject-gh-secrets (push) Waiting to run Details
Run dashboard schema v2 e2e / dashboard-schema-v2-e2e (push) Waiting to run Details
Dispatch sync to mirror / dispatch-job (push) Waiting to run Details
trigger-dashboard-search-e2e / trigger-search-e2e (push) Waiting to run Details
Trivy Scan / trivy-scan (push) Waiting to run Details
2025-05-15 21:36:52 +02:00
Georges Chaudy 15b3de5893
search: add index batching (#104163)
* add basic search backend integration tests

* add search backend benchmark

* add benchmark indexServer

* fix

* lint

* add more tests

* lint

* do not use the poller

* batch write

* refactor and add tests

* improvements

* improvements

* cleanup

* only observe index success

* add monitorIndexEvents method

* nit use switch instead of if

* make newIndexQueueProcessor private

* simplify runProcessor

* go lint
2025-05-09 15:36:21 +02:00
Will Assis 4adebd6058
unified-storage: setup ring to shard requests (#103783)
* Updates the instrumentation_server service to use mux instead of the builtin router, and have it store the router in the module server: this is so we can register the /ring endpoint to check the status of the ring
* Create a new Ring service that depends on the instrumentation server and declares it as a dependency for the storage server
* Create standalone MemberlistKV service for Ring service to use
* Update the storage server Search and GetStats handler to distribute requests if applicable
2025-04-25 13:08:44 -04:00
Jean-Philippe Quéméner 3380ea441a
feat(unified-storage): return guid, group and resource on read/list (#104121) 2025-04-17 12:58:58 +02:00
Georges Chaudy e69052a417
unistore: add check when update the folder of a resource (#102699)
* Add check for move folder

* make the server test generic

* address comment
2025-04-14 16:57:40 +02:00
Peter Štibraný be696dd70c
unified storage: check for iterator errors after each call to iter.Next. (#102804)
* unified storage: check for iterator errors after each call to iter.Next.
* Extracted test to separate method, add 500 events before listing.
2025-04-11 16:25:40 +02:00
Stephanie Hingtgen 68ed0feeff
Unistore: Change to 404 rather than 403 if not found (#103743) 2025-04-09 19:14:39 -05:00
Ryan McKinley af8a70bbab
K8s/Permissions: Enable a grant-permissions annotation action to set default permissions (#102527)
* create permissions

* add key

* lint

* structure as a delayed callback

* legacy API hook

* merge main

* wired up

* and folders

* watch repos

* missing return statement

* Set the correct permissions

* add TestAfterCreatePermissionCreator

* do not add perms on folder create

* fix tests

* add annotation on create

* lint

* lint

* ensure we set permissions when the FT is disabled

* remove custom folder_storage

* fix lint

* change default

* lint

* lint

* fix: annotation

* ensure permissions are added on folder legacy

* remove folderstorage again

* fix tests

* add FT

* undo change to folder

* dashboard on create

* remove annotation for folder

* fix tests

* fix prepare after rebase

* fix tests

* fix tests

* fix tests

* lint

* address comments

* add test for prepareObjectForStorage

* add again skipIfMode as per comment

---------

Co-authored-by: Georges Chaudy <chaudyg@gmail.com>
2025-04-09 13:05:37 +02:00
Mariell Hoversholm f0a6327edc
Unified Storage: Don't read before create (#102906)
* Unified Storage: Don't read before create

* test: use the existing test infra

* fix: support pq

We use pgx, but it seems to be wrapped in a pq driver shim, causing the errors to be remapped to pq's type. Weird
situation.

* feat: support CDK backend

* revert: there is a postgres_tests block

* fix(CDK): only check existence on ADDED updates

* fix(CDK): use ReadResource to deal with deleted files
2025-03-31 15:06:31 +02:00
Stephanie Hingtgen 0087d7bd58
Unistore: Skip label for getting full path (#102928) 2025-03-26 15:50:53 -05:00
Mariell Hoversholm f7b9f1ce69
Unified Storage: Return an already exists error (#102857)
* Unified Storage: Return an already exists error

When inserting a resource that already exists (i.e. race condition), we can safely catch UNIQUE constraint violations
and transform them into a `k8s.io/apimachinery/pkg/api/errors` error that stands the test of `errors.IsAlreadyExists`.

* feat: clarify existing conflict error

* chore: make update-workspace

* feat: make new package for backend error

* fix: assign dependency owner

* feat: use dialect for checking error type

* chore: go generate

* revert: to 5af369166d
2025-03-26 14:44:44 +01:00
Marco de Abreu 6d570db312
Storage: Add ascending order support for NotOlderThan queries and introduce ResourceVersionMatch_Unset as default (#102505)
* Add support for ASC ordering and introduce ResourceVersionMatch_Unset as default
Add SortAscending to continue token and add integration test for pagination.

* Change protobuf order

* Make backwards compatible

* Update pkg/storage/unified/sql/backend.go

Co-authored-by: Jean-Philippe Quéméner <JohnnyQQQQ@users.noreply.github.com>

---------

Co-authored-by: Marco de Abreu <18629099+marcoabreu@users.noreply.github.com>
Co-authored-by: Jean-Philippe Quéméner <JohnnyQQQQ@users.noreply.github.com>
2025-03-21 18:35:32 +02:00
Stephanie Hingtgen c33a53a47a
K8s: Remove restore functionality; can be done with list (#102560) 2025-03-20 16:38:32 -05:00
Will Assis 31a371e385
fix(unified-storage): remove bleve index metric from global scope (#101825)
* refactor grafana_index_server_index_size to calculate in a goroutine instead of at scrape time and remove grafana_index_server_indexed_docs metric

* use wire to inject bleve index metrics

* remove sprinkles metrics from bleve index metrics

* log error when trying to calculate file index size and bump interval to 1m instead of 5s
2025-03-13 10:09:38 -04:00
Ryan McKinley 64f1df69d5
Search: Rename managed object index (#101856) 2025-03-10 19:48:53 +03:00
Ryan McKinley dc2defd84f
K8s/Annotations: Use manager/source annotations rather than repo (#101313)
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-03-05 08:54:20 +02:00
Will Assis f5e5824bab
fix (unified-storage): stop registering unified storage metrics in global state (#101322)
* move prometheus.register for unified storage metrics into metrics.go and do most of the plumbing to get it to work

* convert StorageApiMetrics to pointer and check for nil before using it

* rename type and variables to something more sensible

---------

Co-authored-by: Jean-Philippe Quéméner <jeanphilippe.quemener@grafana.com>
2025-02-28 07:39:39 -05:00
Ryan McKinley 806c043e45
UnifiedStorage: Rename Batch processing to Bulk (#101413) 2025-02-28 08:41:08 +03:00
Ryan McKinley 8f37822050
Unified/Blob: Allow direct access to UIDs (#101339) 2025-02-26 16:18:59 +02:00
Karl Persson fa74d1c36d
Authn: Sync authlib and update how we construct authn client interceptor (#101124)
* Sync authlib and update how we construct authn client interceptor

* Remove namespace from checker
2025-02-26 09:22:09 +01:00
Georges Chaudy 53e91fd5e8
unistore: close event stream on context cancelation (#101293)
* add tests for broacaster

* fix sql notifier not closing the stream

* fix sql notifier not closing the stream

* close sub

* fix broadcaster test

* fix broadcaster test

* suggestion
2025-02-25 08:28:31 -08:00
Georges Chaudy e13bd52da6
unistore: move continue to the resource package (#101206)
* minor improvements to tests

* move continue token

* update sql backend
2025-02-24 18:02:30 +02:00
Georges Chaudy b850c9fa68
unistore: Add missing verb to compile (#100972)
Add missing verb to compile
2025-02-19 17:06:26 +02:00
Ryan McKinley a5355fd66c
Storage: Add command line tool to migrate legacy dashboards (and folders) to unified storage (#99199) 2025-02-11 19:57:46 +02:00
Stephanie Hingtgen 0cef2b9ae7
Dashboard Versions: Make compatible with app platform (#99327) 2025-01-28 08:17:52 -06:00
Misi 437b7a565d
Auth: Add access token to in-proc communication and ServiceIdentity (#98926)
Use fake access token for in-proc grpc and add ServiceIdentity 
---------

Co-authored-by: gamab <gabriel.mabille@grafana.com>
Co-authored-by: Karl Persson <23356117+kalleep@users.noreply.github.com>
2025-01-24 14:03:23 +01:00
Ryan McKinley 680e6bc1f8
Authlib: Use types package rather than claims (#99243) 2025-01-21 12:06:55 +03:00
Ryan McKinley 356b32008b
Storage: Show history+trash using the list command (#99009)
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-01-17 15:54:25 +03:00
Georges Chaudy 98e9f3a534
[unistore] wire authlib compile (#99027)
* Wire authz client compile method

* add verb to metric

* remove tipo
2025-01-16 14:11:55 +01:00
Ryan McKinley cd46f1ddb9
Search: Remove history query (#99026) 2025-01-15 12:49:47 -06:00
Karl Persson 3f71a72c1a
Authz: Remove "wrapper" interface and only check feature toggle for grpc mode (#98933)
* Remove "wrapper" interface and only check feature toggle for grpc and cloud mode

* Only set name for update checks

* Set dashboard permissions for admin user
2025-01-15 09:23:56 +01:00
Ryan McKinley ed39259461
Search: Replace Origin calls with Repository (#98754) 2025-01-10 20:27:10 +02:00