Commit Graph

314 Commits

Author SHA1 Message Date
Stephanie Hingtgen ce50f2cf61
Revert "Secrets: Refactor data_key_id out of the encoded secure value payload" (#112034)
Revert "Secrets: Refactor data_key_id out of the encoded secure value payload…"

This reverts commit acad92864e.
2025-10-05 09:53:43 +03:00
Michael Mandrus acad92864e
Secrets: Refactor data_key_id out of the encoded secure value payload (#111852)
* everything compiles

* tests pass

* remove file included by accident

* add entry to gitignore

* some scaffolding for the migration executor

* remove file

* implement and test the migration

* use xkube.Namespace in our interfaces

* add todo

* update wire deps

* add some logs

* fix wire dependency ordering

* create tests to validate error conditions during migrations
2025-10-03 15:25:46 -04:00
Daniele Stefano Ferru e69cc03ef9
Secret: add ability to configure extra owner decrypters (#111301)
---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-09-19 07:41:56 -05:00
Ryan McKinley 7c95d3c8a9
Folders: Split legacy out of folder.Service (and remove folder.FolderStore) (#110734) 2025-09-08 18:27:49 +03:00
Andrew Hackmann 9445328a59
Prometheus data source: Migration service (#107364)
* copying from secrets migration

* service runs and mig promds type

* creating data source check

* adding aws

* split into azure/aws service. feature flag. auto install

* add tests

* clean up

* lint

* add code owner

* imporvments from andres

* remove prom mig from http_server

* remove interface for testing

* add prom mig to provisining data sources so prov happens before mig

* fit into prov

* comment

* log debug instead of returning in update type

* Trigger Build

* feature flag being weird

* not public method

* copying from secrets migration

* service runs and mig promds type

* creating data source check

* adding aws

* add tests

* clean up

* imporvments from andres

* remove prom mig from http_server

* remove interface for testing

* add prom mig to provisining data sources so prov happens before mig

* fit into prov

* Trigger Build

* not public method

* remove logger import
2025-09-05 15:07:06 -05:00
Igor Suleymanov a07a8d0ba2
Fix listing and getting dashboard versions across different API versions (#109860)
* Fix listing and getting dashboard versions across different API versions

What

This commit updates dashboard version service to use API version aware
API client. The service now also supports parsing different API version
representation of dashboards.

The API version aware client is also updated to support listing across
versions.

Why

Currently listing or getting specific versions is broken for all v2
versions of the dashboard API, especially if the dashboard being checked
is still saved using v1 APIs.

Signed-off-by: Igor Suleymanov <igor.suleymanov@grafana.com>

* Remove superfluous tracing spans

Signed-off-by: Igor Suleymanov <igor.suleymanov@grafana.com>

---------

Signed-off-by: Igor Suleymanov <igor.suleymanov@grafana.com>
2025-09-03 13:51:11 +03:00
Bruno f8cd7049e8
Secrets: garbage collection (#110247)
* clean up older secret versions

* start gargbage collection worker as background service

* make gen-go

* fix typo

* make update-workspace

* undo go mod changes

* undo go work sum changes

* Update pkg/registry/apis/secret/garbagecollectionworker/worker.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/registry/apis/secret/garbagecollectionworker/worker.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* default gc_worker_batch_size to 1 minute

* fix typo

* fix typo

* add test to ensure cleaning up secure values is idempotent

* make gen-go

* make update-workspace

* undo go.mod and .sum changes

* undo enterprise imports

---------

Co-authored-by: Matheus Macabu <macabu.matheus@gmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-09-02 11:11:01 -03:00
Roberto Jiménez Sánchez 4eadc823a9
Provisioning: Move repository package to provisioning app (#110228)
* Move repository package to apps

* Move operators to grafana/grafana

* Go mod tidy

* Own package by git sync team for now

* Merged

* Do not use settings in local extra

* Remove dependency on webhook extra

* Hack to work around issue with secure contracts

* Sync Go modules

* Revert "Move operators to grafana/grafana"

This reverts commit 9f19b30a2e.
2025-09-02 09:45:44 +02:00
Gábor Farkas 2e5b55a855
datasources: querier: renamed the "mt" builder to "qs" builder (#109779) 2025-08-19 12:37:56 +02:00
Mariell Hoversholm 0fc29cbaae
Rendering: Remove SVG sanitization (#109797) 2025-08-19 11:34:12 +02:00
Sofia Papagiannaki 402572c580
Add ConfigProvider and modify quota.Service to use it (#109395)
* Add config provider and integrate with wire setup

* Refactor quota service to use config provider for configuration management

* Enhance OSSConfigProvider to include logging and update ProvideService to return an error. Refactor server initialization to handle potential errors from config provider. Remove unnecessary wire binding for OSSConfigProvider.

* Update CODEOWNERS to include the configprovider package under the grafana-backend-services-squad.

* Refactor quota service initialization to include context in multiple service providers. Update tests and service implementations to ensure proper context handling during service creation.
2025-08-12 09:42:10 +03:00
Matheus Macabu 01c1a6ce5b
SecureValues: Run mutate and validation hooks in service layer (#109379)
* SecureValues: Run mutate and validation hooks in service layer

* add some unit tests
2025-08-08 13:15:23 +02:00
Matheus Macabu fc117afebc
Secrets: Provide grpc and local inline secure value client (#109179)
* TestUtils: Add helper to create certs

* InlineSecureValue: Move to its own package rather than service

* Settings: Rename decrypt server configs to grpc prefixed as it is more generic

* InlineSecureValue: Add gRPC client

* InlineSecureValue: Add service provider grpc/local depending on setting.Cfg

* Wire: Add ProvideInlineSecureValueService to basic wireset

* InlineSecureValue: Re-export in root secret pkg and generate mocks for interface

* Settings: Unify decrypt+inline setting into generic grpc client
2025-08-06 09:04:55 +02:00
Dana Axinte a3cdfce25a
SecretsManager: Consolidation service and ability to run via cli (#108774)
* list all encrypted values and count

* separate interfaces

* add time filter to global queries

* initial secrets consolidation

* Revert defaults

* More verbose description of the operation

* Add consolidation tests and tracing

* Fix lint

* Revert debug log
2025-07-31 14:45:59 +01:00
Sarah Zinger 3fad863fd1
Query Service: Combine SSE handling in single tenant and multi tenant paths (#108041)
* parse via sse

I need to figure out how to handle the pipeline.execute with our own
client. I think this is important for MT reasons, just like using our
own cache (via legacy) is important.

parsing is done though!

* WIP nonsense

* horrible code but i think it works

* Add support for sql expressions config settings

* Cleanup:
- remove spew from nodes.go
- uncomment out plugin context and use in single tenant flow
- make code more readable and add comments

* Cleanup:
- create separate file for mt ds client builder
- ensure error handling is the same for both expressions and regular queries
- other cleanup

* not working but good thoughts

* WIP, vector not working for non sse

* super hacky but i think vectors work now

* delete delete delete

* Comments for future ref

* break out query handling and start test

* add prom debugger

* clean up: remove comments and commented out bits

* fix query_test

* add prom debugger

* create table-driven tests with testsdata files

* Fix test

* Add test

* go mod??

* idk

* Remove comment

* go enterprise issue maybe

* Fix codeowners

* Delete

* Remove test data

* Clean up

* logger

* Remove go changes hopefully

* idk go man

* sad

* idk i ran go mod tidy and this is what it wants

* Fix readme, with much help from adam

* some linting and testing errors

* lint

* fix lint

* fix lint register.go

* another lint

* address lint in test

* fix dead code and linters for query_test

* Go mod?

* Struggling with go mod

* Fix test

* Fix another test

* Revert headers change

* Its difficult to test this in OSS as it depends on functionality defined in enterprise, let's bring these tests back in some form in enterprise

* Fix codeowners

---------

Co-authored-by: Adam Simpson <adam@adamsimpson.net>
2025-07-17 17:22:55 -04:00
Matheus Macabu 3cda233468
Secrets: Remove temporary allowlist (#108118) 2025-07-16 09:39:33 +02:00
Michael Mandrus ab51794bdb
SecretsManager: Unify KMS across OSS and Enterprise (#108085)
* everything is compiling

* tests passing

* remove used object

* write a test for secret key upgrades

* misc cleanup

* clean up some wording

* lint issues

* fix a typo

* import hashicorp dependency explicitly

* simplify oss kmsprovider package structure

* consolidate current provider and available providers

* add a new manager configuration test

* fix hashivault import

* fix import issue

* fix unit tests

* Update go.mod

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

---------

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-07-15 14:39:20 -04:00
Stephanie Hingtgen d39a47a89b
Provisioning: Begin using secrets store (#108044)
- Provisioning: Begin using secrets store
- Refactor integration with secrets store
- Add back the legacy service
- Separate concerns for encrypt and decrypt
- Handle update within Encrypt function
- Add interface for secure value service
- Add feature flag for using secrets service
- Add the dual service for temporary solution.

* Add first integration tests for encrypted tokens
* Add integration test for app platform secrets
* Validate it has the name or not
* Create wire provider
* Always save to the secret if provided secret

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
Co-authored-by: Roberto Jiménez Sánchez <jszroberto@gmail.com>
2025-07-15 15:43:17 +02:00
Matheus Macabu 4954822cbb
Secrets: Add separate package for resource validation (#108097)
* Secrets: Add SecureValueService interface

* Secrets: Move resource validators to their own package/structs for reusing
2025-07-15 12:33:14 +02:00
Tania e079cb3738
Simplify open feature setup (#107632)
* Simplify open feature setup

* Fix linter issues

* Apply review feedback

* Fix integration tests setup
2025-07-14 16:22:04 +02:00
Bruno 8283d35e56
Secrets: make operations sync (#107732)
* Secrets: make operations sync

* k8s gen / update query to list secure values to include the version

* always store new version of a secret

* make update-workspace

* go mod tidy

* update queries

* update queries

* improve and use testutils in decrypt_store_test

* fix broken test

* make update-workspace

* ./hack/update-codegen.sh secret

* update Test_SecureValueMetadataStorage_CreateAndRead

* undo dependency changes

* linter: fix remaining errors

---------

Co-authored-by: Matheus Macabu <macabu.matheus@gmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-07-09 10:43:34 -03:00
Dana Axinte 46c38fdbb7
SecretsManager: Introduce worker and secret async service (#107614)
SecretsManager: Introduce worker and secret aysnc service

Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
2025-07-04 13:13:48 +01:00
Dana Axinte 15e1aa8855
SecretsManager: Introduce decrypt store (#107586)
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
2025-07-04 09:22:10 +01:00
Dana Axinte 4d8678c7f2
SecretsManager: Add base encryption manager (#107562)
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-07-03 11:29:14 +01:00
Dana Axinte 0fccc01ebe
SecretsManager: add data key store (#107396)
* SecretsManager: Add data key store

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>

* SecretsManager: Add wiring of data key store

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>

---------

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-06-30 17:17:07 +01:00
Tania 67a952c34e
Implement OFREP compatible feature flag service (#105632)
* Add ofrep pkg

* api server: Use namespace from request in case user is not authenticated

* Add handlers to ofrep api builder

* Add NewOpenFeatureService to initialize mt apiserver

* allow specifying CA and insecure

* Compare namespace with eval ctx stackID

* Organize ofrep package

* Implement AllowedV0Alpha1Resources

* Revert folderimpl changes

* Handle default namespace

* Fix extracting stack id from eval ctx

* Add more logs

* Update pkg/registry/apis/ofrep/register.go

Co-authored-by: Dave Henderson <dave.henderson@grafana.com>

* Update pkg/registry/apis/ofrep/register.go

Co-authored-by: Dave Henderson <dave.henderson@grafana.com>

* Apply review feedback

* Replace contexthandler with types

* Fix identifying authed request

* Refactor checks in the handlers

* Remove anonymous from isAuthenticatedRequest check

---------

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
Co-authored-by: Gabriel Mabille <gabriel.mabille@grafana.com>
Co-authored-by: Charandas Batra <charandas.batra@grafana.com>
Co-authored-by: Dave Henderson <dave.henderson@grafana.com>
2025-06-27 18:30:38 +02:00
Dana Axinte de28231f2f
SecretsManager: Add outbox store (#106613)
SecretsManager: add outbox store

Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-06-12 13:31:48 +01:00
Dana Axinte c22b4845bb
SecretsManager: Add encrypted value store (#106607)
* SecretsManager: add encrypted value store

Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>

* SecretsManager: wiring of encrypted value store

---------

Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
2025-06-12 11:52:01 +01:00
Dana Axinte 6e5e133f7d
SecretsManager: Introduce db migrator with keeper table (#105538)
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-05-21 15:02:51 +01:00
Dana Axinte a7922912fe
SecretsManager: Introduce secrets database wrapper (#105472)
SecretsManager: Introduce secret database wrapper

Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-05-20 11:48:47 +01:00
Ieva 2cddaf0781
RBAC: Remove dashboard and folder guardians (#104646)
* replace usage of folder guardians with access control evaluators

* remove NewByFolderUID guardian

* bring up to date

* fix test

* more test fixes, and don't fetch the folder before evaluating lib element access

* change what error is returned

* fix alerting test

* try to fix linter errors

* replace the use of newByFolder guardian with direct access control evaluator checks

* remove newByFolder guardian

* get rid of dashboard and folder guardians

* undo unwanted change

* undo unwanted change

* undo unwanted change

* update code owners
2025-05-19 09:48:16 +03:00
Dana Axinte 581d153604
SecretsManager: Introduce DecryptAuthorizer (#105459)
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-05-15 17:52:14 +01:00
Hugo Kiyodi Oshiro 43748e43bb
Plugins: Add plugins auto update feature (#104112) 2025-05-09 15:58:04 +02:00
Leonor Oliveira e9ed7223a6
Use authlib repo. Use otel (#103178)
* Use authlib repo. Use otel

* Use interceptors on the provider level

* Create a new wire set with otel

* Lint

* Fix test

* make update-workflow

* make update-workspace

* make update-workspace. Try to add authlib as enterprise imports

* make update-workspace
2025-04-07 15:47:40 +02:00
Mariell Hoversholm 77fa2271be
AppPlatform: Introduce experimental Github integration for dashboard configuration management (#96329)
* [Provisioning] Pay back some technical debt (#100720)

* Handle pagination in github client

* Add some unit test coverage

* Remove unknown repository

* Remove unknown leftover

* Revert "Add some unit test coverage"

This reverts commit 420c9674d2.

* Revert "Revert "Add some unit test coverage""

This reverts commit f7eca41957.

* Revert unit tests in github package

* Remove S3 case as it's now deprecated

* [Provisioning] Consolidate job status report in JobProgressRecorder (#100718)

* Log also successful operation

* Consolidate stop logic under TooManyErrors

* Use error for TooManyErrors

* Pass the progress recorder

* Define JobProgressRecorder interface

* Do not expect workers to return status

* Remove scenarios due to pointers

* Use recorder to manage the entire state

* Provisioning: Support rotating secrets (#100705)

* Provisioning: Refactor webhook to another interface (#100733)

* POC/Provisioning: Remove S3 references (#100734)

* Remove unused script

* Remove s3 references

* Provisioning: Keep the existing k8s name if it is specified in metadata (#100672)

* keep name

* keep name

* Revert "keep name"

This reverts commit 29f87bcaeb.

* Commit stale go.mod

* Keep name also for sync deletions

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>

* Update preview banner copy

* Query Library: Move backend to enterprise (#100371)

* Fix wire

* Fix jobs table re-renders

* Provisioning: Refactor history to its own interface (#100735)

* Provisioning: Refactor history to its own interface

* refactor: use VersionedRepository

* Update API

* Provisioning: Test the GitHub client directly (#100808)

* Provisioning: Test the GitHub client directly

Instead of mocking the abstracted client, test it as well by mocking the underlying GitHub client. This also lets us
remove the mock for the abstracted client.

* refactor: move out helpers

* chore: set dependency owner

* Provisioning: Better clone/push error support (#100854)

* Provisioning: Replace searcher with one that knows about modes (#100857)

* Provisioning: Start in "mode5" when nothing exists in legacy (#100862)

* [Provisioning] Fix duplicate sync jobs triggered in controller (#100870)

* Improve logging on reasons why the controller triggered

* Fix messaging for sync job

* fix lint

* Provisioning: Move legacy export/import into a single migrate job (#100865)

* [Provisioning] Miscellanenous bug fixes and improvements (#100976)

* Error if found duplicate ID

* Fix issue with manual test button

* Fix issue with health errors not going away

* Display status in sync overview

* Use patch operations instead

* Trigger sync job after status update

* Convert Export Tab into modal

* Remove unused FieldSet import

* Only last 8 jobs

* Remove Links card

* Use button for Github Source Code

* Add actions to resources page

* Add resource column to Repository Resources

* Display Job Spec in RecentJobs

* Display dates in history page

* Display Avatar if available

* Improve styling of the avatar

* Update betterer

* Remove duplicate history header in history

* Commit betterer

* Address code styling issues

* update flags

* github v69

* v69

* POC/Provisioning: Add wizard (#100596)

* Chore: make update-workspace

* Chore: Fix lints (#101039)

* Provisioning: Workflows as write access (#101031)

* workflow as write access

* workflow as write access

* workflow as write access

* Update pkg/registry/apis/provisioning/repository/test.go

Co-authored-by: Mariell Hoversholm <mariell.hoversholm@grafana.com>

* POC/Provisioning: Add wizard (#100596)

* update refs

* update refs

* lint fix

* lint fix

* lint fix

* default everythign to read only

* reuse form components

* remove main

---------

Co-authored-by: Mariell Hoversholm <mariell.hoversholm@grafana.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* merge main

* Fix workflow types

* Betterer

* [Provisioning] Fix webhook and finalizer issues (#101052)

Fix webhook and finalizer issue

The maximum number of webhooks per repository is 20

* [Provisioning] Fix issue with last ref (#101056)

* Fix issue with last ref

* Update frontend code

* Fix the local tmp test

* Use lastRef

* POC/Provisioning: Simplify connect step (#101064)

* Fix sending workflows

* Use write for local

* Move connect action to the next step

* Remove wizard props

* Typo

* Redirect to wizard

* Show repo link after successful export

* Provisioning: Avoid starting sync jobs when using legacy storage (#101114)

* avoid starting sync jobs on legacy

* newlines

* Provisioning: Onboarding landing page (#101112)

* add landing page before wizard

* Update onboarding page

* Update URL

* Remove unused

* Add deleteAll button

* Improved text

* betterer

---------

Co-authored-by: Clarity-89 <homes89@ukr.net>

* Provisioning: use the sync job to finish the migrate job (#101107)

* Provisioning: Show progress more often (#101128)

* show progress bar earlier

* show progress bar earlier

* update wording to be less specific

* POC/Provisioning: Enable sync (#101131)

* update preview banner

* actualy remove and don't crash without provisioning flag

* Update db banner

* Provisioning: Export oldest items first (#101189)

* Provisioning: better branch handling (#101188)

* add missing file

* Provisioning: Fix tests (#101197)

* Provisioning: Refactor tests to be multiple functions

* Provisioning: Fix tests

* fix: make github-example sync

* fix misspell

* Provisioning: avoid migration wizard if things are already in unified storage (#101204)

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* more lint

* POC/Provisioning: Handle connect step errors (#101192)

* Provisioning: Unify status handling in repository and migrate steps

* Refactor: Move WorkflowsField outside RepositoryStep and use proper type

* Refactor: Improve repository verification error handling and UI

* Refactor: Simplify repository verification error handling

* Refactor: Simplify RepositoryStep component structure

* Refactor: Improve error handling in RepositoryStep

* Refactor: Remove redundant repository creation logic from ProvisioningWizard

* Refactor: Simplify RequestErrorAlert component

* show github error

* now will verify

* test .git

* recover from bad config

* Update error handling

* Remove unused prop

* merge upstream

* Show migration summary

* Update text

* Improve text

* Betterer

* [Provisioning] Review controller changes (#101216)

* Review health check conditions

* Move down the logic to set up the sync status

* Skip if it's only a health check rerun

* Fix health check conditions

* Preserve last ref

* Format code

* Rename to shouldSkipSync

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>

* fix promotion step

* In the promotion pipeline, publish should depend on build

* fix promotion pipeline

* [Provisioning] Use smaller methods to process repository events (#101240)

* update codegen

* merge main

* Provisioning: Avoid localhost error loop (#101253)

* Provisioning: Update the recent jobs formatting (#101250)

format history

* [Provisioning] Refactor Pull Request & Lint worker (#101273)

* Refactor the code

* Refactor into separate files

* Consolidate linter flag in one spot

* Use global feature flags

* Commit betterer

* Remove from JSON the intermidiate flag

* Use again spec

* Clean up

* Revert changes in test

* POC/Provisioning: Remove sync confirm modal (#101281)

* [Provisioning] Remove linting from MVP (#101286)

* Remove Linting backend

* Re-generate client

* POC/Provisioning: Unify tags (#101218)

* Unify tags

* add both tags

* add tag types

* Check for the redirect only once

* Add fetch settings with delay hook

* Refetch settings

* Split hooks into separate files

* Cleanup

* Prettier

* Prettier

* Remove lint code

* Betterer

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* Provisioning: Update token instructions (#101280)

* Provisioning: Use blob storage rather than local file system to save images (#101298)

* [Provisioning] Add more explicit setup warnings if webhook integration and image rendering are disabled (#101304)

* Setup warnings if renders or webhooks are not possible

* Improve display

* Use a single Alert

* Make design more compact

* Only display local config is missing critical feature toggles

* Improve styling of required flags

* Add file name to the custom ini itself

* Add copy button

* Add FeatureSection Component

* Commit betterer

* Use an interactive table

* Use a modal for instructions

* Use the same modal for example config

* Improve setup steps

* Improve stepper

* Copy code ctrl + c

* Make it more compact

* Select feature to enable

* Improve the height of alerts

* Separate components

* Better warnings

* Improve the page

* Improve the cards

* Improve cards even more

* Improve cards

* Improve cards

* Optional copy in code block

* Add side bar with steps

* Improve styling

* Style modal

* Clean up Code

* Remove index file

* Simplify Step Component

* Commit betterer

* Simplify components

* Use CodeEditor and Clipboard components

* Do not show scrollback on minimum size

* Fix positioning of footer

* Separate Component for Feature

* Use different styling

* Commit betterer

* Use more Grafana components in the FeatureCard

* Separate sidebar into own components

* Simplify sidebar code

* Commit betterer

* Remove connector

* Simplify styling further

* Use cards

* Improve code

* Use more grafana component in InstructionsModal

* Further simplify

* Simplify the code

* Simplify style

* Clean up

* Simplify the Wizard

* Use little icons

* Improve feature cards

* Improve cards

* Commit betterer

* Add description to feature setup

* Improve instructions for snapshot preview

* Move all files into Setup folder

* Commit betterer

* Clean up the warnings code

* Improve coding

* Move sidebar item to separate fiel

* Rename components

* Fix issues

* Use stack instead

* Improve style

* Don't show setup button if configured already

* Simplify again CSS

* Use secondary actions

* Style a bit more

* Improve wording

* Update warning

* Refer to docs in Image Renderer

* More clean up

* Revert changes in generated client

* Fix typos and imports

* Fix lint errors

* Provisioning: better error support (#101490)

* update openapi snapshot

* fix build

* Provisioning: Only show setup page when feature toggles are missing (#101502)

* form fixing

* form fixing

* always send UID

* Same onboarding page regarless of migration (#101557)

* backend building... frontend still broken

* rename sync with main

* Provisioning: Update dashboard badge (#101599)

* Rename to push / pull everything user-facing (#101577)

* Rename to push / pull all everything user-facing

* Use automatic pulling wording

* Provisioning: Migrate when using unified storage (#101572)

* migrate when not unified

* Update pkg/registry/apis/provisioning/register.go

Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>

* variables

* merge main

---------

Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>

* Add tabs and features tab to listing page (#101570)

* List features in onboarding page (#101558)

* merge main

* POC/Provisioning: Check if the instance is provisioned (#101601)

* Check if the instance is provisioned

* Fix lints

* Fix getting config for new dashboard

* Fixes after merge

* More fixes

* Show success message

* Fix default value

* Add test

* Fix lints

* Provisioning: Include URLs in ResourceWrapper response (#101511)

* Convert Migrate wizard into a Connection Wizard (#101575)

* Convert Migrate wizard into a Connection Wizard

* Remove duplicate empty state

* Allow users to select target in the first step

* Remove file created by merge

* Select target based on existing connections

* Default option for targets and explainatory alert

* Do not display connect button if single connection

* Display target as tag in repository card

* Add Pull Step

* Fix linting

* User decides if migrate or connect

* Improve style based based on review

* Provisioning: Return upsert resource when writing (#101574)

* [Provisioning] Getting Started Page and Tab (#101701)

* merge main

* fix go.mod

* Provisioning: Redirect to the new URL after save (#101757)

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* POC/Provisioning: create folder (#101619)

* Add NewProvisionedFolderForm

* Cleanup

* Add folder api

* Register API

* Do not show provisioned badge for instances

* Enable saving

* Show saved alert

* Fixes

* Fix deps

* Cleanup

* Add test

* Add test

* Updates

* Fix test

* Fix import

* [Provisioning] Display instance connection directly in home (#101720)

* Show tabs directly in home page for instance sync

* Display actions also in the home page

* Add delete button to actions

* Fix issue with files tab

* Display tabs also in instant sync

* Generate connection name for instance sync

* Fix issues when no repositories

* Set active tab

* Remove leftover

* Rename component for listing folder repository

* Fix linting issues

* Fix merge error

* Rename to HomePage

* Split folder list into separate component

* Create own component for repository card

* Improve RepositoryCard readability

* Improve RepositoryCard styling

* Make view primary button

* Fix syntax

* Fix generateName

* merge main

* Provisioning: Move folder management into its own helper (#101864)

* folder

* update folder links

* Fix test

* update

* cleanup

---------

Co-authored-by: Clarity-89 <homes89@ukr.net>

* fixed folder issue

* POC/Provisioning: Create folder from root (#101921)

* Enable creating folder at the root

* Fix test

* Add interceptor

* Provisioning: Expose stats (#101927)

* [Provisioning] Unified onboarding wizard (#101952)

* Spike the solution

* More work

* Add more situations

* Attempt to display count of dashboards and folders

* Attempt with file count

* Do not display options if not possible

* Improve styling resources

* Use another API

* Fix issue with selection

* Style a bit

* Fix more issues

* Make the sync step work

* Improve links

* Use LinkButton

* Start pull automatically

* Start migration automatically

* Fix issue with options

* Fix issues

* Fix loading error

* Improve more things

* Improve styling

* Improve messaging

* Set the autofocus

* Fix some issues

* Fix issue with disabled options

* Only resources

* Finish settings depending on configuration

* Move title to wizard

* Fix title

* Improve styling

* Badge

* Explain on hover

* Improve styling

* Disabled at the bottom

* History & identifiers

* Improve wording

* Add padding left and right disable options

* Delete repository

* Improve buttons

* Give index time to catch up

* Improve buttons

* Handle steps with only forms

* Fix issue with initial migrate or pull

* Commit betterer

* Error messages

* Use memo

* Revampt that a bit

* Attempt to simplify the state and components

* Improve the component for Migrate

* Commit betterer

* Fix issue in next button

* Clean up more

* Start for boostrap step

* Fix issue with running status

* Fix issue with loading bootstrapping

* Improve loading

* Improve more the loading

* Fix issue with loading

* Empty tree

* Handle error

* Fix issue with looping

* Remove commented out lines

* Add comment

* Remove accidental file

* Fix imports

* Improve MigrateStep and PullStep

* Use hook for step status

* JobStep component

* Refactor data fetching

* Validate with Github

* Fix issue with failed error

* Fix next on success

* Address small comments

* Separate file for WizardContent

* Fix linting

* Use step approach also for bootstrap

* Make the logic for moving between steps clearer

* Fix navigation issue

* Clean up some logic

* Use useAsync for JobStep steps

* Revert "Use useAsync for JobStep steps"

This reverts commit 242a275cc9.

* Provisioning: use service to get counts (#101972)

counts

* must migrate when using legacy storage

* Revert "Revert "Use useAsync for JobStep steps""

This reverts commit a420d0ac36.

* Fix async conditions

* Organize imports

* Separate component for BootstrapOptionCard

* BootstrapOptionsList

* Remove duplicate definitions

---------

Co-authored-by: Clarity-89 <homes89@ukr.net>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* Merge

* Fix utils

* POC/Provisioning: Fix folder path for dashboard (#101997)

* Provisioning: Fix folder path for dashboards

* Fix isNew

* Update test

* Fix any error

* Betterer

* [Provisioning] Improve progress recording and updates (#102035)

* do not validate on delete

* Provisioning: Implement authorizer for remaining resources (#101945)

* feat: implement authorizer for remaining resources

* fix: don't allow viewers to write files

* security: harden blob id fetching

* add integration test for admin vs viewer

* feat: only Get is a valid verb for reads in our subresources

Co-Authored-By: Ryan McKinley <ryantxu@gmail.com>

* feat: allow render for all requests

* refactor: use guards

Not changing code that goes `if a { } else if b { } else { }` as the semantic meaning of the different branches is
easier to parse.

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* POC/Provisioning: Improve wizard setup (#102066)

* wizard actions

* workign better

* remove more memo

* show polling interval

* cleanup

* finalizers

* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/provisioning/Wizard/WizardContent.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/provisioning/Wizard/WizardContent.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* updates from alex

* updates from alex

* Simplify actions

* Extract props

* history supported form legacy only (for now)

* More refactor

* change order

* Fix cleanup finalizer

* show kinds

* fix lint

---------

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Clarity-89 <homes89@ukr.net>
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>

* Fix resource condition (#102086)

* Fix job summary stats issue (#102084)

* merge main

* Repository link should point to configured branch (#102092)

* Add getRepoHref

* Fix random string generation

* Redirect to Home on repository deletion (#102096)

* Fix extra commas in pull request comment (#102108)

* Fix image rendering endpoint (#102107)

* POC/Provisioning: Support migrate... when starting with unified storage (#102097)

* use same clone

* now using upsert

* Fix lint

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>

* Clean up unprovisioned resources after unified storage migration (#102126)

* Clean up unprovisioned resources after migrate

* Clean up unprovisioned resources after migrate

* Update pkg/registry/apis/provisioning/jobs/migrate/resources.go

* Reset summary between export and pull (#102101)

* Reset summary between export and pull

* Add reset results to unified storage migration

* Provisioning: always dirty (#102151)

* fix test version

* log the watch line

* POC/Provisioning: Disable repository list watch (#102169)

* Disable watch for repo list endpoint

* Add comment

* Remove another watch

* Provisioning: Avoid calling test on every update (#102161)

test less often

* Provisioning: Support prefixes in GitHub repositories (#101969)

* feat: add a Prefix property to GitHub repo spec

* feat: make nested folders work properly

* feat: use subdir for go-git export

* fix: placeholder for prefix should be grafana/

* feat: rename prefix to path

* fix: json name should be path, too

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* chore: regen apis

* fix: copy 'path'

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* Merge

* Fix duplicate type

* Provisioning: Test export functionality (#101336)

* Provisioning: Test export functionality

* fix: use context.Background

* test: add more cases for local repo path resolving

* test: rework test inputs

* test: try to make github test work

* fix: clear global state

* Update api client imports

* Fix import

* Fix test

* Update codegen

* Provisioning: Make it green (#102271)

* chore: remove unused functions

* chore: update betterer results

* chore: update openapi spec

* chore: yarn generate-apis

* fix: specify default false if undefined

* Use AnnoKeyManagerIdentity

* Add manager kind

* POC/Provisioning: Update component structure (#102297)

* Update project structure

* Update imports

* Remove unused components

* Copy fixes

* Typo

* More copy fixes

* Betterer

* Update test

* merge main

* Provisioning: Replace hardcoded clients with discovery client (#101918)

* disco client

* discovery client

* merge main

* merge main

* keep factory

* keep factory

* find preffered version for delete factory

* use same folders request

* merge main

* with integration test

* POC/Provisioning: Compare spec in test rather than raw JSON (#102352)

* compare spec not json

* compare spec not json

* [Provisioning] Add in-code TODOs in API Server area (#102360)

* Add TODOs for files endpoint

* Add TODO history endpoint

* Add TODO to move files logic to resource package

* Add TODO to not use private fields directly

* Remove unnecessary checks in list connector

* Add pagination TODO in lister

* Add TODO to rename resources

* Add todo about cloning too early

* Add TODO to propose to merge sync and migrate endpoints

* Add TODOs in register

* Add more TODOs in connectors & routes

* Add TODOs about prefix

* Change it to remove

* Update pkg/registry/apis/provisioning/test.go

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* POC/Provisioning: Switch folders to use managedBy (#102362)

* Folders: Switch to managedBy

* Fix create folder

* Fix tests

* Do not allow changing folder from dashboard settings

* Update imports

* Update provisioned meta

* Do not show provisioned badge for child folders

* Fix folder title

* Update folder actions

* Update new provisioned folder form

* Remove unused code

* Fix condition

* Reset default values on change

* Remove duplicate nav item

* Add managedBy to DashboardQueryResult

* Provisioning: support watch over live (in feature branch) (#102408)

* Provisioning: watch cleanup (#102424)

* fix lint

* Provisioning: Add basic usage stats (#102405)

* [Provisioning] Add limitations to Github Repository (#102451)

* Put limits to Github

* File is too large

* Move constants

* Embed ListOptions again

* Remove TODO

* Provisioning: Pick a better default title (#102516)

better title

* Provisioning: sanitize pull request urls (#102517)

* [Provisioning] Clean up clone after export and migrate (#102467)

* Remove clone directory on clone failure

* Defer remove clones

* Log error if removal fails

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* [Provisioning] Limit path length and depth in APIs (#102472)

* Limit filepath length in files API calls

* Add common utility to deal with paths

* Use the existing function

* Fix import

* Update pkg/registry/apis/provisioning/safepath/limit.go

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* Fix issue after website commit

* Fix linting issue in test

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* Add timeout, JSON check and max body size to endpoints (#102443)

* Add timeout, JSON check and max body size to endpoints

* Use http.MaxBytesReader instead

* Use MaxBytesReader also for reading the entire body

* Add empty line

* Add unit tests

* Fix integration tests

* Update pkg/registry/apis/provisioning/render.go

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* Do not use utils :)

* Fix comment on unmarshalJSON

* 25MB for webhook events

* Remove content type check for files write

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* fix imports

* Provisioning: Remove export option from the UI (#102511)

* [Provisioning] Remove unused checkout method in go-git (#102460)

* [Provisioning] Limit max number of repositories to 10 (#102542)

* Limit to maximum 10 repositories in backend

* Change messaging

* Do not display connect button if more than 10

* Only fetch settings once

* watch repos

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* [Provisioning] Limit size and time to git clone and push in go-git (#102458)

* Limit git clone and push time and size

* Fix linting

* Use transport instead to limit

* Remove not supported

* Add TODO to make timeout configurable

* chore: make update-workspace

* Provisioning: Implement a new job queue (#102446)

* feat: implement a new job queue

Outstanding problems:

  * Status isn't saved.
  * Progress updates don't work (due to status not being saved probably?).

* feat: properly save status

* chore: document label

* chore: assumptions do hold

* fix: support multi-tenant job drivers

* fix: use namespace=*

* fix: set resource back to pointer when updating job progress

If we don't do this, we start rejecting job progress updates as the version falls out of sync.

* feat: make job APIs read-only

* fix: complete job when worker returns

* fix: set namespace on requests from controller

* test: check historic jobs

* chore: regen apis

* feat: start augmenting frontend

* feat: add jobs to authorizer

* feat: use watch from input

* fix: make frontend subscribe to historic jobs

* fix: lint

* chore: yarn prettier:write

* fix: frontend lints

* test: allow for empty state in historicjobs

* test: set content type for export request

* fix: always set job name on insert

* fix: import

* fix: use dashes not colons

* fix: job status should expect a historic job transition

* fix: allow PR jobs from multiple PRs

* feat: same name for sync and migrate jobs

* feat: generate a job name in the store

* refactor: rename to persistentStore

* feat: remove status subresources on jobs

* feat: join jobs into one card

* chore: regen openapi snapshot

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>

* Provisioning: Use a complete storage for jobs (#102605)

* feat: add a complete strategy to apiserver

* feat: use the complete storage strategy for jobs

* test: behaviour changed in main

* [Provisioning] Consolidate file path handling (#102617)

* Add more cases for validation

* Call the method dir

* Clean files endpoint

* Simplify further

* Fix issues with folder

* Add Dir function

* Use walk function in folders

* Move things from ID

* Fix some tests

* Add tree

* Sync worker and changes

* Add more TODOs

* Add normal join

* Remove things in local

* Consolidate single Join

* Call it safe

* Add new IsPathSupported action

* Move the depth to resources

* Add more cases

* Improve trie implementation

* Add tests trie

* Fix trie tests

* Improve trie tests

* Add tests for walk

* Fix linting

* Add unit tests filepath

* Remove TODO

* Remove another TODO

* Unsupported file extension error

* Add documentation for IsPathSupported

* Filepath unit tests

* Use safepath to validate github path

* Remove TODO in wrapper

* Use trailing slash in folder internal object

* Fix changes test

* Include dot

* Add TODO to explore own type for path

* Fix frontend lint

* Fix unit tests

* Fix provisioning integration tests

---------

Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Clarity-89 <homes89@ukr.net>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: Kevin Minehart <kmineh0151@gmail.com>
2025-03-25 08:59:03 +01:00
Will Browne 5b67ae1876
Plugins: Remove support for secrets manager plugins (#101467)
* remove secrets manager code

* remove unused struct

* add test toggles back

* rollback golden file changes

* add missing TestMain

* update betterer
2025-03-20 10:00:59 +00:00
Matheus Macabu 3aba5cb2b7
SecretsManager: Bootstrap API service (#102444)
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
2025-03-19 13:41:29 +01:00
Will Assis 31a371e385
fix(unified-storage): remove bleve index metric from global scope (#101825)
* refactor grafana_index_server_index_size to calculate in a goroutine instead of at scrape time and remove grafana_index_server_indexed_docs metric

* use wire to inject bleve index metrics

* remove sprinkles metrics from bleve index metrics

* log error when trying to calculate file index size and bump interval to 1m instead of 5s
2025-03-13 10:09:38 -04:00
Yuri Tseretyan 943b73a682
Alerting: Add scheduled clean-up of deleted rules (#101963)
* add scheduled clean up of deleted rules


---------

Signed-off-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
2025-03-11 22:58:26 +02:00
Tania c9ab1142e1
Add OpenFeature providers (#101071)
* Add OpenFeature provider

* Simplify static provider

* Linting

* Apply review feedback

* Codeowner for deps

* Update pkg/services/featuremgmt/openfeature.go

Co-authored-by: Dave Henderson <dave.henderson@grafana.com>

* Apply review comments part2

* Remove from sample.ini for now

* fixup! Apply review comments part2

* Add example of context config

* Resolve conflict

* Run update workspaces

---------

Co-authored-by: Dave Henderson <dave.henderson@grafana.com>
2025-03-06 15:09:26 +01:00
Will Assis f5e5824bab
fix (unified-storage): stop registering unified storage metrics in global state (#101322)
* move prometheus.register for unified storage metrics into metrics.go and do most of the plumbing to get it to work

* convert StorageApiMetrics to pointer and check for nil before using it

* rename type and variables to something more sensible

---------

Co-authored-by: Jean-Philippe Quéméner <jeanphilippe.quemener@grafana.com>
2025-02-28 07:39:39 -05:00
Ryan McKinley 5a40c84568
DualWriter: Support managed DualWriter (#100881) 2025-02-19 17:50:39 +03:00
Stephanie Hingtgen 67be9aeed6
K8s: Search fallback: Support all sort by methods (#100776) 2025-02-18 12:30:11 -06:00
Georges Chaudy 9d68c4f665
unified: allow customising the ProvideUnifiedStorageClient (#100704)
* unified: allow customising the ProvideUnifiedStorageClient

* fix go mod
2025-02-14 11:26:51 +01:00
Alexander Zobnin 5118e82e8c
Zanzana: Run reconciliation in its own service (#100361)
* Zanzana: Start reconciliation in its own service

* cleanup

* update go workspaces

* refactor

* remove unused code

* move func definition
2025-02-11 14:09:39 +01:00
Gareth Dawson 33d82c43df
Jaeger: run health check through backend (#99322)
run health check through backend
2025-01-24 19:37:36 +07:00
Todd Treece 2622449718
Dashboards: Use dashboard service in access control (#99053) 2025-01-21 22:57:43 +02:00
Arati R. 6957e1f7b7
Folders: Replace sql query with folder service call when collecting folder tree (#98443)
* Replace sql query with folder service call when collecting folder tree
* Update provider for folder service implementation for wire
* Refactor provisioning of oss service in folder permissions test util
2025-01-07 09:53:09 +01:00
Stephanie Hingtgen 68479d844b
Orgs: Remove dependency on dashboard table for deletion (#98501) 2025-01-06 19:05:22 +02:00
owensmallwood 4837585cab
Unified Storage Search: Add sprinkles (#97820)
* Wire up sprinkles to oss and enterprise. Fetching sprinkles not implemented yet.

* Adds wireset for initializing document builders. Had to init it when creating the service to avoid cyclical imports.

* updates to int64 for stats

* adds config for sprinklesApiServer and gets sprinkles from there when its present

* add comment for later

* adds feature toggle for sprinkles. returns empty results when flag not enabled.

* adds unified storage config setting for sprinkles apiserver page limit

* fixes bug where dashboard uid was not getting set

* when creating dashboard summary, use metadata.name as the dashboard uid

* cleans up wire. use existing oss and enterprise sets to generate doc builders

* remove old wireset

* fix linter - adds missing arg for doc builders

* update dashboard stats in tests

* updates test-data dashboards

* log a warning instead of returning an error if we can't get sprinkles for a namespace

* dont read uid from dashboard json
2024-12-18 11:00:54 -06:00