Commit Graph

29 Commits

Author SHA1 Message Date
Jean-Philippe Quéméner 9282e4cad4
fix(unified-storage): don't use in-process if db type is not set (#101254) 2025-02-24 21:44:42 +01:00
Jean-Philippe Quéméner 7be1fd953a
fix(unified-storage): enable in-process events for single-instance (#100807) 2025-02-21 12:25:35 +01:00
Peter Štibraný d0394bfa7e
Extract NewSearchOptions from unified sql client setup. (#100719)
* Extract NewSearchOptions from unified sql client setup.

Co-authored-by: Georges Chaudy <chaudyg@gmail.com>
2025-02-14 16:39:48 +01:00
owensmallwood a34e7e176d
Unified Storage: Sprinkles latency metric (#100542)
* add sprinkles latency metric

* fixes failing tests - forgot to register metric only once
2025-02-12 16:22:18 -06:00
owensmallwood dd483fc17f
Unified Storage: Search permissions put behind feature flag (#99340)
* add feature flag

* puts search permission filtering behind a feature flag

* fixes issue with doc match id. When the match is for an in-memory index, the internal id is a string (this is what we expected). However, when its a file-based index, the internal id is a binary encoded int64 that point to something internally. So to get the id, we need to use ExternalID() instead of relying on the indexInternalID to be the correct format.

* adds debug log

* update comment

* formatting
2025-01-22 05:38:37 -06:00
Ryan McKinley 680e6bc1f8
Authlib: Use types package rather than claims (#99243) 2025-01-21 12:06:55 +03:00
Georges Chaudy 98e9f3a534
[unistore] wire authlib compile (#99027)
* Wire authz client compile method

* add verb to metric

* remove tipo
2025-01-16 14:11:55 +01:00
Karl Persson 3f71a72c1a
Authz: Remove "wrapper" interface and only check feature toggle for grpc mode (#98933)
* Remove "wrapper" interface and only check feature toggle for grpc and cloud mode

* Only set name for update checks

* Set dashboard permissions for admin user
2025-01-15 09:23:56 +01:00
Ryan McKinley 399cbf7c50
Search: Use new folder when creating a bleve index (#98260) 2024-12-19 18:40:04 +02:00
owensmallwood 70c9c3889f
Unified Storage Indexer: Add back metrics (#97310)
* Adds back indexer metrics. Uses config values instead of hardcoded ones.

* cast to int64

* remove unused func

* Index metrics impl doesn't depend on Bleve. Adds a TotalDocs func to SearchBackend interface.

* adds config setting for index_min_count

* rename arg

* rename metric label to namespace instead of slug

* adds default "do nothing" case to satisfy linter

* moves bleve index metrics to search package

* make bleve backend private, dont need to pass in prom reg

* imports

* adds bleve metrics to resource package to avoid circular deps
2024-12-04 15:02:40 -06:00
Ryan McKinley 9d89d8757f
Search: Build index from resource stats (#97320) 2024-12-03 20:20:27 +03:00
Ryan McKinley f6ccf976e5
UnifiedSearch: Use ResourceIndex from dashboards apiserver (v0alpha1 only) (#96939) 2024-11-27 07:57:53 +02:00
Georges Chaudy 8bb59c64f0
unistore: handle auth when fallback is used (#96772)
* handle auth when fallback is used

* handle auth when fallback is used

* add traces
2024-11-21 12:21:22 +02:00
Ryan McKinley 0cb6c3d7bf
UnifiedSearch: Introduce a DocumentBuilder interface (#96738) 2024-11-21 07:53:25 +02:00
Georges Chaudy e270412dbf
unistore: wire the authz client (#96632)
* unistore: wire the authz client

* rename dashboards.grafana.app into dashboard.grafana.app

* wire the authz client

* wire the authz client

* resuse the Standalone constructor

* configure default migration for resource folder

* add tests

* cleanup

* add logging
2024-11-19 15:13:30 +02:00
Ryan McKinley d9b87ef987
Storage: Add authz access client stubs (#95548)
authz stubs

authz stubs

lint fix

Add folder

move check.Name on creat

fix watch
2024-11-13 13:17:15 +01:00
Arati R. 7fa4a979e3
K8s/Folders: Add write access hook before creating index server (#96030)
Change order of when to add write access hook
2024-11-07 16:20:07 +02:00
owensmallwood b6596db75e
Unified Storage Indexer: Build tenant indexes concurrently (#95795)
* WIP - build tenant indexes concurrently

* adds local dev seeders and readme for indexer

* update logging and adds locking in getShard()

* update logs

* Adds Namespaces func. Initializes index after ResourceServer is created.

* fixes Count() and updates test lint issues

* Initialize index separately. Don't do it when creating the resource server. Makes testing really awkward.

* fix lint error

* handles error when getting namespaces

* updates test and index helper funcs
2024-11-06 12:58:07 -06:00
owensmallwood 5e3e7cca0e
Search PoC: Adds traces to indexer (#95372)
* adds traces to indexer

* fix linting errors
2024-10-25 00:05:33 +03:00
owensmallwood 9763199b53
Search Poc: Add metrics (#95111) 2024-10-22 12:25:08 -06:00
Ryan McKinley 3457f219be
Storage: Add blob storage interfaces (#90932)
Co-authored-by: Jean-Philippe Quémémer <jeanphilippe.quemener@grafana.com>
2024-10-17 12:18:29 +02:00
Scott Lepper 52b70ca976
Grafana indexing poc - load and watch index (#93843)
* list resources and load into index
* watch for changes and update index
2024-10-08 09:43:23 -04:00
Arati R. ea5cf7c51f
Unified Storage /Folders: Allow Unified Storage subfolders creation (#94327)
* Add parents field to folder DTO
* Allow subfolder creation when folder flag is enabled
* Update UnstructuredToLegacyFolder
* Include parents field when creating folder
2024-10-07 16:48:56 +02:00
Diego Augusto Molina 7d32d5eff4
Unistore: Reuse MySQL and Postgres Grafana core config instead of the object (#94223)
* Reuse MySQL and Postgres Grafana config instead of the object

- Only reuse the Grafana DB object for SQLite. Support for SQLite will be added in a different PR
- Fail when reusing the Grafana DB object if it is using DB instrumentation
- In the case that we have to reuse a Grafana DB with its instrumentation, fail with an error that describes a workaround
- Add regression tests to reproduce incident 2144

* remove temp file

* fix linter

* fix linter x2

* fix linter x3
2024-10-04 12:07:20 +00:00
owensmallwood 6a3eb276ef
Grafana Indexing PoC: Adds feature flag and gRPC endpoint (#93356)
* adds Filter gRPC and make protobuf

* adds route for querying the filter gRPC

* wires up Filter gRPC call

* [WIP] index from start

* renames gRPC endpoint to "Search"

* adds /apis/search route into k8s routes. Hacky for now.

* updates readme - wrong casing

* adds feature toggle for unified storage search

* hides US search behind feature flag. Clean up print statements.

* removes indexer - will be added in another PR

* Search: Add API Builder

* adds required method

* implementing UpdateAPIGroupInfo (WIP)

* adds groupversion

* commenting out for now

* remove unneeded code from experimenting and update register.go to match interface required

* namespaces search route

---------

Co-authored-by: leonorfmartins <leonorfmartins@gmail.com>
Co-authored-by: Todd Treece <todd.treece@grafana.com>
2024-09-30 13:46:14 -06:00
Ryan McKinley 45eb72e95a
K8s: Use wire to initalize the resource client (#93221) 2024-09-12 17:22:27 +03:00
owensmallwood 28f2a4078d
Unified Storage: Records grafana database metrics (#91932)
* records grafana database metrics for unified storage

* update type to tracing.Tracer

* use nil for tracer in tests
2024-08-15 11:13:36 -06:00
Diego Augusto Molina 399d77a0fd
Resource server improvements and fixes (#90715)
* cleanup dependencies and improve list method
* Improve Resource Server API, remove unnecessary dependencies
* Reduce the API footprint of ResourceDBInterface and its implementation
* Improve LifecycleHooks to use context
* Improve testing
* reduce API size and improve code
* sqltemplate: add DialectForDriver func and improve naming
* improve lifecycle API
* many small fixes after adding more tests
2024-07-22 20:08:30 +03:00
Georges Chaudy 08c611c68b
ResourceServer: Resource store sql backend (#90170) 2024-07-18 08:03:18 -07:00