Commit Graph

66 Commits

Author SHA1 Message Date
Guilherme Caulada a367ad730c
Secrets: Implement basic unified secret store service (#45804)
* wip: Implement kvstore for secrets

* wip: Refactor kvstore for secrets

* wip: Add format key function to secrets kvstore sql

* wip: Add migration for secrets kvstore

* Remove unused Key field from secrets kvstore

* Remove secret values from debug logs

* Integrate unified secrets with datasources

* Fix minor issues and tests for kvstore

* Create test service helper for secret store

* Remove encryption tests from datasources

* Move secret operations after datasources

* Fix datasource proxy tests

* Fix legacy data tests

* Add Name to all delete data source commands

* Implement decryption cache on sql secret store

* Fix minor issue with cache and tests

* Use secret type on secret store datasource operations

* Add comments to make create and update clear

* Rename itemFound variable to isFound

* Improve secret deletion and cache management

* Add base64 encoding to sql secret store

* Move secret retrieval to decrypted values function

* Refactor decrypt secure json data functions

* Fix expr tests

* Fix datasource tests

* Fix plugin proxy tests

* Fix query tests

* Fix metrics api tests

* Remove unused fake secrets service from query tests

* Add rename function to secret store

* Add check for error renaming secret

* Remove bus from tests to fix merge conflicts

* Add background secrets migration to datasources

* Get datasource secure json fields from secrets

* Move migration to secret store

* Revert "Move migration to secret store"

This reverts commit 7c3f872072.

* Add secret service to datasource service on tests

* Fix datasource tests

* Remove merge conflict on wire

* Add ctx to data source http transport on prometheus stats collector

* Add ctx to data source http transport on stats collector test
2022-04-25 13:57:45 -03:00
Kristin Laemmert 8f6877e12a
chore: remove all remaining uses of golang.org/x/net/context (#47564)
* chore: remove all remaining uses of  golang.org/x/net/context

This PR finishes the work started in #47532, replacing all calls to that package with the stdlib context and using http.NewRequestWithContext to include the context where necessary.

Bonus: small formatting fixes to goimports in these files.

closes #44178

* tweak: use context.Background in favor of TODO for tests
2022-04-11 14:20:10 -04:00
Serge Zaitsev 18e93c7077
Chore: Remove bus (#47511)
* Chore: Remove bus

* remove unused const
2022-04-08 16:15:06 +02:00
Kyle Brandt cab509fbe8
SSE: Keep FieldConfig for data source queries (#46328)
for #46325
2022-03-10 10:03:26 -05:00
Kyle Brandt a578cf0f7c
SSE: Fix NoData when some series were no data but others not (#45867)
Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>
2022-02-24 21:01:04 -03:00
Kyle Brandt 040ce40113
SSE: Mode to drop NaN/Inf/Null in Reduction operations (#43583)
Co-authored-by: Yuriy Tseretyan <yuriy.tseretyan@grafana.com>
Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
2022-02-02 14:50:44 +01:00
Santiago 1bed84ea56
Fix InfluxDB unhealthy alerts and false "no data"s (#43441)
* check for TimeSeriesTypeNot to fix InfluxDB alerts

* log a warning when a data frame is ignored
2022-01-27 10:53:27 -03:00
Marius Bezuidenhout 9fc0aee02b
Feature/mathandreduce (#41608)
* Added new math functions: round, ceil, floor 
* Added a new reduce function: last.
2022-01-21 13:15:50 -05:00
Kyle Brandt f17fb76b5e
SSE: Add is_nan() and other like functions and allow '_' in func names (#43555)
* SSE: Add is_nan() and allow '_' in func names

is_null() infn(), is_inf(), is_number()
2021-12-29 11:40:52 -05:00
idafurjes 7936c4c522
Rename AddHandlerCtx to AddHandler (#43557) 2021-12-28 16:08:07 +01:00
Ryan McKinley 2754e4fdf0
Expressions: use datasource model from the query (#41376)
* refactor datasource loading

* refactor datasource loading

* pass uid

* use dscache in alerting to get DS

* remove expr/translate pacakge

* remove dup injection entry

* fix DS type on metrics endpoint, remove SQL DS lookup inside SSE

* update test and adapter

* comment fix

* Make eval run as admin when getting datasource info

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* fmt and comment

* remove unncessary/redundant code

Co-authored-by: Kyle Brandt <kyle@grafana.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>
2021-12-16 13:51:46 -03:00
Marcus Efraimsson 8927a3ca20
Chore: Query endpoint refactor (#41637)
Get rid of using legacydata.RequestHandler in HTTPServer, /api/tsdb/query and pkg/expr 
with the goal of deprecating /api/tsdb/query and remove it completely eventually. This is 
the first step of cleaning up the HTTP API query endpoint.

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2021-11-29 14:21:54 +01:00
Tania B 4014891971
Encryption: Cleanup and add logging (#42084)
* Encryption: Add more logs

* Add logging and checks

* Removed unused methods from secrets service

* Refactor and update tests

* Address review feedback
2021-11-24 15:01:44 +02:00
George Robinson 708bdc80cb
Alerting: Support Unwrap for QueryError in expr package (#41743) 2021-11-17 10:07:24 +00:00
Jean-Philippe Quéméner 2319c52c85
Alerting: fix a bug where the metric in the evaluation string was not correctly populated (#41731)
Instead of using the frame name as the series name, we now use the name of the series value column if the frame name is not set. Queries to some datasources may end up with an empty frame name, which leads to an empty series name. As the series name is used in the frontend to display which metrics were queried etc. it shouldn't be empty.
2021-11-17 11:00:59 +01:00
Tania B bc60ae3c66
Encryption: Refactor secrets service (#41771)
* Refactor kmsproviders pkg

* Update tests

* Fix linting

Co-authored-by: Joan López de la Franca Beltran <joanjan14@gmail.com>
2021-11-17 11:52:45 +02:00
George Robinson 543b1a7e39
Alerting: Add QueryError to expr package (#41737) 2021-11-16 13:42:22 +00:00
Joan López de la Franca Beltran 44837fc592
Replace encryption.Service usages by secrets.Service (#41625)
* Replace encryption.Service by secrets.Service on expr.Service

* Replace encryption.Service by secrets.Service on live pkg

* Rename encryption.Service to encryption.Internal to clarify it must be not used
2021-11-12 12:16:39 +01:00
Marcus Efraimsson baab021fec
Chore: Refactor usage of legacy data contracts (#41218)
Refactor usage of legacy data contracts. Moves legacy data contracts 
to pkg/tsdb/legacydata package.
Refactor pkg/expr to be a proper service/dependency that can be provided 
to wire to remove some unneeded dependencies to SSE in ngalert and other places.
Refactor pkg/expr to not use the legacydata,RequestHandler and use 
backend.QueryDataHandler instead.
2021-11-10 11:52:16 +01:00
Ryan McKinley 3489721ed6
api/ds/query: simplify data sources lookup for queries and expressions (#41172) 2021-11-05 08:12:55 -07:00
Dimitris Sotirakis 469a5e4a85
Comment out "Queries with new datasource ref object" test (#41187) 2021-11-02 13:09:51 +02:00
Ryan McKinley 7319efe077
Dashboard: replace datasource name with a reference object (#33817)
Co-authored-by: Torkel Ödegaard <torkel@grafana.com>
Co-authored-by: Elfo404 <me@giordanoricci.com>
2021-10-29 10:57:24 -07:00
Will Browne e4297006f6
Chore: Propagate context for data source provisioning (#40235)
* context all the things

* apply feedback

* rollback some alerting changes

* rollback some alerting changes #2

* more rollbacks

* more rollbacks #2

* more rollbacks #3

* more rollbacks #4

* fix integration test

* add missing context

* add missing and remove incorrect dispatch
2021-10-18 17:06:19 +02:00
Andres Martinez Gotor 64c8d32fe7
Use sdk pkg for gtime (#39354) 2021-09-21 13:08:52 +02:00
Ikko Ashimine 202f295e51
Chore: Fix typo in nodes.go (#38636)
commmon -> common
2021-08-27 14:57:41 +02:00
Arve Knudsen 78596a6756
Migrate to Wire for dependency injection (#32289)
Fixes #30144

Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
2021-08-25 15:11:22 +02:00
Domas 3e124c854e
Alerting: interpolate variables when creating alert rule from dashboard panel (#37201) 2021-08-10 10:59:48 +03:00
Kyle Brandt b6a7156300
Alerting: Handle marshaling Inf values (#36947)
* SSE: change EvalMatch json value encoding to string
* update MarshalJSON on null.Float to handle inf as null
fixes #36424
2021-07-21 10:04:40 -04:00
Andrej Ocenas ea2ba06b93
CloudWatch/Logs: Fix log alerts in new unified alerting (#36558)
* Pass FromAlert header from new alerting

* Add better error messages
2021-07-09 13:43:22 +02:00
Sofia Papagiannaki abe35c8c01
Alerting: Add error recovery during rule evaluations (#35450)
* Alerting: Eval recovery after query failure

* Apply suggestions from code review
2021-06-15 19:30:21 +03:00
Sofia Papagiannaki c997f646cc
Alerting: Fix alert migration RefID generation (#35667)
* Alerting: Fix alert migration RefID generation

if the alert has more than 26 conditions

* Remove TODO comment

* Make similar change under pkg/expr
2021-06-14 22:13:45 +03:00
Ikko Ashimine e3afb63e62
Chore: Fix typo in nodes.go (#35312)
formating -> formatting
2021-06-09 16:59:44 -07:00
Kyle Brandt f4e9dcd40b
Alerting/SSE: Fix "count_non_null" reducer validation (#35451)
fixes error "reducer 'count_non_null' in condition 2 is not a valid reducer"
2021-06-09 14:42:48 -04:00
Sofia Papagiannaki eeb84d09c2
SSE: Change math expression to accept any value convertible to float (#34996)
* SSE: Change math expression to accept any scalar value

* Apply suggestions from code review

* Update test

* Remove TODO
2021-06-03 18:06:12 +03:00
Marcus Efraimsson b3e9087557
Chore: Refactor backend plugin manager/tsdb query data (#34944)
Move QueryData method into backend plugin manager which HandleRequest uses to 
query data from plugin SDK supported data sources. This allowed us to remove a lot 
of code no longer needed.

Ref #21510

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2021-06-03 14:16:58 +02:00
Kyle Brandt 4093fae99a
SSE: Refactor to simplify Series type (#35063)
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-06-02 12:29:19 -04:00
Kyle Brandt 9016d20c4c
SSE/Chore: cleanup some error messages (#34738)
removes "rpc error: code = InvalidArgument desc =" like strings from error messages.

This came from old grpc stuff left from when SSE was a plugin (that probably should
not have been used even when it was a plugin)
2021-05-26 15:15:21 +02:00
Marcus Andersson e19b3df1a9
Alerting: added possibility to preview grafana managed alert rules. (#34600)
* starting to add eval logic.

* wip

* first version of test rule.

* reverted file.

* add info colum to result to show error or (with CC evalmatches)

* fix labels in evalmatch

* fix be test

* refactored using observables.

* moved widht/height div to outside panel rendere.

* adding docs api level.

* adding container styles to error div.

* increasing size of preview.

Co-authored-by: kyle <kyle@grafana.com>
2021-05-26 10:06:28 +02:00
Will Browne 9726c76429
Chore: Swap out use of backend plugin SDK's backend.Logger (#33783)
* Remove use of backend SDKs backend.Logger

* reorder imports

* FIx logger name

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-05-07 15:16:21 +02:00
Kyle Brandt f5efe97763
SSE: limit allowed input/output to/from classic conditions (#33337)
* add tests for graph and specific cases in this pr
2021-04-27 07:22:11 -04:00
Kyle Brandt 5e818146de
Alerting/Expr: New SSE Request/QueryType, alerting move data source UID (#33282) 2021-04-23 16:52:32 +02:00
Ryan McKinley 7627b55ef4
TSDB: add deprecation comments to many tsdb structs (#33281) 2021-04-23 03:03:11 +02:00
Sofia Papagiannaki b2288f7ef9
[Alerting]: Add alerting endpoint for Query Evaluation (#33174)
* [Alerting]: Add alerting endpoint for Query Evaluation

* Fix passing down now parameter

* Add validations and test

* Fix eval queries and expressions test

* Add eval tests
2021-04-21 22:44:50 +03:00
Kyle Brandt 2c862678ab
AlertingNG/SSE: Datasource UID and UID/ID only (drop name) (#33039)
SSE still will support ID until dashboard/frontend always requests UID
update *.http examples

Co-authored-by: gotjosh <josue@grafana.com>
2021-04-16 15:29:19 +02:00
Kyle Brandt 6395c649c8
SSE: fix casing on classic conditions model. (#32754) 2021-04-07 16:12:22 +02:00
Kyle Brandt 0ec8879436
SSE/Alerting: Fix classic condition logic and some more tests (#32615) 2021-04-02 08:46:32 -04:00
Kyle Brandt 512faa7c9c
SSE: Support time series frames with non-nullable float64 values (#32608)
* SSE: fix reduce to handle non-null
* add type for data.Field that is float64 or *float64
* resample fix for non-null input case, add couple non-null tests
2021-04-01 14:33:35 -04:00
Ryan McKinley 1446d094b8
API: return query results as JSON rather than base64 encoded Arrow (#32303) 2021-03-31 08:35:03 -07:00
Kyle Brandt 05fa9d6ad9
AlertingNG: rename Condition properties to match front end (#32267) 2021-03-24 08:07:29 -04:00
Kyle Brandt 1cd8981be4
SSE/NG: Add expression metadata for EvalMatch (#32213)
- Add the ability to stick metadata attached to a value in expressions. Currently uses Frame.Meta.Custom.
- None of this is consumed by anything yet, so an incremental step.
2021-03-23 12:23:54 -04:00