Commit Graph

46 Commits

Author SHA1 Message Date
Ivana Huckova 51391a762b
Elasticsearch: Improve backend instrumentation of `QueryData` calls (#74172)
* Elasticsearch: Improve backend instrumentation of QueryData calls

* Add fromAlert

* Fix tests, move logger and context to struct

* Add instrumentation for processing response

* Move log for sending request closer to sending request

* Update

* Fix logging, improved messages, fix printing of queries

* Update log text

* Fix tests

* Fix lint

* Update logging to follow our guidelines

* Remove key-value pairs from logs that are going to be passed from instrumentation

* Update pkg/tsdb/elasticsearch/elasticsearch.go
2023-09-07 13:54:16 +02:00
Ryan McKinley 025b2f3011
Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
Gábor Farkas e17540bdcd
Elasticsearch: Fix using multiple indexes with comma separated string (#71284)
* Revert "Elasticsearch: Use array of strings as index in backend queries (#67276)"

This reverts commit d0ced39847.

* updated tests
2023-07-11 09:47:16 +02:00
Ivana Huckova d0ced39847
Elasticsearch: Use array of strings as index in backend queries (#67276)
Elasticsearch: Use array of strings as indice in backend queries
2023-04-26 16:07:15 +02:00
Gábor Farkas 0cff917f2a
Elasticsearch: Removed reference to obsolete esVersion value (#65415)
* elastic: removed reference to obsolete esVersion value

* removed unused code

* cleaned up tests
2023-03-28 17:04:56 +03:00
Ivana Huckova e8bb894a02
Elasticsearch: Remove GetMinInterval method that is not used anymore (#65203) 2023-03-23 09:36:37 +01:00
Ivana Huckova d258c8ef8a
Elasticsearch: Implement processing of logs query results in backend (#63647)
* Elasticsearch: Add processing of logs query to backend

* Add and fix tests

* Add snapshot tests

* Fix test in ES client

* Small updates, remove redundant logic

* Refactor setPreferredVisType to improve readability
2023-03-01 05:50:56 -05:00
Ivana Huckova 772e8cbf60
Elasticsearch: Use interval provided by data request in backend (#60480)
* Elasticsearch: Remove interval caulation and use interval provises by grafana

* Remove redundant code

* Adjust snapshot tests

* Update test

* Fix lint
2023-01-05 12:26:27 +01:00
Gábor Farkas b8b2de1ac4
elastic: remove unused debug-code (#59712)
* elastic: remove unused debug-code

* removed unused structs
2022-12-05 04:21:15 -05:00
Yuriy Tseretyan d9c40ca41e
Chore: Update sqleng, elasticsearch, tempo and opentsdb plugins to support contextual logs. (#57777)
* make sql engine use pick log context for logs
* update tempo to get log context
* update opentsdb to use log context
* update es client to use log context
2022-11-02 10:03:50 -04:00
Gábor Farkas 2efd7fa481
Elasticsearch: Reuse http client in the backend (#55172)
* elastic: backend: reuse http client

* fixed lint error
2022-09-26 14:27:46 +02:00
Gábor Farkas 35253a909d
Elasticsearch: remove already-disabled code (#52932)
* elastic: removed unused code from go-part

* elastic: removed unused code from js-part
2022-08-22 16:25:20 +02:00
Jo 062d255124
Handle ioutil deprecations (#53526)
* replace ioutil.ReadFile -> os.ReadFile

* replace ioutil.ReadAll -> io.ReadAll

* replace ioutil.TempFile -> os.CreateTemp

* replace ioutil.NopCloser -> io.NopCloser

* replace ioutil.WriteFile -> os.WriteFile

* replace ioutil.TempDir -> os.MkdirTemp

* replace ioutil.Discard -> io.Discard
2022-08-10 15:37:51 +02:00
Kristin Laemmert 8f6877e12a
chore: remove all remaining uses of golang.org/x/net/context (#47564)
* chore: remove all remaining uses of  golang.org/x/net/context

This PR finishes the work started in #47532, replacing all calls to that package with the stdlib context and using http.NewRequestWithContext to include the context where necessary.

Bonus: small formatting fixes to goimports in these files.

closes #44178

* tweak: use context.Background in favor of TODO for tests
2022-04-11 14:20:10 -04:00
Will Browne 76d88a6cec
change log level to debug (#41102) 2021-11-01 09:00:00 +00:00
ying-jeanne c989f0fbbe
SQL: Migrate to use SDK contracts (#36635)
* convert SQLs to use sdk contracts

* make draft

* postgres

* intermedia

* get datasourceinfo filled at the beginning of the service

* move the interval into package because of cyclict  import and fix all postgres tests

* fix mysql test

* fix mssql

* fix the test for pr https://github.com/grafana/grafana/issues/35839

* fix some issue about intervalv2 package

* update sql test

* wire migration for SQLs

* add sqls to the background process

* make it register instead of register and start

* revert formatting

* fix tests

* fix linter

* remove integration test

* Postgres test fix

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-09-07 09:35:37 +02:00
Dimitris Sotirakis 0df1b33d71
Plugins: Migrate Elasticsearch to backend plugin SDK (#36132)
* Migrate Elasticsearch to backend plugin SDK

* Fix linting

* Move away from Convey!

* Rebase commit

* Small logger fix

* Fixes according to reviewer's comments

* Fixes according to reviewer's comments

* Fixes according to reviewer's comments

* More cleanup

* Move things around - small refactoring

* Fix typo

* Update calculator - add tests

* Fixes according to reviewer's comments
2021-07-15 16:45:59 +02:00
Giordano Ricci f6b83a4f47
Elasticsearch: Add frozen indices search support (#36018)
* Revert "Revert "Elasticsearch: add frozen indices search support (#27472)" (#27726)"

This reverts commit 4c7131425b.

* Make label width a bit more consistent

* Add documentation for X-Pack & Frozen Indices support in Elasticsearch

* Change UI & docs casing

* create default empty dataframe

* Remove backticks and simplify regex

* small doc improvement
2021-07-15 15:52:02 +02:00
Chris Cowan f580c9149c
Elasticsearch: Add Top Metrics Aggregation and X-Pack support (#33041)
* Elasticsearch: Add Top Metrics Aggregation

* Adding support for non-timeseries visualizations

* removing console.logs

* restoring loadOptions type

* Honor xpack setting

* Adding test for elastic_response

* adding test for query builder

* Adding support of alerting

* Fixing separator spelling

* Fixing linting issues

* attempting to reduce cyclomatic complexity

* Adding elastic77 Docker block

* Update public/app/plugins/datasource/elasticsearch/components/QueryEditor/MetricAggregationsEditor/MetricEditor.test.tsx

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* refactoring MetricsEditor tests

* Fixing typo

* Change getFields type & move TopMetrics to a separate component

* Fix SegmentAsync styles in TopMetrics Settings

* Fix field types for TopMetrics

* WIP

* Refactoring client side to support multiple top metrics

* Adding tests and finishing go implimentation

* removing fmt lib from debugging

* fixing tests

* reducing the cyclomatic complexity

* Update public/app/plugins/datasource/elasticsearch/elastic_response.ts

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* Update public/app/plugins/datasource/elasticsearch/hooks/useFields.ts

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* Checking for possible nil value

* Fixing types

* fix fake-data-gen param

* fix useFields hook

* Removing aggregateBy and size

* Fixing go tests

* Fixing TS tests

* fixing tests

* Fixes

* Remove date from top_metrics fields

* Restore previous formatting

* Update pkg/tsdb/elasticsearch/client/models.go

Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>

* Update pkg/tsdb/elasticsearch/client/models.go

Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>

* Fix code review comments on processTopMetricValue

* Remove underscore from variable names

* Remove intermediate array definition

* Refactor test to use testify

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>
Co-authored-by: Elfo404 <me@giordanoricci.com>
Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>
2021-06-04 11:07:59 +01:00
wangjian 530072fce7
Elasticsearch: fix NewClient not passing httpClientProvider to client impl (#34539)
* Update client.go

fix missing "httpClientProvider" in NewClient

* Update client.go
2021-05-24 14:45:29 +01:00
Marcus Efraimsson 348e76fc8e
Datasource: Shared HTTP client provider for core backend data sources and any data source using the data source proxy (#33439)
Uses new httpclient package from grafana-plugin-sdk-go introduced 
via grafana/grafana-plugin-sdk-go#328. 
Replaces the GetHTTPClient, GetTransport, GetTLSConfig methods defined 
on DataSource model.
Longer-term the goal is to migrate core HTTP backend data sources to use the 
SDK contracts and using httpclient.Provider for creating HTTP clients and such.

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-05-19 23:53:41 +02:00
Giordano Ricci e98a8bd11b
Elasticsearch: use semver strings to identify ES version (#33646)
* Elasticsearch: use proper semver strings to identify ES version

* Update BE & tests

* refactor BE tests

* refactor isValidOption check

* update test

* Update pkg/tsdb/elasticsearch/client/client.go

Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>

* Update pkg/tsdb/elasticsearch/client/search_request_test.go

Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>

* Remove leftover FIXME comment

* Add new test cases for new version format

* Docs: add documentation about version dropdown

* Update docs/sources/datasources/elasticsearch.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/elasticsearch.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/elasticsearch.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update provisioning documentation

Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
2021-05-11 09:44:00 +01:00
Giordano Ricci f7b408f99f
Elasticsearch: use application/x-ndjson content type for multisearch requests (#32282) 2021-03-29 23:41:45 +01:00
Arve Knudsen b79e61656a
Introduce TSDB service (#31520)
* Introduce TSDB service

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
2021-03-08 07:02:49 +01:00
Arve Knudsen c2cad26ca9
Chore: Disable default golangci-lint filter (#29751)
* Disable default golangci-lint filter

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Fix linter warnings

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-12-15 09:32:06 +01:00
Giordano Ricci 4c7131425b
Revert "Elasticsearch: add frozen indices search support (#27472)" (#27726)
This reverts commit 61a3160c34.
2020-09-23 12:24:46 +01:00
Arve Knudsen a5d9196a53
Chore/fix lint issues (#27704)
* Chore: Fix linting issues

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-09-22 16:22:19 +02:00
Giordano Ricci 61a3160c34
Elasticsearch: add frozen indices search support (#27472)
* Elasticsearch: Add frozen indices support for alerting
2020-09-18 18:27:25 +01:00
Arve Knudsen 5070f7a75b
Chore: Start harmonizing linting with plugin SDK (#25854)
* Chore: Harmonize linting with plugin SDK

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Fix linting issues

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-06-29 14:08:32 +02:00
Marcus Efraimsson 58cefe73ee
Datasources: Handle URL parsing error (#25742)
Adds handling of error returned from URL parsing.

Fixes #25714
2020-06-22 16:34:40 +02:00
Marcus Efraimsson 5713048f48
Alerting: Improve alert rule testing (#16286)
* tsdb: add support for setting debug flag of tsdb query

* alerting: adds debug flag in eval context

Debug flag is set when testing an alert rule and this debug
flag is used to return more debug information in test aler rule
response. This debug flag is also provided to tsdb queries so
datasources can optionally add support for returning additional
debug data

* alerting: improve test alert rule ui

Adds buttons for expand/collapse json and copy json to clipboard,
very similar to how the query inspector works.

* elasticsearch: implement support for tsdb query debug flag

* elasticsearch: embedding client response in struct

* alerting: return proper query model when testing rule
2019-06-25 08:52:17 +02:00
zhulongcheng 2fff8f77dc move log package to /infra (#17023)
ref #14679

Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-05-13 08:45:54 +02:00
Alcides Viamontes E cff2be0d66 elasticsearch: add 7.x version support (#16646)
Adds a new version option 7.0+ (70 internally).
Version 7.0+ doesn't include document types in index mappings 
so support for handling this have been added.
Version 7.0+ returns number of hits in a different way so 
support for handling this have been added.
Version 7.0+ doesn't support sending 
max_concurrent_shard_requests in multisearch header so 
support for sending this in query string have been added.
Update elastic6 docker block and dashboards (devenv) to use 
6.7.1 images, filebeat index name is now filebeat-YYYY.MM.DD 
and dashboard include correct tags and links.
Add elastic7 docker block and provisioning (devenv).
Updates documentation regarding new version.

Closes #15622
2019-04-25 09:41:13 +02:00
Andrej Ocenas 66f6e16916
Security: Store datasource passwords encrypted in secureJsonData (#16175)
* Store passwords in secureJsonData

* Revert unnecessary refactors

* Fix for nil jsonSecureData value

* Remove copied encryption code from migration

* Fix wrong field reference

* Remove migration and provisioning changes

* Use password getters in datasource proxy

* Refactor password handling in datasource configs

* Add provisioning warnings

* Update documentation

* Remove migration command, moved to separate PR

* Remove unused code

* Set the upgrade version

* Remove unused code

* Remove double reference
2019-04-15 11:11:17 +02:00
Marcus Efraimsson a022284cb0
fix handle of elasticsearch 6.0+ version 2018-11-23 15:24:16 +01:00
Mario Trangoni 13a1d0a026 pkg/tsdb/elasticsearch/client/client.go: use time.Since instead of time.Now().Sub
$ gometalinter --vendor --disable=all --enable=megacheck --deadline=10m ./...
pkg/tsdb/elasticsearch/client/client.go:147:13⚠️ should use time.Since instead of time.Now().Sub (S1012) (megacheck)
pkg/tsdb/elasticsearch/client/client.go:190:14⚠️ should use time.Since instead of time.Now().Sub (S1012) (megacheck)
pkg/tsdb/elasticsearch/client/client.go:218:13⚠️ should use time.Since instead of time.Now().Sub (S1012) (megacheck)
2018-09-18 23:42:25 +02:00
Sven Klemm 0254a29e35 Interpolate $__interval in backend for alerting with sql datasources (#13156)
add support for interpolate $__interval and  $__interval_ms in sql datasources
2018-09-13 16:51:00 +02:00
yogy rahmawan 8b32dc5847 move go vet out of scripts and fixing warning (#12552) 2018-07-09 23:25:32 -07:00
Sławek Piotrowski 9db964bf35 Fix typo: eleasticsearch -> elasticsearch (#12184) 2018-06-06 13:09:16 -07:00
Marcus Efraimsson b8ff3b1e3f
remove dead code 2018-05-31 19:05:32 +02:00
Marcus Efraimsson dcac63936b
elasticsearch: minor refactor
Handle all replacements if interval template variables in the client.
Fix issue with client and different versions.
Adds better tests of the client
2018-05-31 19:02:20 +02:00
Marcus Efraimsson 4752d7884a
elasticsearch: adds some more/better debug logging to client 2018-05-25 10:31:56 +02:00
Marcus Efraimsson 4436b8da12
elasticsearch: query interval override fix 2018-05-23 22:07:52 +02:00
Marcus Efraimsson 8870e3e85b
elasticsearch: default interval fix
5s instead of 15s
2018-05-23 21:44:09 +02:00
Marcus Efraimsson dce59ccff2
fix: remove deadcode to make gometalinter happy 2018-05-23 15:28:36 +02:00
Marcus Efraimsson e171ed8910
elasticsearch: new simple client for communicating with elasticsearch
Handles minor differences of es 2, 5 and 5.6.
Implements index pattern logic.
Exposes builders for building search requests.
2018-05-23 14:59:12 +02:00