Commit Graph

49 Commits

Author SHA1 Message Date
Florian Verdonck 2c7c2088d9
Logs panel: Add meta field to show total hits; add total hits to ElasticSearch plugin response (#104117)
* feat: Show total amount of hits in Elastic Search query

* Add test with multiple series.
2025-04-28 08:32:28 -05:00
Nathan Vērzemnieks f8c43d0bf3
Elasticsearch: Decouple backend from infra/tracing (#90528) 2024-07-19 09:26:10 +02:00
Nathan Vērzemnieks 9caa8151d8
Elasticsearch: Decouple backend from infra/log (#90527) 2024-07-19 08:51:18 +02:00
Ivana Huckova 494d169980
Elasticsearch: Fix legend for alerting, expressions and previously frontend queries (#84485)
* Elasticsearch: Fix legend for alerting, expressions and previously frontend queries

* Add comment

* Update comment
2024-03-18 18:01:33 +01:00
Ivana Huckova f2936d6695
Elasticsearch: Fix creating of legend so it is backward compatible with frontend produced frames (#81708)
* Elasticsearch: Fix creating of legend so it is backward compatible with frontend produced frames

* Update tests
2024-02-02 12:47:45 +01:00
Ivana Huckova 759c088ac5
Elasticsearch: Fix showing of logs when `__source` is log message field (#80804)
Elasticsearch: Fix showing of logs whe __source is log message field
2024-01-19 10:28:08 +01:00
Ivana Huckova 28f4c3ed44
Elasticsearch: Fix processing of raw_data with not-recognized time format (#78262)
* Elasticsearch: Fix non-standard time field in raw data queries

* Update snapshot tests

* Refactor
2023-11-20 12:01:41 +01:00
Marcus Efraimsson 0128d0403f
Tracing: Use tracing.InitializeTracerForTest (#75479) 2023-09-27 09:51:57 +02:00
Ivana Huckova 4f0b31d21b
Elasticsearch: Add tracing to data source (#74750)
* Elasticsearch: Add tracing do data source

* Fix tests

* Address feedback

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>

* Track error across both spans

* Add span for decoding of response

* Fix test

* Update setting of errors + fix test

---------

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>
2023-09-18 10:49:12 +02:00
Ivana Huckova 51391a762b
Elasticsearch: Improve backend instrumentation of `QueryData` calls (#74172)
* Elasticsearch: Improve backend instrumentation of QueryData calls

* Add fromAlert

* Fix tests, move logger and context to struct

* Add instrumentation for processing response

* Move log for sending request closer to sending request

* Update

* Fix logging, improved messages, fix printing of queries

* Update log text

* Fix tests

* Fix lint

* Update logging to follow our guidelines

* Remove key-value pairs from logs that are going to be passed from instrumentation

* Update pkg/tsdb/elasticsearch/elasticsearch.go
2023-09-07 13:54:16 +02:00
Ryan McKinley 025b2f3011
Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
Sven Grossmann 4c9469fc9e
Elastic: Add `id` field to Elastic responses to allow permalinking (#73382)
* add `id` field to elasticsearch

* add comment

* slightly better perf

* only add `id` to logs frames

* only add `id` for logs responses

* concat `index` and `id`

* change snapshot generation to false

* use better loop

* fix tests

* moved up
2023-08-18 11:39:58 +02:00
Shirley c1f6b91ea9
Elasticsearch: Fix multiple max depth flatten of multi-level objects (#70302) 2023-07-14 11:48:00 +02:00
Ivana Huckova dbbbc46351
Elasticsearch: Fix passing of limit and datalinks to logs data frame (#68554)
* Elasticsearch: Fix passing of limit and datalinks to logs data frame

* Update public/app/core/logsModel.ts

Co-authored-by: François Massot <francois.massot@gmail.com>

---------

Co-authored-by: François Massot <francois.massot@gmail.com>
2023-05-17 14:28:32 +02:00
Ivana Huckova 95cf598423
Elasticsearch: Fix processing of logs with not-recognized time format (#67767)
* Elasticsearch: Fix parsing of invalid time

* Fix lint

* Add more test data to data.js

* Add tests

* Fix lint

* Update pkg/tsdb/elasticsearch/client/search_request.go
2023-05-04 19:33:00 +02:00
Ivana Huckova c308118fc0
Elasticsearch: Move response parsing tests to 1 file (#67288)
* Organize tests

* Organize

* Fix lint

* Fix lint
2023-04-26 18:21:04 +02:00
Ivana Huckova e2e243d1b6
Elasticsearch: Fix processing of duplicated metric types and field (#66973)
* Elasticsearch: Fix processing of duplicated metric types and field

* Fix lint

* Fix linting
2023-04-24 17:26:04 +02:00
Ivana Huckova 3e4eb9c93e
Elasticsearch: Fix legend generated on backend (#66039)
* Elasticsearch: Fix legend

* Remove comments

* fix adding name to config

* Fix lint
2023-04-11 13:29:38 +02:00
Ivana Huckova ae0f9fc44f
Elasticsearch: Fix processing of response with multiple group by for alerting (#65165)
* Elasticsearch: Fix processing of response with multiple group by on backend

* Remove unused param from createFields

* Switch comments
2023-03-22 18:53:09 +02:00
Ivana Huckova 0740d4e911
Elasticsearch: Fix top metrics with terms in backend (#65063)
* Elasticsearch: Fix top metrics with terms in backend

* Refactor to remove cyclomatic complexity
2023-03-22 09:36:43 +01:00
Ivana Huckova bd0fa79542
Elasticsearch: Add log highlight processing on backend (#63924)
* Elasticsearch: Add highlight for logs

* Fix running of queries trough backend only when shouldRunTroughBackend

* Fix incorrect json in test string

* Address feedback

* Keep order of words same
2023-03-09 13:34:53 +01:00
Ivana Huckova aa123e0d50
Elasticsearch: Fix where name of frame is set (#64233)
* Elasticsearch: Use displayName field for naming

* Change solution to frame.Name to be backward compatible

* Fix snapshot tests

* Use Time and Value for time and value fields

* Use variables from grafana-plugin-sdk-go for name
2023-03-08 13:32:01 +01:00
Ivana Huckova 3bd7217aa2
Elasticsearch: Remove _source field when processing raw data on backend (#64119)
* Elasticsearch: Remove _source field when processing raw data on backend

* Update snapshot test
2023-03-07 13:56:34 +01:00
Ivana Huckova adda7819e9
Elasticsearch: Implement processing of raw document query results in backend (#63932) 2023-03-02 09:50:54 +01:00
Ivana Huckova d258c8ef8a
Elasticsearch: Implement processing of logs query results in backend (#63647)
* Elasticsearch: Add processing of logs query to backend

* Add and fix tests

* Add snapshot tests

* Fix test in ES client

* Small updates, remove redundant logic

* Refactor setPreferredVisType to improve readability
2023-03-01 05:50:56 -05:00
Ivana Huckova 89b3663a23
Elasticsearch: Add processing for raw data to backend (#63208)
* WIP

* WIP

* Refactor

* Add tests

* Cleanup

* Fix whitespace

* Fix test and lint

* In snapshot tests update counter to be number

* Add boolean value for snapshot testing

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>

* Use generic to reuse logic when creating fields

* Use nullable fields

* Fix lint

* WIP (#63272)

wip

* Fix snapshot test after we changed field types to nullable

---------

Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>
2023-02-22 13:28:43 +01:00
Gábor Farkas d9fd807375
Elasticsearch: Fix consistent label order in alerting (#62497)
elasticsearch: backend: sort label-values
2023-02-02 09:03:18 +01:00
Serge Zaitsev 0bf4093005
Chore: Fix goimports grouping (#62428)
* fix goimports order

* fix goimports order
2023-01-30 08:50:27 +00:00
Gábor Farkas 57425234c9
elastic: backend: removed unused time-field (#61362)
* elastic: backend: removed unused time-field

* improve comment formatting

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>
2023-01-12 15:52:32 +00:00
Gábor Farkas 12c4098bcd
elastic: backend: handle naming bucket_scripts (#60460) 2023-01-04 15:26:57 +01:00
Ivana Huckova d332dab3ec
Elasticsearch: Refactor parse query (#60440)
* Refactor parse query to functions

* Move parsing to new file

* Create empty result variable and use it when returning early

* Fix linting

* Revert "Create empty result variable and use it when returning early"

This reverts commit 36a503f66e.
2022-12-16 16:30:36 +01:00
Gábor Farkas 5633843f76
Elasticsearch: backend: do not set name for time-fields (#60169)
elastic: backend: improve field-naming
2022-12-13 13:19:03 +01:00
Gábor Farkas 3172d8dd9b
elastic: response parser: simplify code (#59782)
elastic: simplify code
2022-12-12 16:00:15 +01:00
Gábor Farkas b8b2de1ac4
elastic: remove unused debug-code (#59712)
* elastic: remove unused debug-code

* removed unused structs
2022-12-05 04:21:15 -05:00
Giuseppe Guerra 7996a1c8ad
Chore: Update grafana-plugin-sdk-go to v0.142.0 (#58115)
* Chore: Update grafana-plugin-sdk-go to v0.142.0

* Update tests and golden files for 207 status code

* Chore: Move update flag definition at the top in response_parser_test.go

* retrigger

Co-authored-by: Will Browne <will.browne@grafana.com>
2022-11-03 18:34:27 +01:00
Gábor Farkas 0491c19712
elastic: backend-mode: fix trimEdges functionality (#56985) 2022-10-19 10:40:42 +02:00
Jorge Luis Betancourt e4f2006cce
Elasticsearch: Fix calculation of trimEdges (#56148)
* Elasticsearch: Fix calculation of trimEdges

When a value of trimEdges is set greater than 1 we need to drop both the
first and last sample of the data from the response.

* Elasticsearch: Fix reading trimEdges from the query settings

Currently the trimEdges property in the JSON panel is stored as a string
and not directly as a number.

This caused that the reading of the value failed in the go backend
because the simplejson.Int() method doesn't properly handle this case.
This failure when decoding the value goes unnoticed because of the early
return causing the trimEdges configuration to be ignored.

* Refactor castToInt to also return an error

Add a new test case that sets the `trimEdges` property as a quoted
number.
2022-10-14 12:53:12 +02:00
Dimitris Sotirakis 12320dda3c
Elasticsearch: Fix metric names for alert queries (#38546)
* Export fieldName to var

* Add empty string as frame name
2021-08-26 09:20:38 +02:00
Dimitris Sotirakis 0df1b33d71
Plugins: Migrate Elasticsearch to backend plugin SDK (#36132)
* Migrate Elasticsearch to backend plugin SDK

* Fix linting

* Move away from Convey!

* Rebase commit

* Small logger fix

* Fixes according to reviewer's comments

* Fixes according to reviewer's comments

* Fixes according to reviewer's comments

* More cleanup

* Move things around - small refactoring

* Fix typo

* Update calculator - add tests

* Fixes according to reviewer's comments
2021-07-15 16:45:59 +02:00
Dimitris Sotirakis 0d08881876
Elasticsearch: Convert Timeseries and Tables to Dataframes (#34710)
* Rebase (broken tests)

* Removed tables - refactored processAggregationDocs func

* Tests cleanup

* Nit - add space in percentile legend titles

* Fix labels naming - use metricAggType map

* Fix bug which appended same fields over and over again

* Replace test with dataframes

* Fix tests

* Add nolint:gocyclo - will need refactoring

* Move frames tags assignment

* Add failing test fo when ES server is booting up

* Revert "Add failing test fo when ES server is booting up"

This reverts commit fd14a1fd5e.

Co-authored-by: Elfo404 <me@giordanoricci.com>
2021-06-18 13:26:19 +03:00
Chris Cowan f580c9149c
Elasticsearch: Add Top Metrics Aggregation and X-Pack support (#33041)
* Elasticsearch: Add Top Metrics Aggregation

* Adding support for non-timeseries visualizations

* removing console.logs

* restoring loadOptions type

* Honor xpack setting

* Adding test for elastic_response

* adding test for query builder

* Adding support of alerting

* Fixing separator spelling

* Fixing linting issues

* attempting to reduce cyclomatic complexity

* Adding elastic77 Docker block

* Update public/app/plugins/datasource/elasticsearch/components/QueryEditor/MetricAggregationsEditor/MetricEditor.test.tsx

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* refactoring MetricsEditor tests

* Fixing typo

* Change getFields type & move TopMetrics to a separate component

* Fix SegmentAsync styles in TopMetrics Settings

* Fix field types for TopMetrics

* WIP

* Refactoring client side to support multiple top metrics

* Adding tests and finishing go implimentation

* removing fmt lib from debugging

* fixing tests

* reducing the cyclomatic complexity

* Update public/app/plugins/datasource/elasticsearch/elastic_response.ts

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* Update public/app/plugins/datasource/elasticsearch/hooks/useFields.ts

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* Checking for possible nil value

* Fixing types

* fix fake-data-gen param

* fix useFields hook

* Removing aggregateBy and size

* Fixing go tests

* Fixing TS tests

* fixing tests

* Fixes

* Remove date from top_metrics fields

* Restore previous formatting

* Update pkg/tsdb/elasticsearch/client/models.go

Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>

* Update pkg/tsdb/elasticsearch/client/models.go

Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>

* Fix code review comments on processTopMetricValue

* Remove underscore from variable names

* Remove intermediate array definition

* Refactor test to use testify

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>
Co-authored-by: Elfo404 <me@giordanoricci.com>
Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>
2021-06-04 11:07:59 +01:00
Arve Knudsen b79e61656a
Introduce TSDB service (#31520)
* Introduce TSDB service

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
2021-03-08 07:02:49 +01:00
Marcelo Nunes Alves be961c5466
Elasticsearch: Fix using multiple bucket script aggregations when only grouping by terms (#24064)
* In the parser when it was configurate more bucket it put the formula in name column

* Tests

* Tests

* Tests

* Tests

* Tests

* Tests

* Process/Tests - backend (Go)

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* reverse

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2020-07-09 15:21:19 +02:00
Arve Knudsen 5070f7a75b
Chore: Start harmonizing linting with plugin SDK (#25854)
* Chore: Harmonize linting with plugin SDK

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Fix linting issues

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-06-29 14:08:32 +02:00
Marcus Efraimsson 5713048f48
Alerting: Improve alert rule testing (#16286)
* tsdb: add support for setting debug flag of tsdb query

* alerting: adds debug flag in eval context

Debug flag is set when testing an alert rule and this debug
flag is used to return more debug information in test aler rule
response. This debug flag is also provided to tsdb queries so
datasources can optionally add support for returning additional
debug data

* alerting: improve test alert rule ui

Adds buttons for expand/collapse json and copy json to clipboard,
very similar to how the query inspector works.

* elasticsearch: implement support for tsdb query debug flag

* elasticsearch: embedding client response in struct

* alerting: return proper query model when testing rule
2019-06-25 08:52:17 +02:00
Marcus Efraimsson b45f72a140
elasticsearch: support bucket script pipeline aggregations 2018-12-20 01:24:36 +01:00
Marcus Efraimsson 4840adff00
elasticsearch: refactor query handling and use new es simple client
Removes moment dependency.
Adds response parser tests (based on frontend tests).
Adds time series query tests (based on frontend tests).
Fixes various issues related to response parsing and building search request queries.
Added support for extended stats metrics and geo hash grid aggregations.
2018-05-23 15:09:58 +02:00
Marcus Efraimsson 77400cef08
elasticsearch: refactor and cleanup
Move time series query logic to specific file.
Remove model parser and move to time series query file, adds parser test.
2018-05-23 14:36:41 +02:00
wph95 4050fce220
add response_parser test 2018-03-28 12:35:05 +08:00