Commit Graph

142 Commits

Author SHA1 Message Date
Adam Locke 82bfbe1195
[DOCS] Adding headers in TOC for aggregation docs. (#66604) 2020-12-18 11:31:42 -05:00
Igor Motov a065b6d8da
Return an error when a rate aggregation cannot calculate bucket sizes (#65429)
In some cases when the rate aggregation is not a child of a date histogram
aggregation, it is not possible to determine the actual size of the date
histogram bucket. In this case the rate aggregation now throws an exception.

Closes #63703
2020-11-25 10:05:51 -05:00
Tal Levy b514d9bf2e
Add geo_line aggregation (#41612)
A metric aggregation that aggregates a set of points as 
a GeoJSON LineString ordered by some sort parameter.

#### specifics

A `geo_line` aggregation request would specify a `geo_point` field, as well
as a `sort` field. `geo_point` represents the values used in the LineString, 
while the `sort` values will be used as the total ordering of the points.

the `sort` field would support any numeric field, including date.

#### sample usage

```
{
	"query": {
		"bool": {
			"must": [
				{ "term": { "person": "004" } },
				{ "term": { "trajectory": "20090131002206.plt" } }
			]
		}
	},
	"aggs": {
		"make_line": {
			"geo_line": {
				"point": {"field": "location"},
				"sort": { "field": "timestamp" },
                                "include_sort": true,
                                "sort_order": "desc",
                                "size": 15
			}
		}
	}
}
```

#### sample response

```
{
    "took": 21,
    "timed_out": false,
    "_shards": {...},
    "hits": {...},
    "aggregations": {
        "make_line": {
            "type": "LineString",
            "coordinates": [
                [
                    121.52926194481552,
                    38.92878997139633
                ],
                [
                    121.52922699227929,
                    38.92876998055726
                ],
             ]
        }
    }
}
```

#### visual response

<img width="540" alt="Screen Shot 2019-04-26 at 9 40 07 AM" src="https://user-images.githubusercontent.com/388837/56834977-cf278e00-6827-11e9-9c93-005ed48433cc.png">

#### limitations

Due to the cardinality of points, an initial max of 10k points 
will be used. This should support many use-cases.

One solution to overcome this limitation is to keep a PriorityQueue of
points, and simplifying the line once it hits this max. If simplifying
makes sense, it may be a nice option, in general. The ability to use a parameter
to specify how aggressive one wants to simplify. This parameter could be 
the number of points. Example algorithm one could use with a PriorityQueue:
https://bost.ocks.org/mike/simplify/. This would still require O(m) space, where m
is the number of points returned. And would also require heapifying triangles
sorted by their areas, which would be O(log(m)) operations. Since sorting is done, 
anyways, simplifying would still be a O(n log(m)) operation, where n is the total number 
of points to filter........... something to explore


closes #41649
2020-11-23 10:26:27 -08:00
Mark Tozzi f666ccb3bc
Add supports for upper and lower values on boxplot based on the IQR value (#63617) 2020-11-04 14:39:05 -05:00
James Rodewig 2e9f95aa73
[DOCS] Change agg titles to sentence case (#64425) 2020-10-30 13:25:21 -04:00
Igor Motov e6c70f6811
Add value_count mode to rate agg (#63687)
Adds a new value count mode to the rate aggregation.

Closes #63575
2020-10-15 18:00:44 -04:00
Igor Motov 34bff3f776
Add support for histogram fields to rate aggregation (#63289)
The rate aggregation now supports histogram fields. At the moment only sum
is supported. 

Closes #62939
2020-10-08 16:54:25 -04:00
Christos Soulios b857768bb5
Histogram field type support for min/max aggregations (#62532)
Implement min/max aggregations for histogram fields.

Closes #60951
2020-09-19 23:34:43 +03:00
Julie Tibshirani f29c743a47
Support the 'fields' option in inner_hits and top_hits. (#62259)
This PR adds support for the 'fields' option in the following places:
* Anytime `inner_hits` is used, for both fetching nested/ child docs and field collapsing
* The `top_hits` aggregation

Addresses #61949.
2020-09-14 10:08:58 -07:00
Igor Motov f107dba741
Add rate aggregation (#61369)
Adds a new rate aggregation that can calculate a document rate for buckets
of a date_histogram.

Closes #60674
2020-08-25 11:32:20 -04:00
James Rodewig 456c37b186
[DOCS] Add usage tips to `top_hits` agg (#61215) 2020-08-17 12:42:04 -04:00
Adam Locke fdc867e395
[DOCS] Update info about geo_shape bounding boxes (#61214)
* Adding information about geo_shape bounding boxes.

* Fixing cross link and incorporating review feedback.
2020-08-17 11:07:18 -04:00
James Rodewig a94e5cb7c4
[DOCS] Replace Wikipedia links with attribute (#61171) 2020-08-17 09:44:24 -04:00
James Rodewig 6b9b8c5e31
[DOCS] Move script and stored fields content to search fields page (#60826)
Changes:

* Moves `Retrieve selected fields` to its own page and adds a title abbreviation.
* Adds existing script and stored fields content to `Retrieve selected fields`
* Adds a xref for `Retrieve selected fields` to `Search your data`
* Adds related redirects and updates existing xrefs
2020-08-06 12:45:03 -04:00
James Rodewig 929033f9dd
[DOCS] Move named query content to bool query (#60748) 2020-08-05 13:27:10 -04:00
James Rodewig a4dc336c16
[DOCS] Replace `twitter` dataset in search/agg docs (#60667) 2020-08-04 13:31:52 -04:00
Alexander Reelsen c7ac9e7073
[DOCS] http -> https, remove outdated plugin docs (#60380)
Plugin discovery documentation contained information about installing
Elasticsearch 2.0 and installing an oracle JDK, both of which is no
longer valid.

While noticing that the instructions used cleartext HTTP to install
packages, this commit replaces HTTPs links instead of HTTP where possible.

In addition a few community links have been removed, as they do not seem
to exist anymore.
2020-07-31 15:58:38 -04:00
James Rodewig d5b03f668b
[DOCS] Move search sort docs to separate page (#60123)
Moves the search sort docs from the deprecated 'Request Body Search'
page to a new subpage of 'Run a search'.

No substantive changes were made to the content.
2020-07-23 12:58:57 -04:00
Howard b8e3ba783a
[DOCS] Fix missing punctuation in agg docs (#59822) 2020-07-21 10:17:59 -04:00
James Rodewig 2c5d6e9c95
[DOCS] Reformat agg snippets to use two-space indents (#59912) 2020-07-20 15:08:04 -04:00
James Rodewig 8a57800f1b
[DOCS] Add performance warning for scripts (#59890) 2020-07-20 14:04:35 -04:00
James Rodewig aa3ddfeefb
[DOCS] Move highlighting docs to separate page (#59768)
Moves the highlighting docs from the deprecated 'Request Body Search'
chapter to the new subpage of the 'Run a search chapter' section.

No substantive changes were made to the content.
2020-07-17 10:15:20 -04:00
Cris da Rocha b5de14d3f6
Missing comma between value types (#58383)
This applies to all versions of this document (7.7, 7.8, 7.x, current and master).
2020-06-19 23:01:25 +02:00
Tal Levy c765993d82
add geo_shape documentation for supported aggregations (#58284)
This commit adds documentation for geo_shape fields in aggregations

Closes #55495.
2020-06-18 10:17:49 -07:00
James Rodewig 7826bbee87
[DOCS] Move search API's `docvalue_fields` examples (#57760)
Changes:

* Condenses and relocates the `docvalue_fields` example to the 'Run a search' 
   page.
* Adds docs for the `docvalue_fields` request body parameter.
* Updates several related xrefs.

Co-authored-by: debadair <debadair@elastic.co>
2020-06-11 10:57:15 -04:00
andrewjohnson2 a791d6723d
Added standard deviation / variance sampling to extended stats (#49782)
Per 49554 I added standard deviation sampling and variance sampling to the extended stats interface.

Closes #49554

Co-authored-by: Igor Motov <igor@motovs.org>
2020-06-10 15:00:50 -04:00
James Rodewig 51e3d5ab63
[DOCS] Fix source filtering xrefs (#57720) 2020-06-05 08:46:26 -04:00
Christos Soulios caf6c5ac19
Histogram field type support for ValueCount and Avg aggregations (#55933)
Implements value_count and avg aggregations over Histogram fields as discussed in #53285

- value_count returns the sum of all counts array of the histograms
- avg computes a weighted average of the values array of the histogram by multiplying each value with its associated element in the counts array
2020-05-04 10:24:35 +03:00
Christos Soulios cefc6af25b
Histogram field type support for Sum aggregation (#55681)
Implements Sum aggregation over Histogram fields by summing the value of each bucket multiplied by their count as requested in #53285
2020-04-29 11:09:25 +03:00
Igor Motov 6d28596ead
Add support for filters to T-Test aggregation (#54980)
Adds support for filters to T-Test aggregation. The filters can be used to
select populations based on some criteria and use values from the same or
different fields.

Closes #53692
2020-04-10 10:19:07 -04:00
Igor Motov 5fc9fc528d
Add Student's t-test aggregation support (#54469)
Adds t_test metric aggregation that can perform paired and unpaired two-sample
t-tests. In this PR support for filters in unpaired is still missing. It will
be added in a follow-up PR.

Relates to #53692
2020-04-03 11:31:13 -04:00
Gil Raphaelli 4090568797
[DOCS] Fix typos in top metrics agg docs (#54299) 2020-03-27 10:48:01 -04:00
Paweł Krześniak de1229cc2b
[DOCS] link fix (#53973)
Fix bad link in top_metrics.
2020-03-23 13:28:43 -04:00
Zachary Tong 84a59f8447
Add scripting, supported-type tests to ValueCount (#53500)
Also adds a few small notes to the documentation regarding potentially
unintuitive behavior
2020-03-16 15:15:25 -04:00
Lisa Cawley 4a5feab88d
[DOCS] Add anchors for scripted metric aggregations (#53618) 2020-03-16 12:14:01 -07:00
Nik Everett 230a9a8975
Improve top_metrics docs (#53521)
* Removes experimental.
* Replaces `"v"` (for value) with `"m"` (for metric).
* Move the note about tiebreaking into the list of limitations of the
  sort.
* Explain how you ask for `metrics`.
* Clean up some wording.
* Link to the docs from `top_metrics`.

Closes #51813
2020-03-16 13:23:22 -04:00
Nik Everett 8410356c5b
Preserve metric types in top_metrics (#53288)
This changes the `top_metrics` aggregation to return metrics in their
original type. Since it only supports numerics, that means that dates,
longs, and doubles will come back as stored, with their appropriate
formatter applied.
2020-03-11 16:44:08 -04:00
Nik Everett 56058ab6af
Support multiple metrics in `top_metrics` agg (#52965)
This adds support for returning multiple metrics to the `top_metrics`
agg. It looks like:
```
POST /test/_search?filter_path=aggregations
{
  "aggs": {
    "tm": {
      "top_metrics": {
        "metrics": [
          {"field": "v"},
          {"field": "m"}
        ],
        "sort": {"s": "desc"}
      }
    }
  }
}
```
2020-03-05 06:53:37 -05:00
Nik Everett f4223b6a8f
Add size support to `top_metrics` (#52662)
This adds support for returning the top "n" metrics instead of just the
very top.

Relates to #51813
2020-02-27 11:14:57 -05:00
Nik Everett 5b2266601b
Implement top_metrics agg (#51155)
The `top_metrics` agg is kind of like `top_hits` but it only works on
doc values so it *should* be faster.

At this point it is fairly limited in that it only supports a single,
numeric sort and a single, numeric metric. And it only fetches the "very
topest" document worth of metric. We plan to support returning a
configurable number of top metrics, requesting more than one metric and
more than one sort. And, eventually, non-numeric sorts and metrics. The
trick is doing those things fairly efficiently.

Co-Authored by: Zachary Tong <zach@elastic.co>
2020-02-14 07:13:52 -05:00
Igor Motov 0898df4aac
Add histogram field type support to boxplot aggs (#52265)
Add support for the histogram field type to boxplot aggs.

Closes #52233
Relates to #33112
2020-02-13 08:59:44 -05:00
Igor Motov c50cfa0668
Add Boxplot Aggregation (#51948)
Adds a `boxplot` aggregation that calculates min, max, medium and the first
and the third quartiles of the given data set.

Closes #33112
2020-02-07 18:01:20 -05:00
Mark Tozzi 928c663ce0
Fix dangling 'either' in weighted average docs (#51748) 2020-01-31 12:45:46 -05:00
Igor Motov 23be11cf6c
Fix leftover mentions of method parameter in Percentile Aggs (#51272)
The method parameter is not used in the percentile aggs, instead
the method is determined by the presence of `hdr` or `tdigest`
objects.

Relates to #8324
2020-01-22 05:02:48 -10:00
James Rodewig 7f35bcdfc9
[DOCS] Warn about using `geo_centroid` as sub-agg to `geohash_grid` (#50038)
If `geo_point fields` are multi-valued, using `geo_centroid` as a
sub-agg to `geohash_grid` could result in centroids outside of bucket
boundaries.

This adds a related warning to the geo_centroid agg docs.
2020-01-06 07:45:49 -06:00
James Rodewig 3460dc9542
[DOCS] Percentile aggs are non-deterministic (#50468)
Percentile aggregations are non-deterministic. A percentile aggregation
can produce different results even when using the same data.

Based on [this discuss post][0], the non-deterministic property stems
from processes in Lucene that can affect the order in which docs are
provided to the aggregation.

This adds a warning stating that the aggregation is non-deterministic
and what that means.

[0]: https://discuss.elastic.co/t/different-results-for-same-query/111757
2019-12-23 13:11:31 -05:00
James Rodewig 2d9ee5ddfe
[DOCS] Correct percentile rank agg example response (#50052)
The example snippets in the percentile rank agg docs use a test dataset
named `latency`, which is generated from docs/gradle.build.

At some point the dataset and example snippets were updated, but the
text surrounding the snippets was not. This means the text and the
example snippets shown no longer match up.

This corrects that by changing the snippets using /TESTRESPONSE magic comments.
2019-12-12 08:38:48 -05:00
Ignacio Vera eade4f03f4
New Histogram field mapper that supports percentiles aggregations. (#48580)
This commit adds  a new histogram field mapper that consists in a pre-aggregated format of numerical data to be used in percentiles aggregations.
2019-11-28 13:58:20 +01:00
Christos Soulios b0e12c936b
Implement stats aggregation for string terms (#47468)
This PR adds a new metric aggregation called string_stats that operates on string terms of a document and returns the following:

    min_length: The length of the shortest term
    max_length: The length of the longest term
    avg_length: The average length of all terms
    distribution: The probability distribution of all characters appearing in all terms
    entropy: The total Shannon entropy value calculated for all terms

This aggregation has been implemented as an analytics plugin.
2019-11-14 16:07:54 +02:00
Ian Danforth 24cf883792 [DOCS] Fix typo in percentile rank aggregation docs (#47247) 2019-10-15 15:56:32 -04:00