The building block of the eql response is currently the SearchHit. This
is a problem since it is tied to an actual search, and thus has scoring,
highlighting, shard information and a lot of other things that are not
relevant for EQL.
This becomes a problem when doing sequence queries since the response is
not generated from one search query and thus there are no SearchHits to
speak of.
Emulating one is not just conceptually incorrect but also problematic
since most of the data is missed or made-up.
As such this PR introduces a simple class, Event, that maps nicely to
the terminology while hiding the ES internals (the use of SearchHit or
GetResult/GetResponse depending on the API used).
Fix#59764Fix#59779
Co-authored-by: Igor Motov <igor@motovs.org>
Changes:
* Moves sample data to reusable rest test
* Combines EQL index, requirements, and run a search pages
* Combines EQL syntax and limitations pages
* Adds related redirects
Instead of retrieving an entire SearchHit, get just a reference and
postpone the document retrieval when assembling the final results.
Remove sort information from results to make them consistent.
Move TumblingWindow under the sequence package.
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
Changes:
* Swaps the `dev` admonitions for `experimental` admonitions
* Removes `ifdef` statements preventing the docs from appearing in
released branches
Changes:
* Documents the `size` default as `10`.
* Updates `size` param def to note its relation to pipes.
* Updates the `head` and `tail` pipe docs to modify sequences.
* Documents the `fetch_size` parameter.
Relates to #59014 and #59063
ES EQL queries do not support the comparison of a variable, such as
a field value, to another variable.
This adds a related para and example to the EQL syntax docs.
Removes references to partial results from the async EQL search docs.
If an EQL search does not complete during the `wait_for_completion_timeout`
timeout period, it returns no results.
Adds support for async searches to eql search API. This commit is limited to
only submitting search API requests and doesn't provide APIs to get results
nor delete the results. These functions will be added in follow up PRs.
Relates to #49638
Notes that you cannot use EQL in ES to search the values of `nested`
fields or their sub-fields. However, indices containing `nested` field
mappings are otherwise supported.
Per #54411, we plan to handle case sensitivity via a parameter for the
EQL search API (with the possible exception of the `between` function).
This removes references and examples related to case sensitivity from
the EQL functions docs.
Adds a concise list of EQL advantages, based on the "EQL Advantages"
section in the [EQL for the masses][0] blog post.
The intent is to inform users how EQL could benefit at a high level.
[0]: https://www.elastic.co/blog/eql-for-the-masses
Co-Authored-By: Ross Wolf <31489089+rw-access@users.noreply.github.com>
Adds a important admonition to the EQL syntax page noting that
the equal (`==`) operator should not be used to match `text` field
values.
Relates to #52709 and #53020
Changes boilerplate sentence of "If using a field as the argument, this
parameter only supports..." to "...this parameter supports only...".
The latter is a bit more clear and readable.
EQL functions are an easy way for users to transform indexed data
at search time. However, using multiple functions can make
queries difficult to write and slows search speeds.
Users can circumvent this by indexing fields containing the transformed
data, but that usually slows index speeds.
This adds a related tip and example covering these tradeoffs.
Adds documentation for the EQL `substring` function.
Supporting changes:
* Creates a new "EQL function reference" page
* Updates the title of the "EQL syntax reference" page for consistency
* Adds a brief "Functions" section to the EQL syntax docs
* Updates EQL limitations docs to state that only array functions are
unsupported
Adds a tip admonition to the basic example in the EQL search docs.
This tip lets users know they can set up a Beat to automatically
index data in ES, rather than manually indexing using the bulk or index
APIs.
* [DOCS] Document `any` keyword in EQL syntax
Adds documentation for the `any` keyword to the EQL syntax docs.
Includes:
* Definition of an event type and its relationship to the event type
field.
* Example matching all event types using `any` keyword
* Example matching event types beginning with a digit
* Example using `any` with `where true`
* Remove references to `event_type_field` default
* Reuse "Events starting with digits" section
* Updates for #53073
Updates the documented default `event_category_field` and `timestamp_field`
values for the EQL search API. Also updates related guidance in the
EQL requirement docs.
Relates to #53073.
Makes the following updates to the EQL search tutorial:
* Adds an API response to the basic tutorial
* Adds an example using the `event_type_field` parm
* Adds an example using the `timestamp_field`parm
* Adds an example using the `query` parm
* Updates example dataset to support more EQL query variety
I plan to add additional sections to this page with future PRs:
* Specify timestamp and event type fields
* Specify a join key field
* Filter using query DSL
* Paginate a large response
See #51057.
Adds documentation for basic EQL syntax.
Joins, sequences, and other syntax to be added as its supported
in future development.
Co-Authored-By: Ross Wolf <31489089+rw-access@users.noreply.github.com>
* Creates a top-level page for EQL in the ES reference.
This page contains a high-level introduction and will include a nav for other EQL docs pages as they're built.
* Creates a requirements page.
This page outlines the fields needed to use EQL in ES.
James Rodewig
Costin Leau
Andrei Stefan
James Rodewig
DeDe Morton
Costin Leau
Igor Motov
Lisa Cawley
Aleksandr Maus