Since 7.15 transform automatically optimizes the grouping part of the query. We therefore can delete group_by order advise in the transform at scale documentation so it gets simpler.
Allows searching on ip fields when those fields are not indexed (index: false) but just doc values are enabled.
This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.
Relates #81210 and #52728
Closes#69533.
The Docker docs mention bind-mounting the `config`, `data` and
`logs` directories when using an arbitrary UID / GID, but they fail
to mention that the `plugins` dir must also be mounted in order to
install plugins.
* Rolling restart instructions
In a rolling restart scenario, indexing do not need to be stopped. This step is optional. I have updated Rolling restart/ step 2.
Note: I am not sure how to check if the doc, once generated, will keep the numbering as section 1 and 3 are references to the full restart, can you check this detail?
* Clarify that shard recovery can be faster
Co-authored-by: Adam Locke <adam.locke@elastic.co>
User can no longer set location for Hunspell dictionaries. `<config-dir>/hunspell` directory is silently used everytime no matter what configuration is used.
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
(cherry picked from commit 1a4fd34129)
Co-authored-by: Jan Jíša <jenda.jisa@gmail.com>
The `GET _cluster/state` API is really only suitable for debugging or
diagnostics. Its response format is not documented since it changes
fairly freely between versions.
Today we mention in its docs that this API is unstable, and deliberately
omit a description of its response format, but we don't explicitly say
that it's only for diagnostics and is unsuitable for consumption by
external tools that might try and use it for monitoring.
This commit adjusts the docs to give some more explicit guidance about
how it should and shouldn't be used.
Allows searching on boolean fields when those fields are not indexed (index: false) but just doc values are enabled.
This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.
Relates #81210 and #52728
Allows searching on keyword fields when those fields are not indexed (index: false) but just doc values are enabled.
This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.
Relates #81210 and #52728
* Upgrade documentation for xpack.monitoring.history.duration
* A few updates now that https://github.com/elastic/elasticsearch/pull/82498 has been opened with a concrete policy.
* Grammar fix
* Update docs/reference/migration/migrate_8_0/cluster-node-setting-changes.asciidoc
Co-authored-by: James Baiera <james.baiera@gmail.com>
Co-authored-by: James Baiera <james.baiera@gmail.com>
* Add support for HTTP Proxies for the GCS repository
The change adds 3 new client properties for the GCS repository:
* gcs.client.default.proxy.type
* gcs.client.default.proxy.host
* gcs.client.default.proxy.port
They allow to configure a [java.net.Proxy](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/net/Proxy.html)
for the GCS SDK to use when communicating with the GCS API.
Resolves#82444
* [DOCS] Remove extraneous Elasticsearch Docker image information
In the step for starting Elasticsearch with the generated enrollment token, `docker.elastic.co/elasticsearch/elasticsearch:` was included in conjunction with the `{docker-image}` variable. This include led to a duplicate Docker image that displayed as `ocker.elastic.co/elasticsearch/elasticsearch:docker.elastic.co/elasticsearch/elasticsearch:8.0.0-rc1`. This PR removes the duplicate image information.
* Update ifeval statements and add sub-heading for setting JVM heap size
There have been many requests to support repository-s3 authentication via IAM roles in Kubernetes service accounts.
The AWS SDK is supposed to support them out of the box with the aws-java-sdk-sts library. Unfortunately, we can't use WebIdentityTokenCredentialsProvider from the SDK. It reads the token from AWS_WEB_IDENTITY_TOKEN_FILE environment variable which is usually mounted to /var/run/secrets/eks.amazonaws.com/serviceaccount/token and the S3 repository doesn't have the read permission to read it. We don't want to hard-code a file permission for the repository, because the location of AWS_WEB_IDENTITY_TOKEN_FILE can change at any time in the future and we would also generally prefer to restrict the ability of plugins to access things outside of their config directory.
To overcome this limitation, this change adds a custom WebIdentityCredentials provider that reads the service account from a symlink to AWS_WEB_IDENTITY_TOKEN_FILE created in the repository's config directory. We expect the end user to create the symlink to indicate that they want to use service accounts for authentification.
Service accounts are checked and exchanged for session tokens by the AWS STS. To test the authentification flow, this change adds a test fixture which mocks the assume-role-with-web-identity call to the service and returns a response with test credentials.
Fixes#52625
Documents the `EMPTY` and `NONE` `flag` values for the `regexp` query.
Also documents the `""` (empty string) value, which is an alias for `ALL`.
Closes#81978.
Removing the mock nio transport and replacing its usage with the netty transport to make tests
with a more realistic transport implementation. This way improves the real world coverage for
the Netty transport, makes our tests more realistic and saves lots of code.
In particular, coverage on the rather complicated throttling/chunking in the netty message handler
is really ice to have.
The downside of this change is that we lose the slow transport thread warnings that the mock transport
outputs. This isn't a big deal these days in my opinion as we have slow logging in other places
now that makes up for this (we didn't when initially adding the slow logging) and that contains
far more detailed information on what exactly was slow.
Other than that, the mock transport does not come with any features we don't also have in the Netty
transport at this point.
Similar to #82409, but for date fields.
Allows searching on date field types (date, date_nanos) when those fields are not indexed (index: false) but just doc
values are enabled.
This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.
Relates #81210 and #52728
The simulate template api should include the settings that registered IndexSettingProvider generate.
Currently, these settings are not included in the simulate template api response,
but are only used to create a dummy IndexService instance to validate aliases.
This commit adds `search_interval` to the datafeed stats API
`running_state` object. When the datafeed is running, it reports
the last search interval that was searched. It is useful to
understand the point in time where the datafeed is currently
searching.
Closes#82405
### Changes
* Updates the snapshot version compatibility table to use minor versions rather than major versions.
* Adds a index creation version and cluster compatibility table. Updates the index compatibility section to use minor versions.
* Moves the tables to separate files. This'll help prevent merge conflicts.
* Fixes the heading level for the "Warnings" section.
Allows searching on number field types (long, short, int, float, double, byte, half_float) when those fields are not
indexed (index: false) but just doc values are enabled.
This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.
Note to reviewers:
I have split isSearchable into two separate methods isIndexed and isSearchable on MappedFieldType. The former one is
about whether actual indexing data structures have been used (postings or points), and the latter one on whether you
can run queries on the given field (e.g. used by field caps). For number field types, queries are now allowed whenever
points are available or when doc values are available (i.e. searchability is expanded).
Relates #81210 and #52728
Changes:
* Moves the get SLM policy API example _after_ the get SLM stats API. This seems to fit the normal workflow better where a user will drill-down into a particular policy to get more information.
* Notes some more information about what the get SLM policy API returns. In particular, it notes that you can get the error message for the last policy failure.
Replaces a 7.x reference with the `{prev-major-last}` variable, which will use the last minor of the previous major (currently 7.17 for 8.x).
Relates to #70451.
Changes:
* Updates the Cloud setup instructions to note that you open Kibana by default.
* Reorders the API call section to highlight Kibana.
* Fixes the Docker `ifeval` to hide some text on unreleased branches.
This enhances the migrate to data tiers routing API to also iterate over
the existing legacy, composable, and component templates and look if
they define a custom node attribute routing in their settings for either
`index.routing.allocation.require.{nodeAttrName}` or
`index.routing.allocation.include.{nodeAttrName}`. If any does, we
update them to remove all the routings settings for the provided
`nodeAttrName`.
eg. any template with the following setting configuration:
```
"settings": {
index.routing.allocation.require.data: "warm",
index.routing.allocation.include.data: "rack1",
index.routing.allocation.exclude.data: "rack2,rack3"
}
```
will have its settings updated to:
```
"settings": {}
```
David Kyle
István Zoltán Szabó
Hendrik Muhs
Yannick Welsch
James Rodewig
Stef Nestor
Adam Locke
Rory Hunter
Alex Perrin
David Turner
Mat Schaffer
Lisa Cawley
Artem Prigoda
jenish jain
Przemysław Witek
Ievgen Degtiarenko
Armin Braun
Martijn van Groningen
Dimitris Athanasiou
James Rodewig
erictung1999
Chris
Andrei Dan