The simulate template api should include the settings that registered IndexSettingProvider generate.
Currently, these settings are not included in the simulate template api response,
but are only used to create a dummy IndexService instance to validate aliases.
Reverts an anchor change from #46711.
Previous versions of the docs use the `_shrinking_an_index` anchor for this
section. Preserving that anchor will prevent doc build breaks in future releases.
Force merge action is a very costly action. It may take several hours to run for big indices. But current force merge rest api do not support wait_for_completion parameter.
This adds support for the wait_for_completion parameter.
Internally we already kept track of whether a data stream is replicated by CCR.
It is part of the `DataStream` class. This just adds it to the xcontent serialization
of the get data stream api response class.
Relates to elastic/kibana#118899
Removes `testenv` annotations and related code. These annotations originally let you skip x-pack snippet tests in the docs. However, that's no longer possible.
Relates to #79309, #31619
We document that `GET /_index_template/...` accepts a comma-separated
list of template names but in fact today this API accepts only a single
name or pattern. Likewise `GET /_cat/templates/...` (at least it didn't
until #78829 but that's not released yet). This commit fixes the docs to
indicate these APIs accept only a single template name and also adds
some extra validation to reject requests containing a `,` since such a
request cannot match any actual templates.
It also adjusts `GET /_cat/templates` to use the filtering built into
`TransportGetComposableIndexTemplateAction` rather than retrieving all
templates and then filtering them on the coordinating node.
The 'verbose' option to /_segments returns memory information
for each segment. However, lucene 9 has stopped tracking this memory
information as it is largely held off-heap and so is no longer significant.
This commit deprecates the 'verbose' parameter and makes it a no-op.
Fixes#75955
* [DOCS] Update remote cluster docs
* Add files, rename files, write new stuff
* Plethora of changes
* Add test and update snippets
* Redirects, moved files, and test updates
* Moved file to x-pack for tests
* Remove older CCS page and add redirects
* Cleanup, link updates, and some rewrites
* Update image
* Incorporating user feedback and rewriting much of the remote clusters page
* More changes from review feedback
* Numerous updates, including request examples for CCS and Kibana
* More changes from review feedback
* Minor clarifications on security for remote clusters
* Incorporate review feedback
Co-authored-by: Yang Wang <ywangd@gmail.com>
* Some review feedback and some editorial changes
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Yang Wang <ywangd@gmail.com>
Related to issue #77823
This does the following:
- Updates several asciidoc files that contained code snippets with
invalid JSON, most involving unnecessary trailing commas.
- Makes the switch from the Groovy JSON parser to the Jackson parser,
pursuant to the general goal of eliminating Groovy dependence.
- Makes testing of JSON validity at build time more strict.
Note that this update still allows backslash escaping for any
character. Currently that matters because of the file
"docs/reference/ml/anomaly-detection/apis/get-datafeed-stats.asciidoc",
specifically this part:
"attributes" : {
"ml.machine_memory" :
"$body.datafeeds.0.node.attributes.ml\.machine_memory",
"ml.max_open_jobs" : "512"
}
It's not clear to me what change, if any, is appropriate there. So,
I've left in the escaped period and configured the parser to ignore
it for the time being.
This commit changes the es merge policy to apply the maximum segment size
on force merges that only expunge deletes (forceMergeDeletes).
This option is useful for read-write use cases that wants to reclaim deleted docs
more aggressively than the `index.merge.policy.deletes_pct_allowed`.
Closes#61764
Relates #77270
Changes:
* Notes the delete index API can delete multiple indices at once.
* Notes deleting an index deletes its docs, shards, and metadata but does not delete any related Kibana components.
* Relocates a note about deleting a data stream's write index to the description.
* Corrects the default `expand_wildcards` value.
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
This updates the default for the `is_write_index` parameter of the aliases API and create alias API.
The default behavior for `is_write_index` can vary based on:
1. Whether the alias is used for indices or data streams.
2. If an index alias, whether the alias points to multiple indices.
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Co-authored-by: 诗心客 <ishixinke@qq.com>
PRs #73062 and #73043 repurposed the `alias` anchor for a new guide for index
and data stream aliases. Previously, this anchor was used for our field alias
documentation.
Repurposing the anchor has caused continuity errors for users selecting
different versions of the ES docs. It could also cause confusion for users with
a `/current/` link to the `alias` page.
This updates the anchor for the alias guide and adds a redirect page to
disambiguate the `alias` anchor.
It also fixes a bread crumb issue for redirects following the 'Modifying your
Data' redirect page.
Closes#77034.
Adds new field to recovery API to keep track of amount of data
recovered from snapshots.
The normal recovered_bytes field remains and is also increased for
recovery from snapshot but can go backwards in the unlikely case
that recovery from snapshot fails to download a file.
Relates #73496
Changes:
* Use "geopoint" when not referring to the literal field type
* Use "geoshape" when not referring to the literal field type or query type
* Use "GeoJSON" consistently
Adds a field usage API that reports shard-level statistics about which Lucene fields have been accessed, and which
parts of the Lucene data structures have been accessed.
Field usage statistics are automatically captured when queries are runnning on a cluster. A shard-level search request
that accesses a given field, even if multiple times during that request, is counted as a single use.
Clarifies that you cannot specify an alias in the delete index API. You _can_ delete indices with an alias.
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
You can now use a wildcard pattern to remove data stream and index
aliases in the same action/request.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
In #74138 we noted that index settings aren't copied in a clone. In fact
that's not true, we copy everything except explicitly-excluded ones,
`number_of_replicas` and `auto_expand_replicas`. This fixes the mistake.
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Changes:
* Reuses the same `aliases` object properties in the following API docs:
* Clone index API
* Create index API
* Put component template API
* Put legacy index template API
* Put index template API
* Rollover index API
* Shrink index API
* Simulate template API
* Split index API
* Updates the `aliases` object properties for the simulate index API docs.
Closes#73044
Changes:
* Expands the `aliases` parameter for the create index API to better document
supported properties.
* Reuses `aliases` parameter in the following API docs:
* Clone index API
* Shrink index API
* Split index API
Updates an outdated reference to empty `data_stream` objects. The create index
template API's `data_stream` parameter now supports the `hidden` property.
Changes:
* Reuses and reorders the index template API's body parameters in the simulate template API docs.
* Replaces several includes with a shorter xref.
* Reformats a sidebar on naming collisions with built-in index templates.
Updates the exists API docs to better reflect its support of data streams and
aliases.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
The get alias api should take into account the aliases parameter when
returning aliases that refer to data streams and don't return entries
for data streams that don't have any aliases pointing to it.
Relates to #66163
Aliases to data streams can be defined via the existing update aliases api.
Aliases can either only refer to data streams or to indices (not both).
Also the existing get aliases api has been modified to support returning
aliases that refer to data streams.
Aliases for data streams are stored separately from data streams and
and refer to data streams by name and not to the backing indices of
a data stream. This means that when backing indices are added or removed
from a data stream that then the data stream alias doesn't need to be
updated.
The authorization model for aliases that refer to data streams is the
same as for aliases the refer to indices. In security privileges can
be defined on aliases, indices and data streams. When a privilege is
granted on an alias then access is also granted on the indices that
an alias refers to (irregardless whether privileges are granted or denied
on the actual indices). The same will apply for aliases that refer
to data streams. See for more details:
https://github.com/elastic/elasticsearch/issues/66163#issuecomment-824709767
Relates to #66163
This commit adds support for system data streams and also the first use
of a system data stream with the fleet action results data stream. A
system data stream is one that is used to store system data that users
should not interact with directly. Elasticsearch will manage these data
streams. REST API access is available for external system data streams
so that other stack components can store system data within a system
data stream. System data streams will not use the system index read and
write threadpools.
Today the response to `GET _cluster/state` does not include the roles of
the nodes in the cluster. In the past this made sense, roles were
relatively unchanging things that could be determined from elsewhere.
These days we have an increasingly rich collection of roles, with
nontrivial BWC implications, so it is important for debugging to be able
to see the specific roles as viewed by the master. This commit adds the
role names to the cluster state API output.
Relates #71385
This PR sets the default value of `action.destructive_requires_name`
to `true.` Fixes#61074. Additionally, we set this value explicitly in
test classes that rely on wildcard deletions to clear test state.
Add support to delete component templates api to specify multiple template
names separated by a comma.
Change the cleanup template logic for rest tests to remove all component templates via a single delete component template request. This to optimize the cleanup logic. After each rest test we delete all templates. So deleting templates this via a single api call (and thus single cluster state update) saves a lot of time considering the number of rest tests.
Older versions don't support component / composable index templates
and/or data streams. Yet the test base class tries to remove objects
after each test, which adds a significant number of lines to the
log files (which slows the tests down). The ESRestTestCase will
now check whether all nodes have a specific version and then decide
whether data streams and component / composable index templates will
be deleted.
Also ensured that the logstash-index-template and security-index-template
aren't deleted between tests, these templates are builtin templates that
ES will install if missing. So if tests remove these templates between tests
then ES will add these template back almost immediately. These causes
many log lines and a lot of cluster state updates, which slow tests down.
Relates to #69973
Co-authored-by: Lee Hinman <dakrone@users.noreply.github.com>
Add support to delete composable index templates api to specify multiple template
names separated by a comma.
Change to cleanup template logic for rest tests to remove all composable index templates via a single delete composable index template request. This to optimize the cleanup logic. After each rest test we delete all templates. So deleting templates this via a single api call (and thus single cluster state update) saves a lot of time considering the number of rest tests.
If this pr is accepted then I will do the same change for the delete component template api.
Relates to #69973
Dan Hermann
Martijn van Groningen
James Rodewig
mushaoqiong
Yannick Welsch
James Rodewig
Stef Nestor
David Turner
xiaoping
Bo Andersen
Alan Woodward
Adam Locke
edh-oss
Jim Ferenczi
Philipp Krenn
Francisco Fernández Castaño
Adrien Grand
Liam MacPherson
Nhat Nguyen
Robin Clarke
Alexander Reelsen
Pierre Grimaud
Jay Modi
Yash Jipkate
Tommmster
Joe Gallo
Pius
bellengao