elasticsearch/docs/reference
Mary Gouseti ad90d1f0f6
Introduce global retention in data stream lifecycle (cluster settings) (#111972)
In this PR we introduce cluster settings to manage the global data stream retention.

We introduce two settings `data_streams.lifecycle.retention.max` & `data_streams.lifecycle.retention.default` that configure the respective retentions. The settings are loaded and monitored by the `DataStreamGlobalRetentionSettings`. The validation has also moved there.

We preserved the `DataStreamGlobalRetention` record to reduce the impact of this change. The purpose of this method is to be simply a wrapper record that groups the retention settings together.

Temporarily, the `DataStreamGlobalRetentionSettings` is using the DataStreamFactoryRetention which is marked as deprecated for migration purposes.
2024-08-20 09:54:55 +03:00
..
aggregations [DOCS] Refactor book-scoped variables in `docs/reference/index.asciidoc` (#107413) 2024-04-17 14:37:07 +02:00
analysis Gracefully handle invalid synonym rules in updateable synonyms (#110901) 2024-08-06 10:44:23 -04:00
autoscaling Expose `?master_timeout` in autoscaling APIs (#108759) 2024-05-21 04:24:08 -04:00
behavioral-analytics/apis
cat Round up shard allocation / recovery / relocation concepts (#109943) 2024-07-25 14:44:57 -04:00
ccr [Doc+] Add Secure Connection to Setup CCR Tutorial (#103237) 2024-05-03 09:13:42 +02:00
cluster Add docs for shard level stats in node stats (#111082) 2024-08-13 14:59:03 +03:00
commands Add remove index setting command (#109276) 2024-06-03 18:23:13 -07:00
connector/apis [Connector APIs] Add docs for sync job claim endpoint (#110412) 2024-07-15 13:01:12 +02:00
data-management
data-streams Add support for boolean dimensions (#111457) 2024-07-31 23:00:32 +10:00
docs (+Doc) Link setting reference for action.auto_create_index (#110558) 2024-07-11 07:57:18 -06:00
eql [DOCS] Refactor book-scoped variables in `docs/reference/index.asciidoc` (#107413) 2024-04-17 14:37:07 +02:00
esql ESQL: Profile more timing information (#111855) 2024-08-20 07:29:01 +10:00
features/apis Simplify reset-features API (#110866) 2024-07-15 13:23:21 +01:00
fleet
graph
health [DOCS] Fixes the description of 'affected_resources' in health API documentation (#111833) 2024-08-13 18:51:49 +02:00
high-availability Round up shard allocation / recovery / relocation concepts (#109943) 2024-07-25 14:44:57 -04:00
how-to (Doc+) How to resolve shards >50GB (#111254) 2024-07-25 08:28:46 -06:00
ilm Document manual steps in ILM delete phase (#111734) 2024-08-13 08:43:04 +01:00
images [DOCS] Update Using ESQL in Kibana doc (#108715) 2024-05-17 12:36:04 +02:00
index-modules Support index sorting with nested fields (#110251) 2024-07-01 17:24:17 +10:00
indices Round up shard allocation / recovery / relocation concepts (#109943) 2024-07-25 14:44:57 -04:00
inference [DOCS] Expands inference API main page info (#111830) 2024-08-14 16:04:11 +02:00
ingest [DOCS] Documents output_field behavior after multiple inference runs (#111875) 2024-08-15 12:36:59 +02:00
licensing
mapping [DOCS] Clarify copy_to behavior with strict dynamic mappings (#111408) 2024-08-01 14:37:17 +02:00
migration Forward port release notes for v8.15.0 (#111714) 2024-08-08 18:10:39 +01:00
ml [ML] Force time shift documentation (#111668) 2024-08-09 11:12:46 +02:00
modules More detail around packet captures (#111835) 2024-08-13 21:55:38 +01:00
monitoring
query-dsl Update the intervals query docs (#111808) 2024-08-13 13:39:55 +09:00
query-rules/apis [Query rules] Add `exclude` query rule type (#111420) 2024-08-02 08:03:54 -04:00
quickstart [DOCS] Update run-elasticsearch-locally.asciidoc (#108545) 2024-05-13 10:55:39 +02:00
release-notes Add known issue docs for #111854 (#111978) 2024-08-20 07:25:55 +01:00
repositories-metering-api [DOCS] Refactor book-scoped variables in `docs/reference/index.asciidoc` (#107413) 2024-04-17 14:37:07 +02:00
rest-api Add docs for shard level stats in node stats (#111082) 2024-08-13 14:59:03 +03:00
rollup Disallow new rollup jobs in clusters with no rollup usage. (#108624) 2024-05-21 09:38:10 +02:00
scripting Aggs: Scripted metric allow list docs (#109635) 2024-07-11 14:27:51 +02:00
search LTR documentation - Remove tech preview note. (#111803) 2024-08-12 17:57:07 +02:00
search-application/apis Allow typed_keys for search application Search API (#108007) 2024-04-30 11:31:45 +02:00
searchable-snapshots Adds a warning about manually mounting snapshots managed by ILM (#111883) 2024-08-15 18:43:06 +02:00
security Clean up global retention from the cluster state and obsolete transport actions (#111636) 2024-08-08 09:52:35 +03:00
settings Introduce global retention in data stream lifecycle (cluster settings) (#111972) 2024-08-20 09:54:55 +03:00
setup (Doc+) Link Gateway Settings to Full Restart (#110902) 2024-07-25 09:10:19 -06:00
shutdown/apis [DOCS] Align docs to implementation for timeout parameters (#108593) 2024-05-16 13:05:39 +10:00
slm/apis [DOCS] Refactor book-scoped variables in `docs/reference/index.asciidoc` (#107413) 2024-04-17 14:37:07 +02:00
snapshot-restore Clean up dangling S3 multipart uploads (#111955) 2024-08-20 02:49:48 +10:00
sql Removing server from the version compatibility table (#109168) 2024-05-29 11:57:46 -06:00
synonyms/apis Clarify synonyms docs (#110822) 2024-07-18 10:20:26 +02:00
tab-widgets [DOCS] Adds elser service to the inference tutorial (#111406) 2024-07-30 14:49:58 +02:00
text-structure/apis [DOCS] Refactor book-scoped variables in `docs/reference/index.asciidoc` (#107413) 2024-04-17 14:37:07 +02:00
transform [DOCS] updates transforms at scale doc with date rounding. (#109073) 2024-05-27 16:34:01 +02:00
troubleshooting More detail around packet captures (#111835) 2024-08-13 21:55:38 +01:00
upgrade (Doc+) Link Gateway Settings to Full Restart (#110902) 2024-07-25 09:10:19 -06:00
vectors Adds new `bit` element_type for dense_vectors (#110059) 2024-06-27 04:48:41 +10:00
watcher
aggregations.asciidoc
alias.asciidoc Mention alias filters don't apply for get-by-id in docs (#108433) 2024-05-08 15:28:25 -06:00
analysis.asciidoc
api-conventions.asciidoc Add known issue docs for #111854 (#111978) 2024-08-20 07:25:55 +01:00
cat.asciidoc
cluster.asciidoc
data-management.asciidoc
data-rollup-transform.asciidoc
datatiers.asciidoc (Doc+) Flush out Data Tiers (#107981) 2024-07-18 14:35:41 -06:00
dependencies-versions.asciidoc
docs.asciidoc
geospatial-analysis.asciidoc Simple addition of ES|QL to geo overview page (#111158) 2024-07-23 12:00:05 +02:00
gs-index.asciidoc
high-availability.asciidoc
how-to.asciidoc
index-modules.asciidoc [DOCS] The logs index.mode has been renamed logsdb (#111871) 2024-08-14 08:30:31 -07:00
index.asciidoc [DOCS] Add local dev setup instructions (#107913) 2024-05-07 18:10:48 +02:00
index.x.asciidoc
indices.asciidoc
ingest.asciidoc
intro.asciidoc
landing-page.asciidoc
links.asciidoc
mapping.asciidoc
query-dsl.asciidoc Group vector queries into new section (#110722) 2024-07-11 14:45:35 +02:00
redirects.asciidoc
release-notes.asciidoc Forward port release notes for v8.14.3 (#110787) 2024-07-12 08:09:12 +01:00
scripting.asciidoc
search.asciidoc
setup.asciidoc Round up shard allocation / recovery / relocation concepts (#109943) 2024-07-25 14:44:57 -04:00
troubleshooting.asciidoc (Doc+) Capture Elasticsearch diagnostic (#108259) 2024-05-09 10:27:19 -06:00
upgrade.asciidoc [DOCS] Refactor book-scoped variables in `docs/reference/index.asciidoc` (#107413) 2024-04-17 14:37:07 +02:00