This commit adds support data stream support to CCR's auto following by making the following changes:
* When the auto follow coordinator iterates over the candidate indices to follow,
the auto follow coordinator also checks whether the index is part of a data stream and
if the name of data stream also matches with the auto follow pattern then the index
will be auto followed.
* When following an index, the put follow api also checks whether that index is part
of a data stream and if so then also replicates the data stream definition to the
local cluster.
* In order for the follow index api to determine whether an index is part of a data
stream, the cluster state api was modified to also fetch the data stream definition
of the cluster state if only the state is queried for specific indices.
When a data stream is auto followed, only new backing indices are auto followed.
This is in line with how time based indices patterns are replicated today. This
means that the data stream isn't copied 1 to 1 into the local cluster. The local
cluster's data stream definition contains the same name, timestamp field and
generation, but the list of backing indices may be different (depending on when
a data stream was auto followed).
Closes#56259
* First crack at rewriting the CCR introduction.
* Emphasizing Kibana in configuring CCR (part one).
* Many more edits, plus new files.
* Fixing test case.
* Removing overview page and consolidating that information in the main page.
* Adding redirects for moved and deleted pages.
* Removing, consolidating, and adding redirects.
* Fixing duplicate ID in redirects and removing outdated reference.
* Adding test case and steps for recreating a follower index.
* Adding steps for managing CCR tasks in Kibana.
* Adding tasks for managing auto-follow patterns.
* Fixing glossary link.
* Fixing glossary link, again.
* Updating the upgrade information and other stuff.
* Apply suggestions from code review
* Incorporating review feedback.
* Adding more edits.
* Fixing link reference.
* Adding use cases for #59812.
* Incorporating feedback from reviewers.
* Apply suggestions from code review
* Incorporating more review comments.
* Condensing some of the steps for accessing Kibana.
* Incorporating small changes from reviewers.
Co-authored-by: debadair <debadair@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Previously, cross-cluster replication (CCR) documentation was located in
the Stack Overview:
https://www.elastic.co/guide/en/elastic-stack-overview/master/xpack-ccr.html
This adds CCR documentation to the Elasticsearch Reference Guide with a
level offset for headings.
The level offset and CCR Stack Overview docs will be removed in later
commits.
Martijn van Groningen
Adam Locke
Tanguy Leroux
James Rodewig
Jason Tedor
lcawl