adds a health section to the transform stats endpoint and implements reporting assignment, indexing/search and persistence problems, together with a overall health state.
* Adding Getting Help section
Add getting help section in the troubleshooting guide to be pointed by health API when issues are too complicated to be addressed.
This is taken from https://www.elastic.co/guide/en/cloud/current/ec-get-help.html, someone might want to elaborate it a bit more?
* Fix broken partintro, modify headings, and update wording
Co-authored-by: Adam Locke <adam.locke@elastic.co>
When creating and updating transforms, it is possible for clients to provide secondary headers.
When PUT, _preview, _update is called with secondary authorization headers, those are then used or stored with the transform.
closes: https://github.com/elastic/elasticsearch/issues/86731
It was previously required that the _start API caller required the same roles as the create API caller.
This does not make sense as when the transform is actually running (after _start) we rely solely on the roles of the caller who created the transform.
Consequently, this commit does the permission validations and various checks with the roles of user who created the transform, not the one calling _start
When a transform has a `retention_policy` it needs to be able to delete documents in the destination index.
`create_index` does not necessitate that we can delete documents from it. So, even if we create the index, we need to verify that we can delete documents given the `retention_policy` definition.
This is not a crucial bug as the transform will simply fail later. Its nicer to fail sooner.
closes https://github.com/elastic/elasticsearch/issues/85409
Since 7.15 transform automatically optimizes the grouping part of the query. We therefore can delete group_by order advise in the transform at scale documentation so it gets simpler.
Removes `testenv` annotations and related code. These annotations originally let you skip x-pack snippet tests in the docs. However, that's no longer possible.
Relates to #79309, #31619
Add an _upgrade endpoint to bulk upgrade transforms. _upgrade rewrites all transforms and its
artifacts into the latest format to the latest storage(index). If all transforms are upgraded old
indices and outdated documents get deleted. Using the dry_run option it is possible to check if
upgrades are necessary without applying changes.
Przemysław Witek
Lisa Cawley
István Zoltán Szabó
Hendrik Muhs
eltomello
Leaf-Lin
Benjamin Trent
James Rodewig