2019-08-07 20:28:09 +08:00
|
|
|
|
[role="xpack"]
|
2019-09-21 04:26:57 +08:00
|
|
|
|
[[update-transform]]
|
2020-07-23 01:55:48 +08:00
|
|
|
|
= Update {transform} API
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
|
|
|
|
[subs="attributes"]
|
|
|
|
|
++++
|
2019-12-18 01:01:31 +08:00
|
|
|
|
<titleabbrev>Update {transform}</titleabbrev>
|
2019-08-07 20:28:09 +08:00
|
|
|
|
++++
|
|
|
|
|
|
2019-12-18 01:01:31 +08:00
|
|
|
|
Updates certain properties of a {transform}.
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2019-09-21 04:26:57 +08:00
|
|
|
|
[[update-transform-request]]
|
2020-07-23 01:55:48 +08:00
|
|
|
|
== {api-request-title}
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2019-10-07 22:21:51 +08:00
|
|
|
|
`POST _transform/<transform_id>/_update`
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2019-09-21 04:26:57 +08:00
|
|
|
|
[[update-transform-prereqs]]
|
2020-07-23 01:55:48 +08:00
|
|
|
|
== {api-prereq-title}
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2021-05-06 21:19:28 +08:00
|
|
|
|
Requires the following privileges:
|
2020-04-06 23:06:22 +08:00
|
|
|
|
|
2022-03-31 21:41:26 +08:00
|
|
|
|
* cluster: `manage_transform` (the `transform_admin` built-in role grants this
|
2021-05-06 21:19:28 +08:00
|
|
|
|
privilege)
|
|
|
|
|
* source indices: `read`, `view_index_metadata`
|
2022-03-31 21:41:26 +08:00
|
|
|
|
* destination index: `read`, `index`. If a `retention_policy` is configured, `delete` index privilege is
|
|
|
|
|
also required.
|
2020-04-06 23:06:22 +08:00
|
|
|
|
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2019-09-21 04:26:57 +08:00
|
|
|
|
[[update-transform-desc]]
|
2020-07-23 01:55:48 +08:00
|
|
|
|
== {api-description-title}
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2019-12-18 01:01:31 +08:00
|
|
|
|
This API updates an existing {transform}. The list of properties that you can
|
|
|
|
|
update is a subset of the list that you can define when you create a {transform}.
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2019-12-18 01:01:31 +08:00
|
|
|
|
When the {transform} is updated, a series of validations occur to ensure its
|
|
|
|
|
success. You can use the `defer_validation` parameter to skip these checks.
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2019-12-18 01:01:31 +08:00
|
|
|
|
All updated properties except description do not take effect until after the
|
2021-02-09 15:00:47 +08:00
|
|
|
|
{transform} starts the next checkpoint. This is so there is data consistency in
|
|
|
|
|
each checkpoint.
|
2019-12-18 01:01:31 +08:00
|
|
|
|
|
|
|
|
|
[IMPORTANT]
|
|
|
|
|
====
|
|
|
|
|
|
2022-05-17 22:25:23 +08:00
|
|
|
|
* Your {transform} remembers which roles the user who updated it had at the time
|
|
|
|
|
of update and runs with those privileges. If you provide
|
2022-05-16 22:13:21 +08:00
|
|
|
|
<<http-clients-secondary-authorization,secondary authorization headers>>, those
|
|
|
|
|
credentials are used instead.
|
2022-06-09 21:22:06 +08:00
|
|
|
|
* You must use {kib} or this API to update a {transform}. Directly updating any
|
|
|
|
|
{transform} internal, system, or hidden indices is not supported and may cause
|
2022-05-23 16:10:03 +08:00
|
|
|
|
permanent failure.
|
2019-12-18 01:01:31 +08:00
|
|
|
|
|
|
|
|
|
====
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2019-09-21 04:26:57 +08:00
|
|
|
|
[[update-transform-path-parms]]
|
2020-07-23 01:55:48 +08:00
|
|
|
|
== {api-path-parms-title}
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2019-09-21 04:26:57 +08:00
|
|
|
|
`<transform_id>`::
|
2019-12-18 01:01:31 +08:00
|
|
|
|
(Required, string)
|
2020-06-02 02:46:10 +08:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-id]
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2019-09-21 04:26:57 +08:00
|
|
|
|
[[update-transform-query-parms]]
|
2020-07-23 01:55:48 +08:00
|
|
|
|
== {api-query-parms-title}
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
|
|
|
|
`defer_validation`::
|
2021-12-03 05:28:28 +08:00
|
|
|
|
(Optional, Boolean) When `true`, deferrable validations are not run. This
|
|
|
|
|
behavior may be desired if the source index does not exist until after the
|
|
|
|
|
{transform} is updated.
|
|
|
|
|
|
|
|
|
|
`timeout`::
|
|
|
|
|
(Optional, time)
|
|
|
|
|
Period to wait for a response. If no response is received before the timeout
|
|
|
|
|
expires, the request fails and returns an error. Defaults to `30s`.
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2020-03-31 01:08:38 +08:00
|
|
|
|
[role="child_attributes"]
|
2019-09-21 04:26:57 +08:00
|
|
|
|
[[update-transform-request-body]]
|
2020-07-23 01:55:48 +08:00
|
|
|
|
== {api-request-body-title}
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
|
|
|
|
`description`::
|
2019-09-16 23:28:19 +08:00
|
|
|
|
(Optional, string) Free text description of the {transform}.
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2020-03-31 01:08:38 +08:00
|
|
|
|
//Begin dest
|
2019-08-07 20:28:09 +08:00
|
|
|
|
`dest`::
|
2019-12-18 01:01:31 +08:00
|
|
|
|
(Optional, object)
|
2020-06-02 02:46:10 +08:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=dest]
|
2020-03-31 01:08:38 +08:00
|
|
|
|
+
|
|
|
|
|
.Properties of `dest`
|
|
|
|
|
[%collapsible%open]
|
|
|
|
|
====
|
|
|
|
|
|
|
|
|
|
`index`:::
|
2019-12-18 01:01:31 +08:00
|
|
|
|
(Required, string)
|
2020-06-02 02:46:10 +08:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=dest-index]
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2020-03-31 01:08:38 +08:00
|
|
|
|
`pipeline`:::
|
2019-12-18 01:01:31 +08:00
|
|
|
|
(Optional, string)
|
2020-06-02 02:46:10 +08:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=dest-pipeline]
|
2020-03-31 01:08:38 +08:00
|
|
|
|
====
|
|
|
|
|
//End dest
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
|
|
|
|
`frequency`::
|
2019-12-18 01:01:31 +08:00
|
|
|
|
(Optional, <<time-units, time units>>)
|
2020-06-02 02:46:10 +08:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=frequency]
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2021-10-15 14:12:03 +08:00
|
|
|
|
//Begin _meta
|
|
|
|
|
`_meta`::
|
|
|
|
|
(Optional, object)
|
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-metadata]
|
|
|
|
|
//End _meta
|
|
|
|
|
|
2021-02-09 15:00:47 +08:00
|
|
|
|
//Begin retention policy
|
|
|
|
|
`retention_policy`::
|
|
|
|
|
(Optional, object)
|
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention]
|
|
|
|
|
+
|
|
|
|
|
.Properties of `retention_policy`
|
|
|
|
|
[%collapsible%open]
|
|
|
|
|
====
|
|
|
|
|
`time`:::
|
|
|
|
|
(Required, object)
|
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention-time]
|
|
|
|
|
+
|
|
|
|
|
.Properties of `time`
|
|
|
|
|
[%collapsible%open]
|
|
|
|
|
=====
|
|
|
|
|
`field`:::
|
|
|
|
|
(Required, string)
|
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention-time-field]
|
|
|
|
|
|
|
|
|
|
`max_age`:::
|
|
|
|
|
(Required, <<time-units, time units>>)
|
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-retention-time-max-age]
|
|
|
|
|
=====
|
|
|
|
|
====
|
|
|
|
|
//End retention policy
|
|
|
|
|
|
2020-05-14 23:38:57 +08:00
|
|
|
|
//Begin settings
|
|
|
|
|
`settings`::
|
|
|
|
|
(Optional, object)
|
2020-06-02 02:46:10 +08:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-settings]
|
2020-05-14 23:38:57 +08:00
|
|
|
|
+
|
|
|
|
|
.Properties of `settings`
|
|
|
|
|
[%collapsible%open]
|
|
|
|
|
====
|
2020-12-07 22:34:28 +08:00
|
|
|
|
`dates_as_epoch_millis`:::
|
|
|
|
|
(Optional, boolean)
|
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-settings-dates-as-epoch-milli]
|
2020-05-14 23:38:57 +08:00
|
|
|
|
`docs_per_second`:::
|
|
|
|
|
(Optional, float)
|
2020-06-02 02:46:10 +08:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-settings-docs-per-second]
|
2021-08-18 19:58:50 +08:00
|
|
|
|
`align_checkpoints`:::
|
|
|
|
|
(Optional, boolean)
|
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-settings-align-checkpoints]
|
2022-01-18 16:01:23 +08:00
|
|
|
|
`deduce_mappings`:::
|
|
|
|
|
(Optional, boolean)
|
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-settings-deduce-mappings]
|
2020-05-14 23:38:57 +08:00
|
|
|
|
`max_page_search_size`:::
|
|
|
|
|
(Optional, integer)
|
2020-06-02 02:46:10 +08:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-settings-max-page-search-size]
|
2022-06-09 21:22:06 +08:00
|
|
|
|
`num_failure_retries`:::
|
|
|
|
|
(Optional, integer)
|
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=transform-settings-num-failure-retries]
|
2020-05-14 23:38:57 +08:00
|
|
|
|
====
|
|
|
|
|
//End settings
|
|
|
|
|
|
2020-03-31 01:08:38 +08:00
|
|
|
|
//Begin source
|
2019-08-07 20:28:09 +08:00
|
|
|
|
`source`::
|
2019-12-18 01:01:31 +08:00
|
|
|
|
(Optional, object)
|
2020-06-02 02:46:10 +08:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=source-transforms]
|
2020-03-31 01:08:38 +08:00
|
|
|
|
+
|
|
|
|
|
.Properties of `source`
|
|
|
|
|
[%collapsible%open]
|
2020-07-23 01:55:48 +08:00
|
|
|
|
====
|
2020-03-31 01:08:38 +08:00
|
|
|
|
|
|
|
|
|
`index`:::
|
2019-12-18 01:01:31 +08:00
|
|
|
|
(Required, string or array)
|
2020-06-02 02:46:10 +08:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=source-index-transforms]
|
2020-12-07 22:34:28 +08:00
|
|
|
|
|
2020-03-31 01:08:38 +08:00
|
|
|
|
`query`:::
|
2019-12-18 01:01:31 +08:00
|
|
|
|
(Optional, object)
|
2020-06-02 02:46:10 +08:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=source-query-transforms]
|
2020-03-31 01:08:38 +08:00
|
|
|
|
====
|
|
|
|
|
//End source
|
|
|
|
|
|
|
|
|
|
//Begin sync
|
2019-08-07 20:28:09 +08:00
|
|
|
|
`sync`::
|
2019-12-18 01:01:31 +08:00
|
|
|
|
(Optional, object)
|
2020-06-02 02:46:10 +08:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=sync]
|
2020-03-31 01:08:38 +08:00
|
|
|
|
+
|
2021-04-29 22:57:58 +08:00
|
|
|
|
--
|
|
|
|
|
NOTE: You can update these properties only if it is a continuous {transform}. You
|
|
|
|
|
cannot change a batch {transform} into a continuous {transform} or vice versa.
|
|
|
|
|
Instead, clone the {transform} in {kib} and add or remove the `sync` property.
|
|
|
|
|
|
|
|
|
|
--
|
|
|
|
|
+
|
2020-03-31 01:08:38 +08:00
|
|
|
|
.Properties of `sync`
|
|
|
|
|
[%collapsible%open]
|
|
|
|
|
====
|
|
|
|
|
|
|
|
|
|
//Begin sync.time
|
|
|
|
|
`time`:::
|
2019-12-18 01:01:31 +08:00
|
|
|
|
(Required, object)
|
2020-06-02 02:46:10 +08:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=sync-time]
|
2020-03-31 01:08:38 +08:00
|
|
|
|
+
|
|
|
|
|
.Properties of `time`
|
|
|
|
|
[%collapsible%open]
|
|
|
|
|
=====
|
2019-12-18 01:01:31 +08:00
|
|
|
|
|
2020-03-31 01:08:38 +08:00
|
|
|
|
`delay`::::
|
2019-12-18 01:01:31 +08:00
|
|
|
|
(Optional, <<time-units, time units>>)
|
2020-06-02 02:46:10 +08:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=sync-time-delay]
|
2019-12-18 01:01:31 +08:00
|
|
|
|
|
2020-03-31 01:08:38 +08:00
|
|
|
|
`field`::::
|
2019-12-18 01:01:31 +08:00
|
|
|
|
(Required, string)
|
2020-06-02 02:46:10 +08:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=sync-time-field]
|
2019-08-07 20:28:09 +08:00
|
|
|
|
+
|
|
|
|
|
--
|
|
|
|
|
TIP: In general, it’s a good idea to use a field that contains the
|
2021-03-16 00:22:57 +08:00
|
|
|
|
<<access-ingest-metadata,ingest timestamp>>. If you use a different field,
|
2019-08-07 20:28:09 +08:00
|
|
|
|
you might need to set the `delay` such that it accounts for data transmission
|
|
|
|
|
delays.
|
|
|
|
|
|
|
|
|
|
--
|
2020-03-31 01:08:38 +08:00
|
|
|
|
=====
|
|
|
|
|
//End sync.time
|
|
|
|
|
====
|
|
|
|
|
//End sync
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2019-09-21 04:26:57 +08:00
|
|
|
|
[[update-transform-example]]
|
2020-07-23 01:55:48 +08:00
|
|
|
|
== {api-examples-title}
|
2019-08-07 20:28:09 +08:00
|
|
|
|
|
2019-09-10 01:13:41 +08:00
|
|
|
|
[source,console]
|
2019-08-07 20:28:09 +08:00
|
|
|
|
--------------------------------------------------
|
2019-10-07 22:21:51 +08:00
|
|
|
|
POST _transform/simple-kibana-ecomm-pivot/_update
|
2019-08-07 20:28:09 +08:00
|
|
|
|
{
|
|
|
|
|
"source": {
|
|
|
|
|
"index": "kibana_sample_data_ecommerce",
|
|
|
|
|
"query": {
|
|
|
|
|
"term": {
|
|
|
|
|
"geoip.continent_name": {
|
|
|
|
|
"value": "Asia"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
},
|
|
|
|
|
"description": "Maximum priced ecommerce data by customer_id in Asia",
|
|
|
|
|
"dest": {
|
|
|
|
|
"index": "kibana_sample_data_ecommerce_transform_v2",
|
|
|
|
|
"pipeline": "add_timestamp_pipeline"
|
|
|
|
|
},
|
|
|
|
|
"frequency": "15m",
|
|
|
|
|
"sync": {
|
|
|
|
|
"time": {
|
|
|
|
|
"field": "order_date",
|
|
|
|
|
"delay": "120s"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
--------------------------------------------------
|
|
|
|
|
// TEST[setup:simple_kibana_continuous_pivot]
|
|
|
|
|
|
2019-09-16 23:28:19 +08:00
|
|
|
|
When the {transform} is updated, you receive the updated configuration:
|
2019-09-07 02:05:36 +08:00
|
|
|
|
|
|
|
|
|
[source,console-result]
|
2019-08-07 20:28:09 +08:00
|
|
|
|
----
|
|
|
|
|
{
|
2022-07-05 00:51:59 +08:00
|
|
|
|
"id" : "simple-kibana-ecomm-pivot",
|
|
|
|
|
"authorization" : {
|
|
|
|
|
"roles" : [
|
|
|
|
|
"superuser"
|
|
|
|
|
]
|
|
|
|
|
},
|
|
|
|
|
"version" : "8.4.0",
|
|
|
|
|
"create_time" : 1656113450613,
|
|
|
|
|
"source" : {
|
|
|
|
|
"index" : [
|
|
|
|
|
"kibana_sample_data_ecommerce"
|
|
|
|
|
],
|
|
|
|
|
"query" : {
|
|
|
|
|
"term" : {
|
|
|
|
|
"geoip.continent_name" : {
|
|
|
|
|
"value" : "Asia"
|
2019-08-07 20:28:09 +08:00
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
},
|
2022-07-05 00:51:59 +08:00
|
|
|
|
"dest" : {
|
|
|
|
|
"index" : "kibana_sample_data_ecommerce_transform_v2",
|
|
|
|
|
"pipeline" : "add_timestamp_pipeline"
|
|
|
|
|
},
|
|
|
|
|
"frequency" : "15m",
|
|
|
|
|
"sync" : {
|
|
|
|
|
"time" : {
|
|
|
|
|
"field" : "order_date",
|
|
|
|
|
"delay" : "120s"
|
|
|
|
|
}
|
|
|
|
|
},
|
|
|
|
|
"pivot" : {
|
|
|
|
|
"group_by" : {
|
|
|
|
|
"customer_id" : {
|
|
|
|
|
"terms" : {
|
|
|
|
|
"field" : "customer_id"
|
2019-08-07 20:28:09 +08:00
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
},
|
2022-07-05 00:51:59 +08:00
|
|
|
|
"aggregations" : {
|
|
|
|
|
"max_price" : {
|
|
|
|
|
"max" : {
|
|
|
|
|
"field" : "taxful_total_price"
|
2019-08-07 20:28:09 +08:00
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
},
|
2022-07-05 00:51:59 +08:00
|
|
|
|
"description" : "Maximum priced ecommerce data by customer_id in Asia",
|
|
|
|
|
"settings" : { }
|
2019-08-07 20:28:09 +08:00
|
|
|
|
}
|
|
|
|
|
----
|
2022-07-05 00:51:59 +08:00
|
|
|
|
// TESTRESPONSE[s/"version" : "8.4.0"/"version" : $body.version/]
|
|
|
|
|
// TESTRESPONSE[s/"create_time" : 1656113450613/"create_time" : $body.create_time/]
|
|
|
|
|
// TESTRESPONSE[s/"authorization" : \{[^}]*\},//]
|