Commit Graph

6190 Commits

Author SHA1 Message Date
debadair 3d5d96da07
Added request and response body sections. Closes #16035 (#48032) 2019-10-15 11:27:46 -07:00
James Rodewig 17610e740a
[DOCS] Add `wait_for_completion` parm to execute enrich policy API docs (#48077) 2019-10-15 13:46:55 -04:00
James Rodewig 8ac99b23aa
[DOCS] Expand `updated` response parm in reindex API docs (#48008) 2019-10-15 11:16:20 -04:00
Martijn van Groningen 11d4a65853
This commits merges (#48039) the enrich feature branch,
which adds a new ingest processor, named enrich processor,
that allows document being ingested to be enriched with data from other indices.

Besides a new enrich processor, this PR adds several APIs to manage an enrich policy.
An enrich policy is in charge of making the data from other indices available to the enrich processor in an efficient manner.

Closes #32789
2019-10-15 16:26:13 +02:00
Marios Trivyzas 745699f38d
SQL: Implement DATEDIFF function (#47920)
Implement DATEDIFF/TIMESTAMPDIFF function as per the MS-SQL spec:
https://docs.microsoft.com/en-us/sql/t-sql/functions/datediff-transact-sql?view=sql-server-2017
which allows a user to substract two date/datetime fields and return the
difference in the date/time unit specified.

Closes: #47919
2019-10-15 14:42:20 +02:00
Aleh Zasypkin 8aca85bef9
Use `/api/security/saml/callback` as Kibana ACS URL. (#47999) 2019-10-15 12:57:04 +02:00
Marios Trivyzas 0516b6eaf5
SQL: Fix arg verification for DateAddProcessor (#48041)
Previously, the safety check for the 2nd argument of the DateAddProcessor was
restricting it to Integer which was wrong since we allow all non-rational
numbers, so it's changed to a Number check as it's done in other cases.

Enhanced some tests regarding the check for an integer (non-rational
argument).
2019-10-15 12:25:04 +02:00
Tanguy Leroux 58ca9eeb02
Add documentation for Pause/Resume Auto-Follower APIs (#47985)
Relates #47510

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-10-15 08:24:51 +02:00
Martijn van Groningen 85ad27e1e7
Merge remote-tracking branch 'es/master' into enrich 2019-10-15 07:19:25 +02:00
Gordon Brown ef7fdf1800
SLM Start/Stop HLRC and docs (#47966)
This commit adds HLRC support and documentation for the SLM Start and
Stop APIs, as well as updating existing documentation where appropriate.

This commit also ensures that the SLM APIs are properly included in the
HLRC documentation.
2019-10-14 15:19:49 -06:00
Martijn van Groningen ddf3bc25d8
Change how `max_matches` affects `target_field` option. (#47982)
Prior to this change the `target_field` would always be a json array
field in the document being ingested. This to take into account that
multiple enrich documents could be inserted into the `target_field`.

However the default `max_matches` is `1`. Meaning that by default
only a single enrich document would be added to `target_field` json
array field.

This commit changes this; if `max_matches` is set to `1` then the single
document would be added as a json object to the `target_field` and
if it is configured to a higher value then the enrich documents will be
added as a json array (even if a single enrich document happens to be
enriched).
2019-10-14 21:04:47 +02:00
James Rodewig 488a258453
[DOCS] Reformat docs for several snapshot lifecycle policy APIs (#47998) 2019-10-14 12:29:11 -04:00
James Rodewig 64c1dfa727
[DOCS] Remove unsupported `local` and `master_timeout` parms from cat API docs (#47933) 2019-10-14 11:38:46 -04:00
David Roberts d308095b28
[ML] Add option to stop datafeed that finds no data (#47922)
Adds a new datafeed config option, max_empty_searches,
that tells a datafeed that has never found any data to stop
itself and close its associated job after a certain number
of real-time searches have returned no data.
2019-10-14 13:26:06 +01:00
David Roberts fd83c18cc1
[ML] Add lazy assignment job config option (#47726)
This change adds:

- A new option, allow_lazy_open, to anomaly detection jobs
- A new option, allow_lazy_start, to data frame analytics jobs

Both work in the same way: they allow a job to be
opened/started even if no ML node exists that can
accommodate the job immediately. In this situation
the job waits in the opening/starting state until ML
node capacity is available. (The starting state for data
frame analytics jobs is new in this change.)

Additionally, the ML nightly maintenance tasks now
creates audit warnings for ML jobs that are unassigned.
This means that jobs that cannot be assigned to an ML
node for a very long time will show a yellow warning
triangle in the UI.

A final change is that it is now possible to close a job
that is not assigned to a node without using force.
This is because previously jobs that were open but
not assigned to a node were an aberration, whereas
after this change they'll be relatively common.
2019-10-14 12:13:01 +01:00
Martijn van Groningen e06598ba56
Merge remote-tracking branch 'es/master' into enrich 2019-10-14 10:17:18 +02:00
James Rodewig 879c2cb06b
[DOCS] Reformat get snapshot lifecycle policy API docs (#47827) 2019-10-11 12:01:20 -04:00
James Rodewig c14de778f5
[DOCS] Reformat put snapshot lifecycle policy API docs (#47811) 2019-10-11 10:56:03 -04:00
David Turner 694373294f
Allow truncation of clean translog (#47866)
Today the `elasticsearch-shard remove-corrupted-data` tool will only truncate a
translog it determines to be corrupt. However there may be other cases in which
it is desirable to truncate the translog, for instance if an operation in the
translog cannot be replayed for some reason other than corruption. This commit
adds a `--truncate-clean-translog` option to skip the corruption check on the
translog and blindly truncate it.
2019-10-11 16:43:59 +02:00
James Rodewig 8e79bf6b2b
[DOCS] Reformat reload search analyzers API docs (#47824) 2019-10-11 08:29:05 -04:00
Tanguy Leroux 8b82e62ec2
Add Pause/Resume Auto Follower APIs (#47510)
This commit adds two APIs that allow to pause and resume 
CCR auto-follower patterns:

// pause auto-follower
POST /_ccr/auto_follow/my_pattern/pause

// resume auto-follower
POST /_ccr/auto_follow/my_pattern/resume

The ability to pause and resume auto-follow patterns can be 
useful in some situations, including the rolling upgrades of 
cluster using a bi-directional cross-cluster replication scheme 
(see #46665).

This committ adds a new active flag to the AutoFollowPattern 
and adapts the AutoCoordinator and AutoFollower classes so 
that it stops to fetch remote's cluster state when all auto-follow 
patterns associate to the remote cluster are paused.

When an auto-follower is paused, remote indices that match the 
pattern are just ignored: they are not added to the pattern's 
followed indices uids list that is maintained in the local cluster 
state. This way, when the auto-follow pattern is resumed the 
indices created in the remote cluster in the meantime will be 
picked up again and added as new following indices. Indices 
created and then deleted in the remote cluster will be ignored 
as they won't be seen at all by the auto-follower pattern at 
resume time.
2019-10-11 12:33:47 +02:00
Alan Woodward 566e1b7d33
Remove type field from DocWriteRequest and associated Response objects (#47671)
This commit removes the type field from index, update and delete requests, and their
associated responses.

Relates to #41059
2019-10-11 10:23:55 +01:00
Marios Trivyzas e624bc281b
SQL: Implement DATEADD function (#47747)
Implement DATEADD/TIMESTAMPADD function as per the MS-SQL spec:
https://docs.microsoft.com/en-us/sql/t-sql/functions/dateadd-transact-sql?view=sql-server-2017
which allows a user to add/subtract specified number of specified units
to/from a date/datetime field/expression.

Closes: #47746
2019-10-10 15:24:36 +02:00
Martijn van Groningen abdbd843ad
Merge remote-tracking branch 'es/master' into enrich 2019-10-10 14:41:34 +02:00
James Rodewig 15e2de6db8
[DOCS] Document `bytes` and `time` params in cat API docs (#47672) 2019-10-10 08:29:27 -04:00
István Zoltán Szabó 448d19f0ca
[DOCS] Adds supported fields section to the PUT DFA API description (#47842) 2019-10-10 12:34:39 +02:00
David Turner 9e30a57ca5
More bootstrap docs tweaks (#47809)
Clarifies not to set `cluster.initial_master_nodes` on nodes that are joining
an existing cluster.

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-10-10 10:53:27 +02:00
Jim Ferenczi 8f9e77e6f1
Fix tag in the search request timeout option docs (#47776)
and add missing parentheses `search_timeout` param
2019-10-10 10:35:09 +02:00
Martijn van Groningen 4de71334d1
Merge remote-tracking branch 'es/master' into enrich 2019-10-10 09:12:22 +02:00
István Zoltán Szabó ab08c0cd76
[DOCS] Extends the analyzed_fields description in the PUT DFA API docs (#47791) 2019-10-09 18:13:33 +02:00
James Rodewig 17eef81f83
[DOCS] Add docs for `geo_match` enrich policy type (#47745) 2019-10-09 08:39:11 -04:00
Florian Kelbert 1092b39334 [DOCS] Correct split API request format (#47774) 2019-10-09 08:28:23 -04:00
Andrei Stefan 55f5463eee
SQL: use calendar interval of 1y instead of fixed interval for grouping by YEAR and HISTOGRAMs (#47558) 2019-10-09 11:22:41 +03:00
Martijn van Groningen 957f0fad0f
Merge remote-tracking branch 'es/master' into enrich 2019-10-09 08:48:28 +02:00
Lee Hinman 71fab46950
Add Snapshot Lifecycle Retention documentation (#47545)
* Add Snapshot Lifecycle Retention documentation

This commits adds API and general purpose documentation for SLM
retention.

Relates to #43663

* Fix docs tests

* Update default now that #47604 has been merged

* Update docs/reference/ilm/apis/slm-api.asciidoc

Co-Authored-By: Gordon Brown <gordon.brown@elastic.co>

* Update docs/reference/ilm/apis/slm-api.asciidoc

Co-Authored-By: Gordon Brown <gordon.brown@elastic.co>

* Update docs with feedback
2019-10-08 08:32:15 -06:00
James Rodewig 29c6da75db
[DOCS] Fix errors in rollover index API docs (#47702) 2019-10-08 08:34:39 -04:00
David Turner 7b652adfbf
Remove include_relocations setting (#47717)
Setting `cluster.routing.allocation.disk.include_relocations` to `false` is a
bad idea since it will lead to the kinds of overshoot that were otherwise fixed
in #46079. This setting was deprecated in #47443. This commit removes it.
2019-10-08 13:33:49 +02:00
David Turner 9d67a02a56
Deprecate include_relocations setting (#47443)
Setting `cluster.routing.allocation.disk.include_relocations` to `false` is a
bad idea since it will lead to the kinds of overshoot that were otherwise fixed
in #46079. This commit deprecates this setting so it can be removed in the next
major release.
2019-10-08 09:15:13 +02:00
Ryan Ernst 4de0d77777
Switch stored script example to script_score query (#47691)
The example use of a scoring script was incorrectly using a filter
script query, which has no scoring, and thus no _score variable
avialable. This commit converts the example doc to using the newer
script_score query.
2019-10-07 17:08:37 -07:00
Przemko Robakowski fbda3a7e19
Explicit name for doc snippets (#46951)
* Explicit name for doc snippets

This change adds option to specify explicit name for doc snippet.
This name will be used instead of line number when creating
yml file in buildRestTests task.
Stable names should improve tracking changes through history and allow
Gradle to skip tests on non-code docs changes.

* Avoid duplication in names

* Changes id declaration, more examples

* Fix names in examples

* Unit test added

* Throw exception on duplicate names

* Moved UT to Java
2019-10-07 23:06:49 +02:00
Hendrik Muhs e9e121c9ce
[Transform] move root endpoint to _transform with BWC layer (#47127)
move the main endpoint to /_transform/ from /_data_frame/transforms/ with providing backwards compatibility and deprecation warnings
2019-10-07 16:21:51 +02:00
James Rodewig 5cec47d8e3
[DOCS] Correct deprecation note in mapping docs (#47656) 2019-10-07 09:37:23 -04:00
James Rodewig e7ffacf8c0
[DOCS] Correct callouts in search template docs (#47655) 2019-10-07 09:25:03 -04:00
James Rodewig 61ff51a233
[DOCS] Reformat clear cache API docs (#46512)
* [DOCS] Reformat clear cache API docs

* iter
2019-10-07 09:14:49 -04:00
Przemyslaw Gomulka 78e7a2e728
Update deprecation logging doc with logger configuration (#47649)
Explicitly adds a configuration snippet to change logging level
2019-10-07 15:13:38 +02:00
Alan Woodward 7a622f024f
Remove types from BulkRequest (#46983)
This commit removes types entirely from BulkRequest, both as a global
parameter and as individual entries on update/index/delete lines.

Relates to #41059
2019-10-07 13:29:12 +01:00
Dimitris Athanasiou e99435a7f6
[ML] Additional outlier detection parameters (#47600)
Adds the following parameters to `outlier_detection`:

- `compute_feature_influence` (boolean): whether to compute or not
   feature influence scores
- `outlier_fraction` (double): the proportion of the data set assumed
   to be outlying prior to running outlier detection
- `standardization_enabled` (boolean): whether to apply standardization
   to the feature values
2019-10-07 15:28:21 +03:00
debadair d598083051 [DOCS] Comment out tag in Task Managment API Docs so it isn't rendered. (#47618)
The tag for the shared content is being rendered in the output.
2019-10-05 12:47:32 -04:00
debadair 07f4ca799b
Reformats term vectors APIs (#47484)
* Reformat termvectors APIs

* Reformats mtermvectors

* Apply suggestions from code review

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Incorporated review feedback.
2019-10-04 20:09:11 -07:00
debadair 6710279199
Reformats reindex API (#47483)
* Reformats reindex API

* Incorporated review feedback.
2019-10-04 19:41:46 -07:00