You can't update the `analyzer` parameter in the PUT mappings API even if
the index is closed. This adds a TIP to call that out. And adds a TIP
for `search_quote_analyzer` which you *can* update.
The endpoint `_snapshottable_features` is long and implies incorrect
things about this API - it is used not just for snapshots, but also for
the upcoming reset API. Following discussions on the team, this commit
changes the endpoint to `_features` and removes the connection between
this API and snapshots, as snapshots are not the only use for the output
of this API.
We expect runtime fields to perform a little better than our "native"
aggregation script so we should point folks to them instead of the
"native" aggregation script.
* Support audit ignore policy by index privileges
Adding new audit ignore policy - privileges
For example, following policy will filter out all events, which actions
minimal required privilege is either "read" or "delete":
xpack.security.audit.logfile.events.ignore_filters:
example:
privileges: ["read", "delete"]
Resolve: #60877
Related: #10836
Related: #37148
* Support audit ignore policy by index privileges
Adding new audit ignore policy - privileges
For example, following policy will filter out all events, which actions
required privilege is either "read" or "delete":
xpack.security.audit.logfile.events.ignore_filters:
example:
privileges: ["read", "delete"]
Resolve: #60877
Related: #10836
Related: #37148
* To avoid ambiguity (as cluster and index policies may have the same
name) changing implementation to have to separate policies for
`index_privileges` and `cluster_privileges`.
If both are set for the same policy, throw the IllegalArgumentException.
* To avoid ambiguity (as cluster and index policies may have the same
name) changing implementation to have to separate policies for
`index_privileges` and `cluster_privileges`.
If both are set for the same policy, throw the IllegalArgumentException.
* Fixing Api key related privilege check which expects request and
authentication by introducing overloaded
version of findPrivilegesThatGrant
just checking if privileges which can grant the action regardless of the
request and authentication context.
* Fixing a test; adding a caching mechanism to avoid calling
findPrivilegesThatGrant each
time.
* Support audit ignore policy by index privileges
Addressing review feedback
* Support audit ignore policy by index privileges
Addressing review comments + changing approach:
- use permission check instead of simple "checkIfGrants"
- adding more testing
* Support audit ignore policy by index privileges
Addressing review comments + changing approach:
- use permission check instead of simple "checkIfGrants"
- adding more testing
* Support audit ignore policy by index privileges
Addressing review comments + changing approach:
- use permission check instead of simple "checkIfGrants"
- adding more testing
* Support audit ignore policy by index privileges
Addressing review comments + changing approach:
- use permission check instead of simple "checkIfGrants"
- adding more testing
* Revert "Support audit ignore policy by index privileges"
This reverts commit 152821e7
* Revert "Support audit ignore policy by index privileges"
This reverts commit 79649e9a
* Revert "Support audit ignore policy by index privileges"
This reverts commit 96d22a42
* Revert "Support audit ignore policy by index privileges"
This reverts commit 67574b2f
* Revert "Support audit ignore policy by index privileges"
This reverts commit 35573c8b
* Revert "Fixing a test; adding a caching mechanism to avoid calling findPrivilegesThatGrant each time."
This reverts commit 7faa52f3
* Revert "Fixing Api key related privilege check which expects request and authentication by introducing overloaded version of findPrivilegesThatGrant just checking if privileges which can grant the action regardless of the request and authentication context."
This reverts commit 72b9aefe
* Revert "To avoid ambiguity (as cluster and index policies may have the same name) changing implementation to have to separate policies for `index_privileges` and `cluster_privileges`. If both are set for the same policy, throw the IllegalArgumentException."
This reverts commit 7dd8fe7d
* Revert "To avoid ambiguity (as cluster and index policies may have the same name) changing implementation to have to separate policies for `index_privileges` and `cluster_privileges`. If both are set for the same policy, throw the IllegalArgumentException."
This reverts commit cb5bc09c
* Revert "Support audit ignore policy by index privileges"
This reverts commit a918da10
* Support audit ignore policy by actions
Getting back to action filtering
* Support audit ignore policy by actions
Cleaning up some tests
* Support audit ignore policy by actions
Cleaning up some tests
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This commit adds a new `_preview` endpoint for data frame analytics.
This allows users to see the data on which their model will be trained. This is especially useful
in the arrival of custom feature processors.
The API design is a similar to datafeed `_preview` and data frame analytics `_explain`.
Adds support for the include_unloaded_segments flag in node stats, which helps with understanding resource usage of
shared_cache-style searchable snapshots on a per-node basis.
This commit moves away from the static `rollup-{indexName}` rollup index
naming strategy and moves towards a randomized rollup index name scheme.
This will reduce the complications that exist if the RollupStep fails and retries
in any way. A separate cleanup will still be required for failed temporary indices,
but at least there will not be a conflict.
This commit generates the new rollup index name in the LifecycleExecutionState so
that it can be used in RollupStep and UpdateRollupIndexPolicyStep on a per-index
basis.
This adds additional statistics into the usage API for data frame analytics
and trained models.
For data frame analytics the added stats are:
- count of jobs by analysis type
- stats for peak_usage_bytes
For trained models the added stats are:
- counts of: total, prepackaged, other (not created by data frame analytics)
- counts by analysis type based on the inference config
- stats for estimated heap usage
- stats for estimated number of operations
This commit adds support for the Gold+ licensed `geo_line` aggregation.
This aggregation takes a collection of `geo_point` values and constructs a line
according to some sort value. Adding to transforms allows users to create these
potentially expensive lines out of band of visualizations and then do additional aggs/queries
against the pivoted data.
Examples would be:
"Do these daily user paths ever intersect?"
"Does this path enter and leave this area?"
* [DOCS] Adding grok support for runtime fields.
* Update response.
* Adding testresponse replacements.
* Update runtime field context and add dissect.
* Fixing backslash in the response.
* Fixing testresponse.
* Incorporating review feedback.
* Updates emit and adds cross link from ES runtime fields page.
To avoid confusion for the users replace the `YYYY` and `uuuu` year
patterns in the examples of `DATETIME_FORMAT/PARSE` with the most common
`yyyy` to avoid any confusion for users that might just copy paste those
queries for their own use case.
Relates to #68030
Today if an index is set to `auto_expand_replicas: N-all` then we will
try and create a shard copy on every node that matches the applicable
allocation filters. This conflits with shard allocation awareness and
the same-host allocation decider if there is an uneven distribution of
nodes across zones or hosts, since these deciders prevent shard copies
from being allocated unevenly and may therefore leave some unassigned
shards.
The point of these two deciders is to improve resilience given a limited
number of shard copies but there is no need for this behaviour when the
number of shard copies is not limited, so this commit supresses them in
that case.
Closes#54151Closes#2869
Autoscaling expects data tiers to be used exclusively both for node
roles and in ILM policies. This commit adds a test demonstrating that
as well as documentation for the behavior.
Users can now specify runtime mappings as part of the source config
of a data frame analytics job. Those runtime mappings become part of
the mapping of the destination index. This ensures the fields are
accessible in the destination index even if the relevant data frame
analytics job gets deleted.
Closes#65056
Nik Everett
Joe Gallo
James Rodewig
Mike Barretta
Gordon Brown
Andrei Stefan
Lisa Cawley
Lyudmila Fokina
Benjamin Trent
Yannick Welsch
José Arthur Benetasso Villanova
RomainGeffraye
David Turner
István Zoltán Szabó
Tal Levy
Adam Locke
Dimitris Athanasiou
Igor Motov
Marios Trivyzas
Henning Andersen
Jean-Louis Leysens