This commit upgrades the Azure SDK to 12.11.0 and Jackson to 2.12.2. The
Jackson upgrade must happen at the same time due to Azure depending on
this new version of Jackson.
closes#66555closes#67214
Co-authored-by: Francisco Fernández Castaño <francisco.fernandez.castano@gmail.com>
In 7.14+, you can create data stream aliases. This updates the related security documentation. It also streamlines the docs to remove redundant examples.
Fixes the incorrect assumption in the snapshot state machine that a finished
snapshot delete could only start shard snapshots: in fact it can also move
snapshots to a completed state.
Currently when attempting to an alias to points to both data streams and regular indices
the alias does get created, but points only to data streams. With this change attempting
to add such aliases results in a client error.
Currently when adding data stream aliases with unsupported parameters (e.g. filter or routing)
the alias does get created, but without the unsupported parameters. With this change
attempting to create aliases to point to data streams with unsupported parameters will result
in a client error.
Relates to #66163
This object should be completely immutable. Also added a useful
assertion that makes sure we don't accidentally overwrite a valid
generation with `null` when dealing dealing failed status updates.
In #70625 we added the total data set size of shards
to the Indices Stats API and we enhanced the test
testCreateAndRestorePartialSearchableSnapshot to
also verify the correctness of this data set size.
Because restoring a searchable snapshot shard
creates a new in-memory segment size, the
verification of the data set size was implemented
in an approximative fashion: between the
expected size and twice the expected size. This
approximation sometimes fails for shards that
have no documents indexed (see #73194).
This commit changes the test so that it now
verifies the exact data set size returned by the
Indices Stats API, which should be the sum of
the original expected size of the snapshotted
size + the length of the extra segment file in
memory.
Closes#73194
Add authentication.token.type to audit log. This is to complement the
authentication.token.name field added by #72198. The log is now unambiguous
about exactly where the service token is from. It also helps if we decide to
log additional information for other types of tokens in future.
Relates: #73135, #72198
Previously, we would always return 0 total hits when there were no groups. Now
that collapsing supports search_after, it's possible for total hits to be
greater than 0 but no groups to return.
This PR also fixes a test bug where we set the wrong missing value for sorting
on doubles.
Fixes#73270.
* [DOCS] Changing event table to a list for easier viewing
* Make audit event types more readable
* Adding system_access_granted user and cross-links
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
When a search or search_quote analyzer on a text mapper is not defined,
we fallback to a configured default search/search_quote analyzer if it
exists. However, if an index analyzer has been configured on the mapper
then we should first fall back to that.
Fixes#73333
Memory estimates for categorization were increased in #68859,
but testing has shown some data sets where categorization
legitimately creates far more categories that usual.
This change doubles the categorization memory estimate to
20MB per partition, and also applies the multiplier when
stop-on-warn is disabled to single partition categorization
(where it is not possible to enable stop-on-warn).
The Grok pattern creator used within the get categories action
works by looking at the examples for each category. Sometimes
these examples are truncated, and cannot be used for Grok pattern
determination. Previously when this happened we would log a warning,
but that caused a feedback loop in the case where the Elasticsearch
logs themselves were being categorized: the warning messages would
end up creating new categories in the Elasticsearch log categorization,
with very long examples that would be truncated, leading to the same
problem occurring yet again.
This change reduces the warning log to trace for truncated log
messages, and also removes the example from the log message to
shorten it.
This method is taking about 4% of CPU time with internal cluster tests
for me. 80% of that were coming from the slow immutability assertion,
the rest was due to the slow way we were building up the new map.
The CPU time slowness likely translates into outright test slowness,
because this was mainly hit through adding transport handlers when starting
nodes (which happens on the main test thread).
Fixed both to save a few % of test runtime.
When data stream aliases are resolved then the includeDataStreams flag of an action request should be taken into account,
so that data stream aliases aren't resolved to backing indices for apis that don't support data streams.
Closes#73195
Unfreezing a snapshot backed index does not work well
because the unfreeze action always removes the
index.blocks.write block, causing shards to be failed
when the cluster state is applied on data nodes. This
is because searchable snapshots shards always
expect the index.blocks.write to be set to true.
This commit changes the freeze/unfreeze action
so that the write block is not removed when
unfreezing searchable snapshots indices. It also
changes the toggling of index.frozen and
index.search.throttled settings so that they are
just removed (instead of being turned to false)
when unfreezing.
Previously, when a subquery was used with an alias in combination with
a nested GROUP BY, the collapsing of the nested queries into a flattened
`Aggregate` query, lead to wrong attribute qualifier on the external
projection, which was still referencing the removed subquery. e.g.:
For the following query:
```
SELECT languages FROM (
SELECT languages FROM test_emp GROUP BY languages
) AS subquery
```
The `languages` of the top level SELECT, was qualified with `subquery`
which was removed during the flattening optimisation leading to
Exception of not being able to resolve the refenced group:
`test_emp.languages`.
Fix this behaviour by introducing a new rule which precedes the
`PruneSubqueryAliases` rules and updates the `qualifier` for the
`FieldAttributes`.
Fixes: #69263
This changes the result of AuthorizationEngine.loadAuthorizedIndices
(and dependent methods) from List<String> to Set<String>.
This has the following performance benefits:
1. `contains` checks are faster
2. RBACEngine always formed this collections as a Set, so this
change reduces unnecessary copying.
An additional performance improvement was added when resolve authorized
index names for data streams.
Service token of the same name can come from either a file or the
security index. Add the token source information to the authentication
metadata to differentiate between them. This information is also
serialised under token.type in the rest response back to users.
* Validate that system indices aren't also hidden inidices
* Remove hidden from ingest geo system index
* Add test coverage
* Remove hidden setting from system index even if not upgrading
The repository analyzer API spec was incorrectly stored in the plugin
directory rather than in the main `rest-api-spec` directory. This commit
fixes that.
* [DOCS] Expand information on when to use a runtime field without a script
* Reworking information based on review feedback
* Clarify case where doc_values are disabled
* A few minor changes from review feedback
Mark Vieira
James Rodewig
Rene Groeschke
Ryan Ernst
Armin Braun
Adam Locke
Martijn van Groningen
Yang Wang
Tanguy Leroux
Julie Tibshirani
Julie Tibshirani
David Turner
Ignacio Vera
Alan Woodward
David Roberts
Przemysław Witek
István Zoltán Szabó
Marios Trivyzas
Tamara Braun
William Brafford