Boolean-only privilege checks, i.e. the ones currently used in the
"profile has privilege" API, now benefit from a performance improvement,
because the check will now stop upon first encountering a privilege that
is NOT granted over a resource (and return `false` overall). Previously,
all the privileges were always checked over all the resources in order
to assemble a comprehensive response with all the privileges that are
not granted.
This introduces a new Security API `_security/profile/_has_privileges`
that can be used to verify which Users have the requested privileges,
given their associated User Profiles. Multiple profile uids can be specified
in a single has privileges request.
This is analogous to the existing Has privileges API. It also uses the same
format for specifying the privileges to be checked, and should be used in
the same situations (ie to run an authorization preflight check or to verify
privileges over application resources). However, unlike the existing
has privilege API, this can be used to check the privileges of multiple
users (not only of the currently authenticated one), but the users must
have an existing profile, and the response is binary only (either it has or
it does not have the requested privileges).
Calling this API requires the `manage_user_profile` cluster privilege.
In #86206, we closed down Authentication constructors to favour
dedicated convenient methods for instantiation. The constructor usages
in the example project were however left out (another refactor fallout).
Relates: #86206Resolves: #86378
* Make ForbiddenApisPrecommitPlugin plugin Gradle 8.0 compatible
* Fix deprecations on ignoring empty folders for task inputs
* Update Gradle wrapper to 7.4 GA
Follow-up from #77144 (comment) with converting id/_id to always be strings instead of integers. This makes the type value in the Elasticsearch specification be only string instead of string | number.
this change was generated using following command on ubuntu
find . -type f -name "*.yml" -print0 | xargs -0 sed -i -r 's/([^a-zA-Z0-9_\.]id|[^a-zA-Z0-9_]_id):(\s*)([0-9]+)/\1:\2"\3"/g'
Fixes split packages between server and the LLRC (and HLRC), by renaming
the server package to a more appropriate name that represents the fact
that is in an internal client. That is, rename server's
org.elasticsearch.client to org.elasticsearch.client.internal.
This fixes two classpath issues for integration tests in the example
build
1. adds log4j to the runtime classpath where needed
2. removes exclude of jakarta json which was added as a workaround for
https://github.com/elastic/elasticsearch-java/issues/47
This commit removes the dependency on the Joda library. It removes
many remaining references to joda, though not all because some comments
are worthwhile for historical reasoning.
Fix the split package org.elasticsearch.common.xcontent, between server and the x-content lib. Move the x-content lib exported package from org.elasticsearch.common.xcontent to org.elasticsearch.xcontent ( following the naming convention of similar libraries ). Removing split packages is a prerequisite to modularization.
This commit detects a specific case when a child action (e.g. a shard
level action, or a phased action) acts on the same indices (or a
subset of the indices) or that parent request, and we can retain the
original authorization result.
The optimization is only effective for the invocation of the child
action on the same node as the parent - if the transport action needs
to be executed on a remote node then that authorization will not be
optimized and will perform the full check as existed before this
change.
This change is primarily benefitial for actions where a single parent
action on a coordinating node triggers the execution of multiple
children (e.g. a child action per shard) as it allows the
coordinating node to trigger those action and allow the load
to be passed to the remote nodes as quickly as possible rather than
having authorization on the coordinating node become a bottleneck.
* Do not create unused testCluster
This avoids creating test clusters that are not required during the build.
We use lazy configuration here on testClusters and only instantiate them as theyre
* Do not fail on run task (debug)
* Create more test cluster lazy
* Make more test cluster lazy
* Avoid creating unused testcluster
* Fix PluginBuildPlugin
* Fix disabling geo db download
* Fix cluster setup in repository-multi-version
* Polishing
* Fix issue with irretic groovy ogic
* Fix bwc tests
* Fix more bwcTests
* Fix more bwc tests
* Fix more bwc tests
* Fix more bwc tests
* Fix typo
* Minor polishing
* Fix rolling upgrade tests
* Fix cluster config in sql qa mixedcluster project
* Fix more bwc tests
* Clean up before review
* Document test cluster usage
* Api polising after Review
provide useCluster(Provider) method to TestClusterAware
Ideally we take this a step further and realize those test clusters only on use.
But out of scope of this PR.
* Allow gradle provider as value for nonSystemProperties
* Some simplification on test configuration
* Fix typo in rest test config
* Fix more typos
* Fix another typo
* Fix more typos
This is a follow up to #78140 and fixes a few instances where the build
was resolving artifacts from our snapshot Maven repository insteady of
using project dependency substitutions from the included build. To
ensure we don't miss these going forward, when building in a composite
we omit the snapshot repo definition altogether, which would instead
result in an error during dependency resolution.
This introduces a basic public yaml rest test plugin that is supposed to be used by external
elasticsearch plugin authors. This is driven by #76215
- Rename yaml-rest-test to intern-yaml-rest-test
- Use public yaml plugin in example plugins
Co-authored-by: Mark Vieira <portugee@gmail.com>
Adds minimal fields API support to sort and score scripts.
Example: `field('myfield').getValue(123)` where `123` is the default if the field has no values.
Refs: #61388
Aiming for configuring less during the build,
this removes non required configuration from qa build scripts that do not
contain any sources. We also remove a few non required afterEvaluate hooks
ParseField is part of the x-content lib, yet it doesn't exist under the
same root package as the rest of the lib. This commit moves the class to
the appropriate package.
relates #73784
When libs/core was created, several classes were moved from server's
o.e.common package, but they were not moved to a new package. Split
packages need to go away long term, so that Elasticsearch can even think
about modularization. This commit moves all the classes under o.e.common
in core to o.e.core.
relates #73784
This changes the result of AuthorizationEngine.loadAuthorizedIndices
(and dependent methods) from List<String> to Set<String>.
This has the following performance benefits:
1. `contains` checks are faster
2. RBACEngine always formed this collections as a Set, so this
change reduces unnecessary copying.
An additional performance improvement was added when resolve authorized
index names for data streams.
Extract usage of internal API from TestClustersPlugin and PluginBuildPlugin and related plugins and build logic
This includes a refactoring of ElasticsearchDistribution to handle types
better in a way we can differentiate between supported Elasticsearch
Distribution types supported in TestCkustersPlugin and types only supported
in internal plugins.
It also introduces a set of internal versions of public plugins.
As part of this we also generate the plugin descriptors now.
As a follow up on this we can actually move these public used classes into
an extra project (declared as included build)
We keep LoggedExec and VersionProperties effectively public And workaround for RestTestBase
Related to #71593 we move all build logic that is for elasticsearch build only into
the org.elasticsearch.gradle.internal* packages
This makes it clearer if build logic is considered to be used by external projects
Ultimately we want to only expose TestCluster and PluginBuildPlugin logic
to third party plugin authors.
This is a very first step towards that direction.
* Warn users if security is implicitly disabled
Elasticsearch has security features implicitly disabled by default for
Basic and Trial licenses, unless explicitly set in the configuration
file.
This may be good for onboarding, but it also lead to unintended insecure
clusters.
This change introduces clear warnings when security features are
implicitly disabled.
- a warning header in each REST response if security is implicitly
disabled;
- a log message during cluster boot.
As per the new licensing change for Elasticsearch and Kibana this commit
moves existing Apache 2.0 licensed source code to the new dual license
SSPL+Elastic license 2.0. In addition, existing x-pack code now uses
the new version 2.0 of the Elastic license. Full changes include:
- Updating LICENSE and NOTICE files throughout the code base, as well
as those packaged in our published artifacts
- Update IDE integration to now use the new license header on newly
created source files
- Remove references to the "OSS" distribution from our documentation
- Update build time verification checks to no longer allow Apache 2.0
license header in Elasticsearch source code
- Replace all existing Apache 2.0 license headers for non-xpack code
with updated header (vendored code with Apache 2.0 headers obviously
remains the same).
- Replace all Elastic license 1.0 headers with new 2.0 header in xpack.
We decided to rename `QueryShardContext` to clarify that it supports all parts
of search request execution. Before there was confusion over whether it should
only be used for building queries, or maybe only used in the query phase. This
PR also updates the javadocs.
Closes#64740.
We were depending on the BouncyCastle FIPS own mechanics to set
itself in approved only mode since we run with the Security
Manager enabled. The check during startup seems to happen before we
set our restrictive SecurityManager though in
org.elasticsearch.bootstrap.Elasticsearch , and this means that
BCFIPS would not be in approved only mode, unless explicitly
configured so.
This commit sets the appropriate JVM property to explicitly set
BCFIPS in approved only mode in CI and adds tests to ensure that we
will be running with BCFIPS in approved only mode when we expect to.
It also sets xpack.security.fips_mode.enabled to true for all test clusters
used in fips mode and sets the distribution to the default one. It adds a
password to the elasticsearch keystore for all test clusters that run in fips
mode.
Moreover, it changes a few unit tests where we would use bcrypt even in
FIPS 140 mode. These would still pass since we are bundling our own
bcrypt implementation, but are now changed to use FIPS 140 approved
algorithms instead for better coverage.
It also addresses a number of tests that would fail in approved only mode
Mainly:
Tests that use PBKDF2 with a password less than 112 bits (14char). We
elected to change the passwords used everywhere to be at least 14
characters long instead of mandating
the use of pbkdf2_stretch because both pbkdf2 and
pbkdf2_stretch are supported and allowed in fips mode and it makes sense
to test with both. We could possibly figure out the password algorithm used
for each test and adjust password length accordingly only for pbkdf2 but
there is little value in that. It's good practice to use strong passwords so if
our docs and tests use longer passwords, then it's for the best. The approach
is brittle as there is no guarantee that the next test that will be added won't
use a short password, so we add some testing documentation too.
This leaves us with a possible coverage gap since we do support passwords
as short as 6 characters but we only test with > 14 chars but the
validation itself was not tested even before. Tests can be added in a followup,
outside of fips related context.
Tests that use a PKCS12 keystore and were not already muted.
Tests that depend on running test clusters with a basic license or
using the OSS distribution as FIPS 140 support is not available in
neither of these.
Finally, it adds some information around FIPS 140 testing in our testing
documentation reference so that developers can hopefully keep in
mind fips 140 related intricacies when writing/changing docs.
Mark Vieira
Artem Prigoda
Rene Groeschke
Ryan Ernst
Albert Zaharovits
Yang Wang
Przemyslaw Gomulka
Rory Hunter
Mary Gouseti
Chris Hegarty
Ryan Ernst
Nikola Grcevski
Tim Vernum
Stuart Tettemer
Tamara Braun
Lyudmila Fokina
Julie Tibshirani
Ioannis Kakavas
Martijn Laarman