elasticsearch/qa/remote-clusters/build.gradle

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

102 lines
3.3 KiB
Groovy
Raw Normal View History

/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
2024-09-14 04:28:03 +08:00
* or more contributor license agreements. Licensed under the "Elastic License
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
* Public License v 1"; you may not use this file except in compliance with, at
* your election, the "Elastic License 2.0", the "GNU Affero General Public
* License v3.0 only", or the "Server Side Public License, v 1".
*/
import org.elasticsearch.gradle.Architecture
import org.elasticsearch.gradle.VersionProperties
import org.elasticsearch.gradle.internal.testfixtures.TestFixturesPlugin
import static org.elasticsearch.gradle.internal.distribution.InternalElasticsearchDistributionTypes.DOCKER;
apply plugin: 'elasticsearch.standalone-rest-test'
apply plugin: 'elasticsearch.test.fixtures'
apply plugin: 'elasticsearch.internal-distribution-download'
Ensure CI is run in FIPS 140 approved only mode (#64024) We were depending on the BouncyCastle FIPS own mechanics to set itself in approved only mode since we run with the Security Manager enabled. The check during startup seems to happen before we set our restrictive SecurityManager though in org.elasticsearch.bootstrap.Elasticsearch , and this means that BCFIPS would not be in approved only mode, unless explicitly configured so. This commit sets the appropriate JVM property to explicitly set BCFIPS in approved only mode in CI and adds tests to ensure that we will be running with BCFIPS in approved only mode when we expect to. It also sets xpack.security.fips_mode.enabled to true for all test clusters used in fips mode and sets the distribution to the default one. It adds a password to the elasticsearch keystore for all test clusters that run in fips mode. Moreover, it changes a few unit tests where we would use bcrypt even in FIPS 140 mode. These would still pass since we are bundling our own bcrypt implementation, but are now changed to use FIPS 140 approved algorithms instead for better coverage. It also addresses a number of tests that would fail in approved only mode Mainly: Tests that use PBKDF2 with a password less than 112 bits (14char). We elected to change the passwords used everywhere to be at least 14 characters long instead of mandating the use of pbkdf2_stretch because both pbkdf2 and pbkdf2_stretch are supported and allowed in fips mode and it makes sense to test with both. We could possibly figure out the password algorithm used for each test and adjust password length accordingly only for pbkdf2 but there is little value in that. It's good practice to use strong passwords so if our docs and tests use longer passwords, then it's for the best. The approach is brittle as there is no guarantee that the next test that will be added won't use a short password, so we add some testing documentation too. This leaves us with a possible coverage gap since we do support passwords as short as 6 characters but we only test with > 14 chars but the validation itself was not tested even before. Tests can be added in a followup, outside of fips related context. Tests that use a PKCS12 keystore and were not already muted. Tests that depend on running test clusters with a basic license or using the OSS distribution as FIPS 140 support is not available in neither of these. Finally, it adds some information around FIPS 140 testing in our testing documentation reference so that developers can hopefully keep in mind fips 140 related intricacies when writing/changing docs.
2020-12-24 03:00:49 +08:00
tasks.register("copyNodeKeyMaterial", Sync) {
from project(':x-pack:plugin:core')
Ensure CI is run in FIPS 140 approved only mode (#64024) We were depending on the BouncyCastle FIPS own mechanics to set itself in approved only mode since we run with the Security Manager enabled. The check during startup seems to happen before we set our restrictive SecurityManager though in org.elasticsearch.bootstrap.Elasticsearch , and this means that BCFIPS would not be in approved only mode, unless explicitly configured so. This commit sets the appropriate JVM property to explicitly set BCFIPS in approved only mode in CI and adds tests to ensure that we will be running with BCFIPS in approved only mode when we expect to. It also sets xpack.security.fips_mode.enabled to true for all test clusters used in fips mode and sets the distribution to the default one. It adds a password to the elasticsearch keystore for all test clusters that run in fips mode. Moreover, it changes a few unit tests where we would use bcrypt even in FIPS 140 mode. These would still pass since we are bundling our own bcrypt implementation, but are now changed to use FIPS 140 approved algorithms instead for better coverage. It also addresses a number of tests that would fail in approved only mode Mainly: Tests that use PBKDF2 with a password less than 112 bits (14char). We elected to change the passwords used everywhere to be at least 14 characters long instead of mandating the use of pbkdf2_stretch because both pbkdf2 and pbkdf2_stretch are supported and allowed in fips mode and it makes sense to test with both. We could possibly figure out the password algorithm used for each test and adjust password length accordingly only for pbkdf2 but there is little value in that. It's good practice to use strong passwords so if our docs and tests use longer passwords, then it's for the best. The approach is brittle as there is no guarantee that the next test that will be added won't use a short password, so we add some testing documentation too. This leaves us with a possible coverage gap since we do support passwords as short as 6 characters but we only test with > 14 chars but the validation itself was not tested even before. Tests can be added in a followup, outside of fips related context. Tests that use a PKCS12 keystore and were not already muted. Tests that depend on running test clusters with a basic license or using the OSS distribution as FIPS 140 support is not available in neither of these. Finally, it adds some information around FIPS 140 testing in our testing documentation reference so that developers can hopefully keep in mind fips 140 related intricacies when writing/changing docs.
2020-12-24 03:00:49 +08:00
.files(
'src/test/resources/org/elasticsearch/xpack/security/transport/ssl/certs/simple/testnode.pem',
'src/test/resources/org/elasticsearch/xpack/security/transport/ssl/certs/simple/testnode.crt'
)
into "${buildDir}/certs"
doLast {
file("${buildDir}/certs").setReadable(true, false)
Ensure CI is run in FIPS 140 approved only mode (#64024) We were depending on the BouncyCastle FIPS own mechanics to set itself in approved only mode since we run with the Security Manager enabled. The check during startup seems to happen before we set our restrictive SecurityManager though in org.elasticsearch.bootstrap.Elasticsearch , and this means that BCFIPS would not be in approved only mode, unless explicitly configured so. This commit sets the appropriate JVM property to explicitly set BCFIPS in approved only mode in CI and adds tests to ensure that we will be running with BCFIPS in approved only mode when we expect to. It also sets xpack.security.fips_mode.enabled to true for all test clusters used in fips mode and sets the distribution to the default one. It adds a password to the elasticsearch keystore for all test clusters that run in fips mode. Moreover, it changes a few unit tests where we would use bcrypt even in FIPS 140 mode. These would still pass since we are bundling our own bcrypt implementation, but are now changed to use FIPS 140 approved algorithms instead for better coverage. It also addresses a number of tests that would fail in approved only mode Mainly: Tests that use PBKDF2 with a password less than 112 bits (14char). We elected to change the passwords used everywhere to be at least 14 characters long instead of mandating the use of pbkdf2_stretch because both pbkdf2 and pbkdf2_stretch are supported and allowed in fips mode and it makes sense to test with both. We could possibly figure out the password algorithm used for each test and adjust password length accordingly only for pbkdf2 but there is little value in that. It's good practice to use strong passwords so if our docs and tests use longer passwords, then it's for the best. The approach is brittle as there is no guarantee that the next test that will be added won't use a short password, so we add some testing documentation too. This leaves us with a possible coverage gap since we do support passwords as short as 6 characters but we only test with > 14 chars but the validation itself was not tested even before. Tests can be added in a followup, outside of fips related context. Tests that use a PKCS12 keystore and were not already muted. Tests that depend on running test clusters with a basic license or using the OSS distribution as FIPS 140 support is not available in neither of these. Finally, it adds some information around FIPS 140 testing in our testing documentation reference so that developers can hopefully keep in mind fips 140 related intricacies when writing/changing docs.
2020-12-24 03:00:49 +08:00
file("${buildDir}/certs/testnode.pem").setReadable(true, false)
file("${buildDir}/certs/testnode.crt").setReadable(true, false)
}
}
elasticsearch_distributions {
docker {
type = DOCKER
architecture = Architecture.current()
version = VersionProperties.getElasticsearch()
failIfUnavailable = false // This ensures we skip this testing if Docker is unavailable
}
}
interface Injected {
@Inject
FileSystemOperations getFs()
}
tasks.named("preProcessFixture").configure {
Ensure CI is run in FIPS 140 approved only mode (#64024) We were depending on the BouncyCastle FIPS own mechanics to set itself in approved only mode since we run with the Security Manager enabled. The check during startup seems to happen before we set our restrictive SecurityManager though in org.elasticsearch.bootstrap.Elasticsearch , and this means that BCFIPS would not be in approved only mode, unless explicitly configured so. This commit sets the appropriate JVM property to explicitly set BCFIPS in approved only mode in CI and adds tests to ensure that we will be running with BCFIPS in approved only mode when we expect to. It also sets xpack.security.fips_mode.enabled to true for all test clusters used in fips mode and sets the distribution to the default one. It adds a password to the elasticsearch keystore for all test clusters that run in fips mode. Moreover, it changes a few unit tests where we would use bcrypt even in FIPS 140 mode. These would still pass since we are bundling our own bcrypt implementation, but are now changed to use FIPS 140 approved algorithms instead for better coverage. It also addresses a number of tests that would fail in approved only mode Mainly: Tests that use PBKDF2 with a password less than 112 bits (14char). We elected to change the passwords used everywhere to be at least 14 characters long instead of mandating the use of pbkdf2_stretch because both pbkdf2 and pbkdf2_stretch are supported and allowed in fips mode and it makes sense to test with both. We could possibly figure out the password algorithm used for each test and adjust password length accordingly only for pbkdf2 but there is little value in that. It's good practice to use strong passwords so if our docs and tests use longer passwords, then it's for the best. The approach is brittle as there is no guarantee that the next test that will be added won't use a short password, so we add some testing documentation too. This leaves us with a possible coverage gap since we do support passwords as short as 6 characters but we only test with > 14 chars but the validation itself was not tested even before. Tests can be added in a followup, outside of fips related context. Tests that use a PKCS12 keystore and were not already muted. Tests that depend on running test clusters with a basic license or using the OSS distribution as FIPS 140 support is not available in neither of these. Finally, it adds some information around FIPS 140 testing in our testing documentation reference so that developers can hopefully keep in mind fips 140 related intricacies when writing/changing docs.
2020-12-24 03:00:49 +08:00
dependsOn "copyNodeKeyMaterial", elasticsearch_distributions.docker
def injected = project.objects.newInstance(Injected)
doLast {
// tests expect to have an empty repo
injected.fs.delete {
it.delete("${testFixturesDir}/repo")
it.delete("${testFixturesDir}/oss-repo")
}
createAndSetWritable(
"${testFixturesDir}/repo",
"${testFixturesDir}/oss-repo",
"${testFixturesDir}/logs/default-1",
"${testFixturesDir}/logs/default-2",
"${testFixturesDir}/logs/oss-1",
"${testFixturesDir}/logs/oss-2"
)
}
}
dockerCompose {
tcpPortsToIgnoreWhenWaiting = [9600, 9601]
if ('default'.equalsIgnoreCase(providers.systemProperty('tests.distribution').getOrElse('default'))) {
useComposeFiles = ['docker-compose.yml']
} else {
useComposeFiles = ['docker-compose-oss.yml']
}
}
def createAndSetWritable(Object... locations) {
locations.each { location ->
println "location = $location"
File file = new File(location)
file.mkdirs()
file.setWritable(true, false)
}
}
tasks.named("processTestResources").configure {
from project(':x-pack:plugin:core')
Ensure CI is run in FIPS 140 approved only mode (#64024) We were depending on the BouncyCastle FIPS own mechanics to set itself in approved only mode since we run with the Security Manager enabled. The check during startup seems to happen before we set our restrictive SecurityManager though in org.elasticsearch.bootstrap.Elasticsearch , and this means that BCFIPS would not be in approved only mode, unless explicitly configured so. This commit sets the appropriate JVM property to explicitly set BCFIPS in approved only mode in CI and adds tests to ensure that we will be running with BCFIPS in approved only mode when we expect to. It also sets xpack.security.fips_mode.enabled to true for all test clusters used in fips mode and sets the distribution to the default one. It adds a password to the elasticsearch keystore for all test clusters that run in fips mode. Moreover, it changes a few unit tests where we would use bcrypt even in FIPS 140 mode. These would still pass since we are bundling our own bcrypt implementation, but are now changed to use FIPS 140 approved algorithms instead for better coverage. It also addresses a number of tests that would fail in approved only mode Mainly: Tests that use PBKDF2 with a password less than 112 bits (14char). We elected to change the passwords used everywhere to be at least 14 characters long instead of mandating the use of pbkdf2_stretch because both pbkdf2 and pbkdf2_stretch are supported and allowed in fips mode and it makes sense to test with both. We could possibly figure out the password algorithm used for each test and adjust password length accordingly only for pbkdf2 but there is little value in that. It's good practice to use strong passwords so if our docs and tests use longer passwords, then it's for the best. The approach is brittle as there is no guarantee that the next test that will be added won't use a short password, so we add some testing documentation too. This leaves us with a possible coverage gap since we do support passwords as short as 6 characters but we only test with > 14 chars but the validation itself was not tested even before. Tests can be added in a followup, outside of fips related context. Tests that use a PKCS12 keystore and were not already muted. Tests that depend on running test clusters with a basic license or using the OSS distribution as FIPS 140 support is not available in neither of these. Finally, it adds some information around FIPS 140 testing in our testing documentation reference so that developers can hopefully keep in mind fips 140 related intricacies when writing/changing docs.
2020-12-24 03:00:49 +08:00
.files(
'src/test/resources/org/elasticsearch/xpack/security/transport/ssl/certs/simple/testnode.pem',
'src/test/resources/org/elasticsearch/xpack/security/transport/ssl/certs/simple/testnode.crt'
)
}
tasks.register("integTest", Test) {
outputs.doNotCacheIf('Build cache is disabled for Docker tests') { true }
maxParallelForks = '1'
include '**/*IT.class'
}
tasks.named("check").configure { dependsOn "integTest" }