Commit Graph

8080 Commits

Author SHA1 Message Date
James Rodewig 9fb7bacbc1
[DOCS] Clarify memlock settings in `/etc/security/limits.conf` (#66694) 2021-01-05 09:02:40 -05:00
István Zoltán Szabó d3ad9fe632
[DOCS] Improves inference processor linking and docs (#66119) 2021-01-05 09:42:06 +01:00
Przemyslaw Gomulka 5e74f79e22
Support response content-type with versioned media type (#65500)
This commit allows returning a correct requested response content-type - it did not work for versioned media types.
It is done by adding new vendor specific instances to XContent and TextFormat enums. These instances can then "format" the response content type string when provided with parameters. This is similar to what SQL plugin does with its media types.

#51816
2021-01-05 09:23:22 +01:00
James Rodewig 0f50732068
[DOCS] Note breaking change applies to legacy rollup jobs (#66894) 2020-12-30 09:59:24 -05:00
James Rodewig 751bc28baa
[DOCS] Add `nodes` and `parent_task_id` parms (#66562) 2020-12-30 08:50:27 -05:00
James Rodewig 68a83473d3
[DOCS] Update rollup def (#66870) 2020-12-29 14:10:35 -05:00
James Rodewig b1e747d8a9
[DOCS] Fix attribute typo (#66858) 2020-12-29 09:45:06 -05:00
James Rodewig a488794063 [DOCS] Minor reword 2020-12-28 12:50:51 -05:00
James Rodewig 00837b3ffc
[DOCS] Add xref to agg metric double field type (#66831) 2020-12-28 09:45:09 -05:00
Ioannis Kakavas bd873698bc
Ensure CI is run in FIPS 140 approved only mode (#64024)
We were depending on the BouncyCastle FIPS own mechanics to set
itself in approved only mode since we run with the Security
Manager enabled. The check during startup seems to happen before we
set our restrictive SecurityManager though in
org.elasticsearch.bootstrap.Elasticsearch , and this means that
BCFIPS would not be in approved only mode, unless explicitly
configured so.

This commit sets the appropriate JVM property to explicitly set
BCFIPS in approved only mode in CI and adds tests to ensure that we
will be running with BCFIPS in approved only mode when we expect to.
It also sets xpack.security.fips_mode.enabled to true for all test clusters
used in fips mode and sets the distribution to the default one. It adds a
password to the elasticsearch keystore for all test clusters that run in fips
mode.
Moreover, it changes a few unit tests where we would use bcrypt even in
FIPS 140 mode. These would still pass since we are bundling our own
bcrypt implementation, but are now changed to use FIPS 140 approved
algorithms instead for better coverage.

It also addresses a number of tests that would fail in approved only mode
Mainly:

    Tests that use PBKDF2 with a password less than 112 bits (14char). We
    elected to change the passwords used everywhere to be at least 14
    characters long instead of mandating
    the use of pbkdf2_stretch because both pbkdf2 and
    pbkdf2_stretch are supported and allowed in fips mode and it makes sense
    to test with both. We could possibly figure out the password algorithm used
    for each test and adjust password length accordingly only for pbkdf2 but
    there is little value in that. It's good practice to use strong passwords so if
    our docs and tests use longer passwords, then it's for the best. The approach
    is brittle as there is no guarantee that the next test that will be added won't
    use a short password, so we add some testing documentation too.
    This leaves us with a possible coverage gap since we do support passwords
    as short as 6 characters but we only test with > 14 chars but the
    validation itself was not tested even before. Tests can be added in a followup,
    outside of fips related context.

    Tests that use a PKCS12 keystore and were not already muted.

    Tests that depend on running test clusters with a basic license or
    using the OSS distribution as FIPS 140 support is not available in
    neither of these.

Finally, it adds some information around FIPS 140 testing in our testing
documentation reference so that developers can hopefully keep in
mind fips 140 related intricacies when writing/changing docs.
2020-12-23 21:00:49 +02:00
James Rodewig 6493e6575e
[DOCS] Update rollup xrefs (#66799) 2020-12-23 13:49:15 -05:00
Nik Everett 3e3152406a
Bust the request cache when the mapping changes (#66295)
This makes sure that we only serve a hit from the request cache if it
was build using the same mapping and that the same mapping is used for
the entire "query phase" of the search.

Closes #62033
2020-12-23 13:19:02 -05:00
James Rodewig ed43be8c42
[DOCS] Fix typo (#66779) (#66781)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: Sandeep Kanabar <sandeepkanabar@gmail.com>
2020-12-23 09:24:30 -05:00
James Rodewig 3f528945f0 [DOCS] Fix typo 2020-12-22 13:36:03 -05:00
Lisa Cawley 6b463a7b7a
[DOCS] Clarify use of CCS on ML nodes (#66616)
Co-authored-by: David Roberts <dave.roberts@elastic.co>
2020-12-22 10:11:09 -08:00
James Rodewig d8d5b8c49a
[DOCS] Increment section blocks (#66751) 2020-12-22 10:29:11 -05:00
James Rodewig c24f9e125b
[DOCS] Minor rewording (#66744) 2020-12-22 09:24:04 -05:00
Alan Tan ee6cbef7c2
[DOCS] Fix typo (#66721) 2020-12-22 09:02:33 -05:00
James Rodewig e9c1cded98
[DOCS] Fix outdated heap references (#66646) 2020-12-21 14:01:12 -05:00
James Rodewig 523324c1c8
Add info on how to disable cleaner service (#64655) (#66704)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: Julien Guay <guay_j@yahoo.fr>
2020-12-21 13:22:45 -05:00
James Rodewig 10b036e934
[DOCS] Fix timeout parameter defaults (#66111) 2020-12-21 09:02:06 -05:00
bellengao d14492ca13
[DOCS] Fix some typos in docs (#66672) 2020-12-21 12:45:51 +02:00
Mark Vieira ea9df4e66b
Update heap setting documentation in light of machine dependent heap (#66567) 2020-12-18 11:14:56 -08:00
Lisa Cawley d5892c50cd
[DOCS] Drafts API changes for transform latest function (#66481) 2020-12-18 09:50:09 -08:00
Adam Locke 82bfbe1195
[DOCS] Adding headers in TOC for aggregation docs. (#66604) 2020-12-18 11:31:42 -05:00
James Rodewig 14b381a2ad
[DOCS] EQL: Change `result_position` default to `tail` (#66550) 2020-12-18 08:38:45 -05:00
David Turner b622adeb7a Revert "Document new waiting on shards on index close (#66543)"
This reverts commit 1c059e79e8.
2020-12-17 21:32:16 +00:00
James Rodewig da0188fb45
[DOCS] Fix Fleet links (#66553) (#66557) 2020-12-17 15:27:15 -05:00
David Turner 1c059e79e8
Document new waiting on shards on index close (#66543)
In 8.x the default for `?wait_for_active_shards` changes from `NONE` to
`DEFAULT` on calls to `POST /index/_close`. This commit adds this change
to the breaking changes docs.

Relates #66419, #66542
2020-12-17 17:51:24 +00:00
Howard acdbe02103
[DOCS] Fix rollup docs formatting (#66425) 2020-12-17 12:27:28 -05:00
James Rodewig 0991e35607 [DOCS] Fix formatting 2020-12-16 20:47:09 -05:00
Lisa Cawley 709068da7c
[DOCS] Adds xpack.ml.max_ml_node_size (#66285) 2020-12-16 12:55:34 -08:00
Rory Hunter e49fd15e0c Fix docs typo 2020-12-16 20:29:20 +00:00
Lisa Cawley 35cc6bf8a0
[DOCS] Changes static ML setting to dynamic (#66286) 2020-12-16 08:32:57 -08:00
James Rodewig 7c0f193b2c
[DOCS] Fix formatting (#66450) 2020-12-16 11:09:55 -05:00
James Rodewig b5d2d30599
[DOCS] Remove duplicate word (#66320) (#66446)
Co-authored-by: Gao Ruifeng <gaoruifeng@users.noreply.github.com>
2020-12-16 10:49:46 -05:00
Yang Wang b018c761e9
Remove support of creating CA on the fly when generating certificates (#65590)
Generating certificates with the cert sub-command now requires either: 1) a CA
to be provided with --ca or --ca-cert/--ca-key; or 2) make them self-signed
with the --self-signed option. Generating a CA on the fly is no longer
supported. The --keep-ca-key option is removed and the tool throws an error 
saying the CA needs to be generated separately if the option is specified.

This is a follow-up PR for #61884, which deprecated the "ca-on-the-fly" usage.
2020-12-16 13:54:32 +11:00
James Rodewig 234b9437c3
[DOCS] Fix docs integ tests for agg metric field docs (#66397) 2020-12-15 13:57:42 -05:00
David Roberts c5bef7f9a7
[ML] Deprecate anomaly detection post data endpoint (#66347)
There is little evidence of this endpoint being used
and there is quite a lot of code complexity associated
with the various formats that can be used to upload
data and the different errors that can occur when direct
data upload is open to end users.

In a future release we can make this endpoint internal
so that only datafeeds can use it, and remove all the
options and formats that are not used by datafeeds.

End users will have to store their input data for
anomaly detection in Elasticsearch indices (which we
believe all do today) and use a datafeed to feed it
to anomaly detection jobs.
2020-12-15 18:37:20 +00:00
James Rodewig 9e0de16c5d
[DOCS] Mute metric agg docs tests (#66394) 2020-12-15 13:21:21 -05:00
David Turner 32b20fd36f
Fix Log4J config example in docs (#66386)
We lost the `logger.transport.name` line in #65169 and I incorrectly
extrapolated from what was left and mangled it further in #66318. This
commit fixes things.
2020-12-15 17:42:46 +00:00
Christos Soulios 30efc2b81b
[DOCS] Added docs for aggregate_metric_double field (#66306)
Added documentation for the aggregate_metric_double field that was merged in #56745

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2020-12-15 19:27:10 +02:00
Dimitris Athanasiou 3bed6661de
[ML] Add log_time to AD data_counts and decide current based on it (#66343)
This commit is fixing a potential bug if we support anomaly detection
results index rollover in the future.

In particular, we determine the current `data_counts` by sorting on the
latest record time. However, this is not correct if the job reverts
to an older model snapshot. To fix this we add `log_time` to `data_counts`
(similarly to `model_size_stats`) and sort on `log_time` to figure
out the current counts for the job.
2020-12-15 19:09:13 +02:00
David Turner b706e0dd69
Give a less dangerous example of logger config (#66318)
Today the docs use `logger.org.elasticsearch.transport: TRACE` as the
example for adjusting the logger config. This is a dangerous thing to
suggest since that's one of the most verbose loggers we have. An
accidental copy-and-paste of this example into a busy cluster can
cause damage.

This commit suggests `logger.org.elasticsearch.discovery: DEBUG`
instead, which is much more benign.

It also corrects the order of the levels and notes that `DEBUG` and
`TRACE` are only for expert use.
2020-12-15 17:04:26 +00:00
Adam Locke be3bc46111
[DOCS] Add description for node info settings. (#66362) 2020-12-15 11:27:42 -05:00
Adam Locke 4145d544e5
Incorporating changes from review. (#66346) (#66358) 2020-12-15 10:30:33 -05:00
James Rodewig 56651fbd78
[DOCS] Note `tar.gz` does not include `systemd` (#66298) 2020-12-15 08:58:13 -05:00
James Rodewig f63e54ad4a
[DOCS] Fix ingest node.roles example (#66287) 2020-12-15 08:19:54 -05:00
James Rodewig 77dc63b2de
[DOCS] Fix `search.max_buckets` default (#66311) 2020-12-14 21:55:27 -05:00
Dan Hermann 83a5256dc2
Include date in data stream backing index names (#65205) 2020-12-14 16:46:54 -06:00