Commit Graph

13 Commits

Author SHA1 Message Date
David Turner cf97e967a9
More S3-compatible repo deflection (#100754)
Call out explicitly that users need to reproduce issues with the real S3
before reporting them to ES.
2023-10-12 05:41:50 -04:00
Tanguy Leroux 583a787618
[Docs] Link to AWS SDK documentation for requests logging (#100491)
Co-authored-by: David Turner david.turner@elastic.co
2023-10-12 09:48:45 +02:00
David Turner 3691312aca
Slightly adjust docs about S3 incompatibilities (#99624)
It's often useful to quote these docs to users encountering problems
with their not-quite-S3-compatible storage system. In practice we don't
need to quote the bits in the middle but we do need the last sentence
about working with the supplier to address incompatibilities. This
commit reorders things so that the most commonly quoted sentences form a
standalone paragraph.
2023-09-18 08:26:03 -04:00
debadair 777598d602
[DOCS] Remove redirect pages (#88738)
* [DOCS] Remove manual redirects

* [DOCS] Removed refs to modules-discovery-hosts-providers

* [DOCS] Fixed broken internal refs

* Fixing bad cross links in ES book, and adding redirects.asciidoc[] back into docs/reference/index.asciidoc.

* Update docs/reference/search/point-in-time-api.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/setup/restart-cluster.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/sql/endpoints/translate.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/snapshot-restore/restore-snapshot.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update repository-azure.asciidoc

* Update node-tool.asciidoc

* Update repository-azure.asciidoc

---------

Co-authored-by: amyjtechwriter <61687663+amyjtechwriter@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Amy Jonsson <amy.jonsson@elastic.co>
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
2023-05-24 12:32:46 +01:00
Francisco Fernández Castaño da387b430c
Link to the time-units doc in S3 repository docs instead of explaining it in words (#93351) 2023-01-31 11:59:20 +01:00
Francisco Fernández Castaño ed9246f8d4
Amend read_timeout S3 repository setting description (#93136) 2023-01-23 15:34:46 +01:00
David Turner 82ed1fbcc9
Clarify use of S3 lifecycle policies (#92427)
Clarifies that it doesn't work to transition to Glacier tiers, nor does
it work to use object expiry, and that the consequences can be severe.
2022-12-19 09:04:29 +00:00
David Turner 76b05bfd8e
Note that S3 compat includes performance (#84798)
Today the note in the docs about S3-compatible repositories notes that
the repo must behave correctly, but it's also important that it has the
same performance profile. This commit extends the docs to include this
info.
2022-03-09 12:28:17 +00:00
Dan Roscigno 302ce75a88
Add note about base_path and ECE to the snapshot repository docs (#83526)
Elastic Cloud Enterprise (ECE) shares snapshot repositories across multiple deployments. As a result, the `base_path` is generated by ECE, and the `base_path` setting is not allowed.  This PR adds a note to the S3, Azure, and GCS snapshot repository docs.
2022-02-04 11:34:48 -05:00
David Turner 9633883c64
Add note on truststore for S3-compatible repos (#82669)
Today we note that the `repository-s3` plugin uses the JVM-wide
truststore in the docs for the `protocol` client setting, but it turns
out that this is easy to overlook since most installations will not need
to change the `protocol`. This commit adds the same detail to the
section on S3-compatible repositories where it is more likely to be
found.
2022-01-26 11:16:43 +00:00
James Rodewig 6b841325f1
[DOCS] Fix headings for Azure, GCS, and S3 snapshot repo pages (#82996)
Updates the headings to use sentence case.
2022-01-24 17:14:55 -05:00
Artem Prigoda e47b7a63f4
[s3-repository] Support IAM roles for Kubernetes service accounts (#81255)
There have been many requests to support repository-s3 authentication via IAM roles in Kubernetes service accounts.

The AWS SDK is supposed to support them out of the box with the aws-java-sdk-sts library. Unfortunately, we can't use WebIdentityTokenCredentialsProvider from the SDK. It reads the token from AWS_WEB_IDENTITY_TOKEN_FILE environment variable which is usually mounted to /var/run/secrets/eks.amazonaws.com/serviceaccount/token and the S3 repository doesn't have the read permission to read it. We don't want to hard-code a file permission for the repository, because the location of AWS_WEB_IDENTITY_TOKEN_FILE can change at any time in the future and we would also generally prefer to restrict the ability of plugins to access things outside of their config directory.

To overcome this limitation, this change adds a custom WebIdentityCredentials provider that reads the service account from a symlink to AWS_WEB_IDENTITY_TOKEN_FILE created in the repository's config directory. We expect the end user to create the symlink to indicate that they want to use service accounts for authentification.

Service accounts are checked and exchanged for session tokens by the AWS STS. To test the authentification flow, this change adds a test fixture which mocks the assume-role-with-web-identity call to the service and returns a response with test credentials.

Fixes #52625
2022-01-19 14:03:11 +01:00
Rory Hunter d2dbef5063
Convert repository plugins to modules (#81870)
Closes #81652.

Convert the `repository-azure`, `repository-gcs` and `repository-s3`
plugins into modules, so that they are always included in the
Elasticsearch distribution. Also change plugin installation, removal
and syncing so that attempting to add or remove these plugins still
succeeds but is now a no-op.
2022-01-10 10:45:42 +00:00