Commit Graph

33 Commits

Author SHA1 Message Date
James Rodewig f1e1661088
[DOCS] Reuse `aliases` object properties in API docs (#73421)
Changes:
* Reuses the same `aliases` object properties in the following API docs:
  * Clone index API
  * Create index API
  * Put component template API
  * Put legacy index template API
  * Put index template API
  * Rollover index API
  * Shrink index API
  * Simulate template API
  * Split index API
* Updates the `aliases` object properties for the simulate index API docs.

Closes #73044
2021-06-02 08:31:42 -04:00
James Rodewig 0f1fb46d58
[DOCS] Expand and reuse `aliases` parameters (#73296)
Changes:

* Expands the `aliases` parameter for the create index API to better document
  supported properties.

* Reuses `aliases` parameter in the following API docs:
  * Clone index API
  * Shrink index API
  * Split index API
2021-05-24 08:39:05 -04:00
James Rodewig 693807a6d3
[DOCS] Fix double spaces (#71082) 2021-03-31 09:57:47 -04:00
James Rodewig 20df933e6b
[DOCS] Update data stream rollover references (#70993) 2021-03-29 10:37:14 -04:00
Joe Gallo 7e7c5db74e
Rename max_single_primary_size to max_primary_shard_size (#69239) 2021-02-18 21:13:58 -05:00
bellengao d69c03359f
Support max_single_primary_size in Resize Action and exposed in ILM (#67705) 2021-01-29 15:29:45 -05:00
James Rodewig 81c3cba8a8
[DOCS] Add security privileges to index API docs (#68071) 2021-01-28 08:53:10 -05:00
James Rodewig ed43be8c42
[DOCS] Fix typo (#66779) (#66781)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: Sandeep Kanabar <sandeepkanabar@gmail.com>
2020-12-23 09:24:30 -05:00
James Rodewig 3728ae6811
[DOCS] Replace `twitter` dataset in index API docs (#60473) 2020-07-31 09:09:54 -04:00
James Rodewig 98d21fe551
[DOCS] Fix shrink index API prereqs (#59985) 2020-07-22 13:10:10 -04:00
Dan Hermann 484918ca70
[DOCS] Prohibit cloning, splitting, and shrinking a data stream's write index (#58105) 2020-06-22 07:14:21 -05:00
Lisa Cawley 8b9293b3bf
[DOCS] Replace docdir attribute with es-repo-dir (#57489) 2020-06-01 15:55:05 -07:00
James Rodewig 7ad45f406e
[DOCS] Reformat shrink index API docs (#46711) 2019-10-04 12:07:46 -04:00
James Rodewig 7583c07fa8
[DOCS] Reorder index APIs alphabetically (#46981) 2019-10-01 15:13:27 -04:00
James Rodewig 97802d8aff
[DOCS] Change // CONSOLE comments to [source,console] (#46441) 2019-09-06 10:55:16 -04:00
James Rodewig fab98dfa55
[DOCS] Remove heading offsets for REST APIs (#44568)
Several files in the REST APIs nav section are included using
:leveloffset: tags. This increments headings (h2 -> h3, h3 -> h4, etc.)
in those files and removes the :leveloffset: tags.

Other supporting changes:
* Alphabetizes top-level REST API nav items.
* Change 'indices APIs' heading to 'index APIs.'
* Changes 'Snapshot lifecycle management' heading to sentence case.
2019-07-19 14:35:36 -04:00
Peter Dyson 38ea4efe1d shrink may full copy when using multi data paths (#42913)
Additional scenario for full segment copy if hard link
cannot work across disks.
2019-06-06 13:00:00 -04:00
Jason Tedor d68c44b76c
Default copy settings to true and deprecate on the REST layer (#30598)
This commit defaults the copy_settings REST parameter to the shrink and
split APIs to true, and deprecates the parameter.
2018-05-18 10:12:08 -04:00
Jason Tedor 4a4e3d70d5
Default to one shard (#30539)
This commit changes the default out-of-the-box configuration for the
number of shards from five to one. We think this will help address a
common problem of oversharding. For users with time-based indices that
need a different default, this can be managed with index templates. For
users with non-time-based indices that find they need to re-shard with
the split API in place they no longer need to resort only to
reindexing.

Since this has the impact of changing the default number of shards used
in REST tests, we want to ensure that we still have coverage for issues
that could arise from multiple shards. As such, we randomize (rarely)
the default number of shards in REST tests to two. This is managed via a
global index template. However, some tests check the templates that are
in the cluster state during the test. Since this template is randomly
there, we need a way for tests to skip adding the template used to set
the number of shards to two. For this we add the default_shards feature
skip. To avoid having to write our docs in a complicated way because
sometimes they might be behind one shard, and sometimes they might be
behind two shards we apply the default_shards feature skip to all docs
tests. That is, these tests will always run with the default number of
shards (one).
2018-05-14 12:22:35 -04:00
Jason Tedor 901436148b Adjust copy settings versions
This commit adjusts the versions on the copy settings behavior now
that the default behavior is configured in 7.0.0.
2018-05-13 22:23:13 -04:00
Jason Tedor 593fdd40ed
Deprecate not copy settings and explicitly disallow (#30404)
We want copying settings to be the default behavior. This commit
deprecates not copying settings, and disallows explicitly not copying
settings. This gives users a transition path to the future default
behavior.
2018-05-13 10:30:05 -04:00
Jason Tedor 5de6f4ff7b Adjust copy settings on resize BWC version
This commit adjusts the BWC version for copy settings on resize
operations after the behavior was backported to 6.x.
2018-05-01 08:49:16 -04:00
Jason Tedor 50535423ff
Allow copying source settings on resize operation (#30255)
Today when an index is created from shrinking or splitting an existing
index, the target index inherits almost none of the source index
settings. This is surprising and a hassle for operators managing such
indices. Given this is the default behavior, we can not simply change
it. Instead, we start by introducing the ability to copy settings. This
flag can be set on the REST API or on the transport layer and it has the
behavior that it copies all settings from the source except non-copyable
settings (a property of a setting introduced in this
change). Additionally, settings on the request will always override.

This change is the first step in our adventure:
 - this flag is added here in 7.0.0 and immediately deprecated
 - this flag will be backported to 6.4.0 and remain deprecated
 - then, we will remove the ability to set this flag to false in 7.0.0
 - finally, in 8.0.0 we will remove this flag and the only behavior will
   be for settings to be copied
2018-05-01 08:48:19 -04:00
Jason Tedor 811f5b4efc
Do not ignore request analysis/similarity on resize (#30216)
Today when a resize operation is performed, we copy the analysis,
similarity, and sort settings from the source index. It is possible for
the resize request to include additional index settings including
analysis, similarity, and sort settings. We reject sort settings when
validating the request. However, we silently ignore analysis and
similarity settings on the request that are already set on the source
index. Since it is possible to change the analysis and similarity
settings on an existing index, this should be considered a bug and the
sort of leniency that we abhor. This commit addresses this bug by
allowing the request analysis/similarity settings to override the
existing analysis/similarity settings on the target.
2018-04-30 07:31:36 -04:00
Nik Everett bfd072bc10 Switch more docs to CONSOLE
Related to #18160
2016-09-02 18:22:30 -04:00
Ali Beyad 4641254ea6 Parameter improvements to Cluster Health API wait for shards (#20223)
* Params improvements to Cluster Health API wait for shards

Previously, the cluster health API used a strictly numeric value
for `wait_for_active_shards`. However, with the introduction of
ActiveShardCount and the removal of write consistency level for
replication operations, `wait_for_active_shards` is used for
write operations to represent values for ActiveShardCount. This
commit moves the cluster health API's usage of `wait_for_active_shards`
to be consistent with its usage in the write operation APIs.

This commit also changes `wait_for_relocating_shards` from a
numeric value to a simple boolean value `wait_for_no_relocating_shards`
to set whether the cluster health operation should wait for
all relocating shards to complete relocation.

* Addresses code review comments

* Don't be lenient if `wait_for_relocating_shards` is set
2016-08-31 11:58:19 -04:00
Ali Beyad a21dd80f1b Documentation changes for wait_for_active_shards (#19581)
Documentation changes and migration doc changes for introducing 
wait_for_active_shards and removing write consistency level.

Closes #19581
2016-08-02 09:15:01 -04:00
Glen Smith d7099f05b9 slight clarification 2016-07-07 20:46:18 -04:00
Christoph Wurm d1727653dd Update shrink-index.asciidoc
Fix half-finished sentence
2016-07-05 13:34:58 +02:00
Simon Willnauer b2c4c323e1 Allow `_shrink` to N shards if source shards is a multiple of N (#18699)
Today we allow to shrink to 1 shard but that might not be possible due to
too many document or a single shard doesn't meet the requirements for the index.
The logic can be expanded to N shards if the source index shards is a multiple of N.
This guarantees that there are not hotspots created due to different number of shards
being shrunk into one.
2016-06-07 10:06:41 +02:00
Clinton Gormley e6aaaf11ed Reworked docs for index-shrink API (#18705) 2016-06-03 09:50:51 +02:00
Simon Willnauer 22dfc41521 Only filter intial recovery (post API) when shrinking an index (#18661)
Today we use `index.routing.allocation.include._id` to filter the allocation
for the shrink target index. That has the sideeffect that the user has to
delete that setting / change it once the primary has been recovered (shrink is done)
This PR adds a dedicated filter that can only be set internally that only filters
allocation for unassigned shards.
2016-06-02 15:38:51 +02:00
Simon Willnauer 502a775a7c Add primitive to shrink an index into a single shard (#18270)
This adds a low level primitive operations to shrink an existing
index into a new index with a single shard. This primitive expects
all shards of the source index to allocated on a single node. Once the target index is initializing on the shrink node it takes a snapshot of the source index shards and copies all files into the target indices data folder. An [optimization](https://issues.apache.org/jira/browse/LUCENE-7300) coming in Lucene 6.1 will also allow for optional constant time copy if hard-links are supported by the filesystem. All mappings are merged into the new indexes metadata once the snapshots have been taken on the merge node.

To shrink an existing index all shards must be moved to a single node (one instance of each shard) and the index must be read-only:

```BASH
$ curl -XPUT 'http://localhost:9200/logs/_settings' -d '{
    "settings" : {
        "index.routing.allocation.require._name" : "shrink_node_name",
        "index.blocks.write" : true 
    }
}
```
once all shards are started on the shrink node. the new index can be created via:

```BASH
$ curl -XPUT 'http://localhost:9200/logs/_shrink/logs_single_shard' -d '{
    "settings" : {
        "index.codec" : "best_compression",
        "index.number_of_replicas" : 1
    }
}'
```

This API will perform all needed check before the new index is created and selects the shrink node based on the allocation of the source index. This call returns immediately, to monitor shrink progress the recovery API should be used since all copy operations are reflected in the recovery API with byte copy progress etc.

The shrink operation does not modify the source index, if a shrink operation should
be canceled or if the shrink failed, the target index can simply be deleted and
all resources are released.
2016-05-31 10:41:44 +02:00