root
/
elasticsearch
mirror of https://github.com/elastic/elasticsearch.git
1
0
Fork 0
Code Issues Actions 1 Packages Projects Releases Wiki Activity
elasticsearch/docs/reference/rest-api/usage.asciidoc

419 lines
9.5 KiB
Plaintext
Raw Normal View History

[DOCS] Adds X-Pack usage API (#52496)
2020-02-21 01:25:57 +08:00
[role="xpack"]
[testenv="basic"]
[[usage-api]]
== Usage API
Provides usage information about the installed {xpack} features.
[discrete]
[[usage-api-request]]
=== {api-request-title}
`GET /_xpack/usage`
[DOCS] Add security privileges to API docs (#67939)
2021-01-27 22:06:06 +08:00
[discrete]
[[usage-api-prereqs]]
=== {api-prereq-title}
* If the {es} {security-features} are enabled, you must have the `monitor` or
`manage` <<privileges-list-cluster,cluster privilege>> to use this API.
[DOCS] Adds X-Pack usage API (#52496)
2020-02-21 01:25:57 +08:00
[discrete]
[[usage-api-desc]]
=== {api-description-title}
This API provides information about which features are currently enabled and
Add analytics plugin usage stats to _xpack/usage (#54911) Adds analytics plugin usage stats to _xpack/usage. Closes #54847
2020-04-14 01:16:12 +08:00
available under the current license and some usage statistics.
[DOCS] Adds X-Pack usage API (#52496)
2020-02-21 01:25:57 +08:00
[discrete]
[[usage-api-query-parms]]
=== {api-query-parms-title}
[DOCS] Replace docdir attribute with es-repo-dir (#57489)
2020-06-02 06:55:05 +08:00
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=master-timeout]
[DOCS] Adds X-Pack usage API (#52496)
2020-02-21 01:25:57 +08:00
[discrete]
[[usage-api-example]]
=== {api-examples-title}
[source,console]
------------------------------------------------------------
GET /_xpack/usage
------------------------------------------------------------
[DOCS] Fixing Painless tests (#68157) * Fixing Painless tests. * Update runtime field context to fix test cases. * Remove watcher logging from usage API and replace test. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2021-02-10 00:24:14 +08:00
// TEST[s/usage/usage?filter_path=-watcher.execution.actions.logging*/]
// This response filter removes watcher logging results if they are included
// to avoid errors in the CI builds.
[DOCS] Adds X-Pack usage API (#52496)
2020-02-21 01:25:57 +08:00
[source,console-result]
------------------------------------------------------------
{
"security" : {
"available" : true,
"enabled" : false,
"ssl" : {
"http" : {
"enabled" : false
},
"transport" : {
"enabled" : false
}
}
},
"monitoring" : {
"available" : true,
"enabled" : true,
"collection_enabled" : false,
"enabled_exporters" : {
"local" : 1
}
},
"watcher" : {
[DOCS] Update Painless examples to use seat data (#68029) * Adds datetime as a date, which is necessary in setup. * Updating field context example. * Fixing sample data, updating context example, and updating runtime example. * Updating field context and changing runtime field to use seats data. * Update filter context to use the seats data. * Updating min-should-match context to use seats data. * Replacing last mentions of TEST[skip]. * Update usage with watcher response for build error. * Updating usage API again for watcher. * Third time's a charm for fixing test cases. * Adding specific test replacement for watcher logging total. * Change actors to keyword based on review feedback. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2021-01-28 05:42:22 +08:00
"available" : true,
"enabled" : true,
"execution" : {
"actions" : {
"_all" : {
"total" : 0,
"total_time_in_ms" : 0
}
[DOCS] Adds X-Pack usage API (#52496)
2020-02-21 01:25:57 +08:00
}
[DOCS] Update Painless examples to use seat data (#68029) * Adds datetime as a date, which is necessary in setup. * Updating field context example. * Fixing sample data, updating context example, and updating runtime example. * Updating field context and changing runtime field to use seats data. * Update filter context to use the seats data. * Updating min-should-match context to use seats data. * Replacing last mentions of TEST[skip]. * Update usage with watcher response for build error. * Updating usage API again for watcher. * Third time's a charm for fixing test cases. * Adding specific test replacement for watcher logging total. * Change actors to keyword based on review feedback. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2021-01-28 05:42:22 +08:00
},
[DOCS] Adds X-Pack usage API (#52496)
2020-02-21 01:25:57 +08:00
"watch" : {
"input" : {
"_all" : {
"total" : 0,
"active" : 0
}
},
"trigger" : {
"_all" : {
"total" : 0,
"active" : 0
}
}
},
"count" : {
"total" : 0,
"active" : 0
}
},
"graph" : {
"available" : true,
"enabled" : true
},
"ml" : {
"available" : true,
"enabled" : true,
"jobs" : {
"_all" : {
"count" : 0,
"detectors" : {
...
},
"created_by" : { },
"model_size" : {
...
},
"forecasts" : {
"total" : 0,
"forecasted_jobs" : 0
}
}
},
"datafeeds" : {
"_all" : {
"count" : 0
}
},
"data_frame_analytics_jobs" : {
"_all" : {
"count" : 0
[ML] Expand usage stats for data frame analytics and trained models (#69477) This adds additional statistics into the usage API for data frame analytics and trained models. For data frame analytics the added stats are: - count of jobs by analysis type - stats for peak_usage_bytes For trained models the added stats are: - counts of: total, prepackaged, other (not created by data frame analytics) - counts by analysis type based on the inference config - stats for estimated heap usage - stats for estimated number of operations
2021-02-24 21:45:26 +08:00
},
"analysis_counts": { },
"memory_usage": {
"peak_usage_bytes": {
"min": 0.0,
"max": 0.0,
"avg": 0.0,
"total": 0.0
}
[DOCS] Adds X-Pack usage API (#52496)
2020-02-21 01:25:57 +08:00
}
},
"inference" : {
"ingest_processors" : {
"_all" : {
"num_docs_processed" : {
"max" : 0,
"sum" : 0,
"min" : 0
},
"pipelines" : {
"count" : 0
},
"num_failures" : {
"max" : 0,
"sum" : 0,
"min" : 0
},
"time_ms" : {
"max" : 0,
"sum" : 0,
"min" : 0
}
}
},
"trained_models" : {
"_all" : {
"count" : 0
[ML] Expand usage stats for data frame analytics and trained models (#69477) This adds additional statistics into the usage API for data frame analytics and trained models. For data frame analytics the added stats are: - count of jobs by analysis type - stats for peak_usage_bytes For trained models the added stats are: - counts of: total, prepackaged, other (not created by data frame analytics) - counts by analysis type based on the inference config - stats for estimated heap usage - stats for estimated number of operations
2021-02-24 21:45:26 +08:00
},
"count": {
"total": 1,
"classification": 0,
"regression": 0,
"prepackaged": 1,
"other": 0
},
"estimated_heap_memory_usage_bytes": {
"min": 0.0,
"max": 0.0,
"avg": 0.0,
"total": 0.0
},
"estimated_operations": {
"min": 0.0,
"max": 0.0,
"avg": 0.0,
"total": 0.0
[DOCS] Adds X-Pack usage API (#52496)
2020-02-21 01:25:57 +08:00
}
}
},
"node_count" : 1
},
"logstash" : {
"available" : true,
"enabled" : true
},
"eql" : {
"available" : true,
"enabled" : true
},
"sql" : {
"available" : true,
"enabled" : true,
"features" : {
"having" : 0,
"subselect" : 0,
"limit" : 0,
"orderby" : 0,
"where" : 0,
"join" : 0,
"groupby" : 0,
"command" : 0,
"local" : 0
},
"queries" : {
"rest" : {
"total" : 0,
"paging" : 0,
"failed" : 0
},
"cli" : {
"total" : 0,
"paging" : 0,
"failed" : 0
},
"canvas" : {
"total" : 0,
"paging" : 0,
"failed" : 0
},
"odbc" : {
"total" : 0,
"paging" : 0,
"failed" : 0
},
"jdbc" : {
"total" : 0,
"paging" : 0,
"failed" : 0
},
"odbc32" : {
"total" : 0,
"paging" : 0,
"failed" : 0
},
"odbc64" : {
"total" : 0,
"paging" : 0,
"failed" : 0
},
"_all" : {
"total" : 0,
"paging" : 0,
"failed" : 0
},
"translate" : {
"count" : 0
}
}
},
"rollup" : {
"available" : true,
"enabled" : true
},
"ilm" : {
"policy_count" : 3,
"policy_stats" : [
...
]
},
"slm" : {
"available" : true,
"enabled" : true
},
"ccr" : {
"available" : true,
"enabled" : true,
"follower_indices_count" : 0,
"auto_follow_patterns_count" : 0
},
"transform" : {
"available" : true,
"enabled" : true
},
"vectors" : {
"available" : true,
"enabled" : true,
"dense_vector_fields_count" : 0,
"dense_vector_dims_avg_count" : 0
},
"voting_only" : {
"available" : true,
"enabled" : true
},
Add basic searchable snapshots usage information (#58828) Adds super basic usage information for searchable snapshots, to be extended later.
2020-07-07 16:41:42 +08:00
"searchable_snapshots" : {
"available" : true,
"enabled" : true,
Extend searchable snapshots feature usage stats (#70441) Extends the feature usage stats, distinguishing between full_copy and shared_cache style searchable snapshot indices.
2021-03-17 01:02:34 +08:00
"indices_count" : 0,
"full_copy_indices_count" : 0,
"shared_cache_indices_count" : 0
Add basic searchable snapshots usage information (#58828) Adds super basic usage information for searchable snapshots, to be extended later.
2020-07-07 16:41:42 +08:00
},
[DOCS] Adds X-Pack usage API (#52496)
2020-02-21 01:25:57 +08:00
"frozen_indices" : {
"available" : true,
"enabled" : true,
"indices_count" : 0
},
"spatial" : {
"available" : true,
"enabled" : true
},
"analytics" : {
"available" : true,
Add analytics plugin usage stats to _xpack/usage (#54911) Adds analytics plugin usage stats to _xpack/usage. Closes #54847
2020-04-14 01:16:12 +08:00
"enabled" : true,
"stats": {
"boxplot_usage" : 0,
"top_metrics_usage" : 0,
Add Normalize Pipeline Aggregation (#56399) This aggregation will perform normalizations of metrics for a given series of data in the form of bucket values. The aggregations supports the following normalizations - rescale 0-1 - rescale 0-100 - percentage of sum - mean normalization - z-score normalization - softmax normalization To specify which normalization is to be used, it can be specified in the normalize agg's `normalizer` field. For example: ``` { "normalize": { "buckets_path": <>, "normalizer": "percent" } } ``` Closes #51005.
2020-05-15 04:32:42 +08:00
"normalize_usage" : 0,
Add analytics plugin usage stats to _xpack/usage (#54911) Adds analytics plugin usage stats to _xpack/usage. Closes #54847
2020-04-14 01:16:12 +08:00
"cumulative_cardinality_usage" : 0,
"t_test_usage" : 0,
Add rate aggregation (#61369) Adds a new rate aggregation that can calculate a document rate for buckets of a date_histogram. Closes #60674
2020-08-25 23:32:20 +08:00
"rate_usage" : 0,
Add moving percentiles pipeline aggregation (#55441) Similar to what the moving function aggregation does, except merging windows of percentiles sketches together instead of cumulatively merging final metrics
2020-05-12 16:30:52 +08:00
"string_stats_usage" : 0,
Add multi_terms aggs (#67597) Adds a multi_terms aggregation support. The multi terms aggregation works very similarly to the terms aggregation but supports multiple terms. The goal of this PR is to add the basic functionality so it is not optimized at the moment. It will be done in follow up PRs. Closes #65623
2021-02-04 02:13:33 +08:00
"moving_percentiles_usage" : 0,
"multi_terms_usage" : 0
Add analytics plugin usage stats to _xpack/usage (#54911) Adds analytics plugin usage stats to _xpack/usage. Closes #54847
2020-04-14 01:16:12 +08:00
}
Add telemetery for data streams (#59433) This commit adds data stream info to the `/_xpack` and `/_xpack/usage` APIs. Currently the usage is pretty minimal, returning only the number of data streams and the number of indices currently abstracted by a data stream: ``` ... "data_streams" : { "available" : true, "enabled" : true, "data_streams" : 3, "indices_count" : 17 } ... ```
2020-07-14 02:12:22 +08:00
},
"data_streams" : {
"available" : true,
"enabled" : true,
"data_streams" : 0,
"indices_count" : 0
Add telemetry for data tiers (#63031) This commit adds telemetry for our data tier formalization. This telemetry helps determine the topology of the cluster with regard to the content, hot, warm, & cold tiers/roles. An example of the telemetry looks like: ``` GET /_xpack/usage?human { ... "data_tiers" : { "available" : true, "enabled" : true, "data_warm" : { ... }, "data_cold" : { ... }, "data_content" : { "node_count" : 1, "index_count" : 6, "total_shard_count" : 6, "primary_shard_count" : 6, "doc_count" : 71, "total_size" : "59.6kb", "total_size_bytes" : 61110, "primary_size" : "59.6kb", "primary_size_bytes" : 61110, "primary_shard_size_avg" : "9.9kb", "primary_shard_size_avg_bytes" : 10185, "primary_shard_size_median" : "8kb", "primary_shard_size_median_bytes" : 8254, "primary_shard_size_mad" : "7.2kb", "primary_shard_size_mad_bytes" : 7391 }, "data_hot" : { ... } } } ``` The fields are as follows: - node_count :: number of nodes with this tier/role - index_count :: number of indices on this tier - total_shard_count :: total number of shards for all nodes in this tier - primary_shard_count :: number of primary shards for all nodes in this tier - doc_count :: number of documents for all nodes in this tier - total_size_bytes :: total number of bytes for all shards for all nodes in this tier - primary_size_bytes :: number of bytes for all primary shards on all nodes in this tier - primary_shard_size_avg_bytes :: average shard size for primary shard in this tier - primary_shard_size_median_bytes :: median shard size for primary shard in this tier - primary_shard_size_mad_bytes :: [median absolute deviation](https://en.wikipedia.org/wiki/Median_absolute_deviation) of shard size for primary shard in this tier Relates to #60848
2020-10-01 22:35:10 +08:00
},
"data_tiers" : {
"available" : true,
"enabled" : true,
"data_warm" : {
"node_count" : 0,
"index_count" : 0,
"total_shard_count" : 0,
"primary_shard_count" : 0,
"doc_count" : 0,
"total_size_bytes" : 0,
"primary_size_bytes" : 0,
"primary_shard_size_avg_bytes" : 0,
"primary_shard_size_median_bytes" : 0,
"primary_shard_size_mad_bytes" : 0
},
Add the frozen tier node role and ILM phase (#68605) This commit adds the `data_frozen` node role as part of the formalization of data tiers. It also adds the `"frozen"` phase to ILM, currently allowing the same actions as the existing cold phase. The frozen phase is intended to be used for data even less frequently searched than the cold phase, and will eventually be loosely tied to data using partial searchable snapshots (as oppposed to full searchable snapshots in the cold phase). Relates to #60848
2021-02-06 05:38:13 +08:00
"data_frozen" : {
"node_count" : 1,
"index_count" : 0,
"total_shard_count" : 0,
"primary_shard_count" : 0,
"doc_count" : 0,
"total_size_bytes" : 0,
"primary_size_bytes" : 0,
"primary_shard_size_avg_bytes" : 0,
"primary_shard_size_median_bytes" : 0,
"primary_shard_size_mad_bytes" : 0
},
Add telemetry for data tiers (#63031) This commit adds telemetry for our data tier formalization. This telemetry helps determine the topology of the cluster with regard to the content, hot, warm, & cold tiers/roles. An example of the telemetry looks like: ``` GET /_xpack/usage?human { ... "data_tiers" : { "available" : true, "enabled" : true, "data_warm" : { ... }, "data_cold" : { ... }, "data_content" : { "node_count" : 1, "index_count" : 6, "total_shard_count" : 6, "primary_shard_count" : 6, "doc_count" : 71, "total_size" : "59.6kb", "total_size_bytes" : 61110, "primary_size" : "59.6kb", "primary_size_bytes" : 61110, "primary_shard_size_avg" : "9.9kb", "primary_shard_size_avg_bytes" : 10185, "primary_shard_size_median" : "8kb", "primary_shard_size_median_bytes" : 8254, "primary_shard_size_mad" : "7.2kb", "primary_shard_size_mad_bytes" : 7391 }, "data_hot" : { ... } } } ``` The fields are as follows: - node_count :: number of nodes with this tier/role - index_count :: number of indices on this tier - total_shard_count :: total number of shards for all nodes in this tier - primary_shard_count :: number of primary shards for all nodes in this tier - doc_count :: number of documents for all nodes in this tier - total_size_bytes :: total number of bytes for all shards for all nodes in this tier - primary_size_bytes :: number of bytes for all primary shards on all nodes in this tier - primary_shard_size_avg_bytes :: average shard size for primary shard in this tier - primary_shard_size_median_bytes :: median shard size for primary shard in this tier - primary_shard_size_mad_bytes :: [median absolute deviation](https://en.wikipedia.org/wiki/Median_absolute_deviation) of shard size for primary shard in this tier Relates to #60848
2020-10-01 22:35:10 +08:00
"data_cold" : {
"node_count" : 0,
"index_count" : 0,
"total_shard_count" : 0,
"primary_shard_count" : 0,
"doc_count" : 0,
"total_size_bytes" : 0,
"primary_size_bytes" : 0,
"primary_shard_size_avg_bytes" : 0,
"primary_shard_size_median_bytes" : 0,
"primary_shard_size_mad_bytes" : 0
},
"data_content" : {
"node_count" : 0,
"index_count" : 0,
"total_shard_count" : 0,
"primary_shard_count" : 0,
"doc_count" : 0,
"total_size_bytes" : 0,
"primary_size_bytes" : 0,
"primary_shard_size_avg_bytes" : 0,
"primary_shard_size_median_bytes" : 0,
"primary_shard_size_mad_bytes" : 0
},
"data_hot" : {
"node_count" : 0,
"index_count" : 0,
"total_shard_count" : 0,
"primary_shard_count" : 0,
"doc_count" : 0,
"total_size_bytes" : 0,
"primary_size_bytes" : 0,
"primary_shard_size_avg_bytes" : 0,
"primary_shard_size_median_bytes" : 0,
"primary_shard_size_mad_bytes" : 0
}
Implement aggregations on aggregate metric fields (#56745) In the process of developing a new implementation for the Elasticsearch Rollups functionality we came up with the concept of the aggregate metric field type. The aggregate_metric_double field type can store the results of aggregations (currently min, max, sum, value_count and avg are supported - more to come). This field allows us to run (min, max, sum, value_count, avg) aggregations on the container field and the field will return the correct metric depending on the aggregation that is computed.
2020-11-11 01:31:59 +08:00
},
"aggregate_metric" : {
"available" : true,
"enabled" : true
[DOCS] Adds X-Pack usage API (#52496)
2020-02-21 01:25:57 +08:00
}
}
------------------------------------------------------------
// TESTRESPONSE[s/"detectors" : \{[^\}]*\},/"detectors" : $body.$_path,/]
// TESTRESPONSE[s/"model_size" : \{[^\}]*\},/"model_size" : $body.$_path,/]
// TESTRESPONSE[s/"eql" : \{[^\}]*\},/"eql" : $body.$_path,/]
// TESTRESPONSE[s/"policy_stats" : \[[^\]]*\]/"policy_stats" : $body.$_path/]
// TESTRESPONSE[s/"slm" : \{[^\}]*\},/"slm" : $body.$_path,/]
// TESTRESPONSE[s/ : true/ : $body.$_path/]
// TESTRESPONSE[s/ : false/ : $body.$_path/]
// TESTRESPONSE[s/ : (\-)?[0-9]+/ : $body.$_path/]
[DOCS] Fixing Painless tests (#68157) * Fixing Painless tests. * Update runtime field context to fix test cases. * Remove watcher logging from usage API and replace test. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2021-02-10 00:24:14 +08:00
[DOCS] Adds X-Pack usage API (#52496)
2020-02-21 01:25:57 +08:00
// These replacements do a few things:
// 1. Ignore the contents of the `policy_stats` object because we don't know all
// of the policies that will be in it. And because we figure folks don't need
// to see an exhaustive list anyway.
// 2. Handling eql, which is disabled by default on release builds and enabled
// everywhere else during the initial implementation phase until its release
// 3. Ignore the contents of the `slm` object because it might contain policies
// 4. All of the numbers and strings on the right hand side of *every* field in
// the response are ignored. So we're really only asserting things about the
Remove xpack actions for the flattened field. (#53076) The current consensus is that we don't need info actions for smaller items like field mappers. We can also remove the usage action since the cluster stats API now tracks information about mappings, like what field types are defined.
2020-03-05 00:35:03 +08:00
// the shape of this response, not the values in it.