2019-12-19 19:19:44 +08:00
[role="xpack"]
[testenv="basic"]
[[inference-processor]]
2020-08-12 23:28:00 +08:00
=== {infer-cap} processor
++++
<titleabbrev>{infer-cap}</titleabbrev>
++++
2019-12-19 19:19:44 +08:00
2020-09-30 22:20:38 +08:00
2020-07-03 15:36:57 +08:00
Uses a pre-trained {dfanalytics} model to infer against the data that is being
2019-12-19 19:19:44 +08:00
ingested in the pipeline.
[[inference-options]]
.{infer-cap} Options
[options="header"]
|======
| Name | Required | Default | Description
2021-02-25 05:12:39 +08:00
| `model_id` | yes | - | (String) The ID or alias for the trained model.
2019-12-19 19:19:44 +08:00
| `target_field` | no | `ml.inference.<processor_tag>` | (String) Field added to incoming documents to contain results objects.
2020-07-03 15:36:57 +08:00
| `field_map` | no | If defined the model's default field map | (Object) Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.
| `inference_config` | no | The default settings defined in the model | (Object) Contains the inference type and its options. There are two types: <<inference-processor-regression-opt,`regression`>> and <<inference-processor-classification-opt,`classification`>>.
2019-12-19 19:19:44 +08:00
include::common-options.asciidoc[]
|======
[source,js]
--------------------------------------------------
{
2020-01-21 23:21:17 +08:00
"inference": {
"model_id": "flight_delay_regression-1571767128603",
"target_field": "FlightDelayMin_prediction_infer",
2020-07-03 15:36:57 +08:00
"field_map": {
"your_field": "my_field"
},
2020-02-05 18:49:36 +08:00
"inference_config": { "regression": {} }
2020-01-21 23:21:17 +08:00
}
2019-12-19 19:19:44 +08:00
}
--------------------------------------------------
// NOTCONSOLE
[discrete]
[[inference-processor-regression-opt]]
==== {regression-cap} configuration options
2020-04-02 22:34:17 +08:00
Regression configuration for inference.
2019-12-19 19:19:44 +08:00
`results_field`::
(Optional, string)
2020-06-29 17:28:17 +08:00
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=inference-config-results-field-processor]
2019-12-19 19:19:44 +08:00
2020-04-02 22:34:17 +08:00
`num_top_feature_importance_values`::
2020-02-22 05:36:21 +08:00
(Optional, integer)
2020-06-02 06:55:05 +08:00
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=inference-config-regression-num-top-feature-importance-values]
2019-12-19 19:19:44 +08:00
2020-03-26 16:22:12 +08:00
2019-12-19 19:19:44 +08:00
[discrete]
[[inference-processor-classification-opt]]
==== {classification-cap} configuration options
2020-04-02 22:34:17 +08:00
Classification configuration for inference.
2019-12-19 19:19:44 +08:00
`num_top_classes`::
2020-04-02 22:34:17 +08:00
(Optional, integer)
2020-06-02 06:55:05 +08:00
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=inference-config-classification-num-top-classes]
2019-12-19 19:19:44 +08:00
2020-04-02 22:34:17 +08:00
`num_top_feature_importance_values`::
2020-02-22 05:36:21 +08:00
(Optional, integer)
2020-06-02 06:55:05 +08:00
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=inference-config-classification-num-top-feature-importance-values]
2020-03-26 16:22:12 +08:00
2020-04-02 22:34:17 +08:00
`results_field`::
(Optional, string)
2020-06-29 17:28:17 +08:00
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=inference-config-results-field-processor]
2020-04-02 22:34:17 +08:00
`top_classes_results_field`::
(Optional, string)
2020-06-02 06:55:05 +08:00
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=inference-config-classification-top-classes-results-field]
2019-12-19 19:19:44 +08:00
2020-04-15 20:32:48 +08:00
`prediction_field_type`::
(Optional, string)
2020-06-02 06:55:05 +08:00
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=inference-config-classification-prediction-field-type]
2020-04-15 20:32:48 +08:00
2019-12-19 19:19:44 +08:00
[discrete]
[[inference-processor-config-example]]
==== `inference_config` examples
[source,js]
--------------------------------------------------
2021-01-05 16:42:06 +08:00
"inference":{
"model_id":"my_model_id"
2019-12-19 19:19:44 +08:00
"inference_config": {
2020-01-21 23:21:17 +08:00
"regression": {
"results_field": "my_regression"
2019-12-19 19:19:44 +08:00
}
2020-01-21 23:21:17 +08:00
}
2019-12-19 19:19:44 +08:00
}
--------------------------------------------------
// NOTCONSOLE
2020-07-03 15:36:57 +08:00
This configuration specifies a `regression` inference and the results are
written to the `my_regression` field contained in the `target_field` results
2019-12-19 19:19:44 +08:00
object.
[source,js]
--------------------------------------------------
2021-01-05 16:42:06 +08:00
"inference":{
"model_id":"my_model_id"
2019-12-19 19:19:44 +08:00
"inference_config": {
2020-01-21 23:21:17 +08:00
"classification": {
"num_top_classes": 2,
"results_field": "prediction",
"top_classes_results_field": "probabilities"
2019-12-19 19:19:44 +08:00
}
2020-01-21 23:21:17 +08:00
}
2019-12-19 19:19:44 +08:00
}
--------------------------------------------------
// NOTCONSOLE
2020-07-03 15:36:57 +08:00
This configuration specifies a `classification` inference. The number of
categories for which the predicted probabilities are reported is 2
(`num_top_classes`). The result is written to the `prediction` field and the top
classes to the `probabilities` field. Both fields are contained in the
2019-12-19 19:19:44 +08:00
`target_field` results object.
2020-03-26 16:22:12 +08:00
2021-01-05 16:42:06 +08:00
Refer to the
2021-07-08 22:37:16 +08:00
{ml-docs}/ml-dfa-lang-ident.html#ml-lang-ident-example[language identification]
2021-01-05 16:42:06 +08:00
trained model documentation for a full example.
2020-03-26 16:22:12 +08:00
[discrete]
[[inference-processor-feature-importance]]
==== {feat-imp-cap} object mapping
2021-01-05 16:42:06 +08:00
To get the full benefit of aggregating and searching for
{ml-docs}/ml-feature-importance.html[{feat-imp}], update your index mapping of
the {feat-imp} result field as you can see below:
2020-03-26 16:22:12 +08:00
[source,js]
--------------------------------------------------
"ml.inference.feature_importance": {
"type": "nested",
"dynamic": true,
"properties": {
"feature_name": {
"type": "keyword"
},
"importance": {
"type": "double"
}
}
}
--------------------------------------------------
// NOTCONSOLE
2021-01-05 16:42:06 +08:00
The mapping field name for {feat-imp} (in the example above, it is
`ml.inference.feature_importance`) is compounded as follows:
2020-03-26 16:22:12 +08:00
`<ml.inference.target_field>`.`<inference.tag>`.`feature_importance`
2021-01-05 16:42:06 +08:00
* `<ml.inference.target_field>`: defaults to `ml.inference`.
* `<inference.tag>`: if is not provided in the processor definition, then it is
not part of the field path.
2020-03-26 16:22:12 +08:00
2021-01-05 16:42:06 +08:00
For example, if you provide a tag `foo` in the definition as you can see below:
2020-03-26 16:22:12 +08:00
[source,js]
--------------------------------------------------
{
"tag": "foo",
...
}
--------------------------------------------------
// NOTCONSOLE
2021-01-05 16:42:06 +08:00
Then, the {feat-imp} value is written to the
`ml.inference.foo.feature_importance` field.
2020-03-26 16:22:12 +08:00
2021-01-05 16:42:06 +08:00
You can also specify the target field as follows:
2020-03-26 16:22:12 +08:00
[source,js]
--------------------------------------------------
{
"tag": "foo",
"target_field": "my_field"
}
--------------------------------------------------
// NOTCONSOLE
2020-07-03 15:36:57 +08:00
In this case, {feat-imp} is exposed in the
2020-04-02 22:34:17 +08:00
`my_field.foo.feature_importance` field.