127 lines
3.5 KiB
Plaintext
127 lines
3.5 KiB
Plaintext
[[esql-enrich-data]]
|
|
=== Enrich data
|
|
|
|
++++
|
|
<titleabbrev>Enrich data</titleabbrev>
|
|
++++
|
|
|
|
You can use {esql}'s <<esql-enrich>> processing command to enrich a table with
|
|
data from indices in {es}.
|
|
|
|
For example, you can use `ENRICH` to:
|
|
|
|
* Identify web services or vendors based on known IP addresses
|
|
* Add product information to retail orders based on product IDs
|
|
* Supplement contact information based on an email address
|
|
|
|
[[esql-how-enrich-works]]
|
|
==== How the `ENRICH` command works
|
|
|
|
The `ENRICH` command adds new columns to a table, with data from {es} indices.
|
|
It requires a few special components:
|
|
|
|
image::images/esql/esql-enrich.png[align="center"]
|
|
|
|
[[esql-enrich-policy]]
|
|
Enrich policy::
|
|
+
|
|
--
|
|
A set of configuration options used to add the right enrich data to the input
|
|
table.
|
|
|
|
An enrich policy contains:
|
|
|
|
include::../ingest/enrich.asciidoc[tag=enrich-policy-fields]
|
|
|
|
After <<esql-create-enrich-policy,creating a policy>>, it must be
|
|
<<esql-execute-enrich-policy,executed>> before it can be used. Executing an
|
|
enrich policy uses data from the policy's source indices to create a streamlined
|
|
system index called the _enrich index_. The `ENRICH` command uses this index to
|
|
match and enrich an input table.
|
|
--
|
|
|
|
[[esql-source-index]]
|
|
Source index::
|
|
An index which stores enrich data that the `ENRICH` command can add to input
|
|
tables. You can create and manage these indices just like a regular {es} index.
|
|
You can use multiple source indices in an enrich policy. You also can use the
|
|
same source index in multiple enrich policies.
|
|
|
|
[[esql-enrich-index]]
|
|
Enrich index::
|
|
+
|
|
--
|
|
A special system index tied to a specific enrich policy.
|
|
|
|
Directly matching rows from input tables to documents in source indices could be
|
|
slow and resource intensive. To speed things up, the `ENRICH` command uses an
|
|
enrich index.
|
|
|
|
include::../ingest/enrich.asciidoc[tag=enrich-index]
|
|
--
|
|
|
|
[[esql-set-up-enrich-policy]]
|
|
==== Set up an enrich policy
|
|
|
|
To start using `ENRICH`, follow these steps:
|
|
|
|
. Check the <<enrich-prereqs, prerequisites>>.
|
|
. <<esql-create-enrich-source-index>>.
|
|
. <<esql-create-enrich-policy>>.
|
|
. <<esql-execute-enrich-policy>>.
|
|
. <<esql-use-enrich>>
|
|
|
|
Once you have enrich policies set up, you can <<esql-update-enrich-data,update
|
|
your enrich data>> and <<esql-update-enrich-policies, update your enrich
|
|
policies>>.
|
|
|
|
[IMPORTANT]
|
|
====
|
|
The `ENRICH` command performs several operations and may impact the speed of
|
|
your query.
|
|
====
|
|
|
|
[[esql-enrich-prereqs]]
|
|
==== Prerequisites
|
|
|
|
include::{es-repo-dir}/ingest/apis/enrich/put-enrich-policy.asciidoc[tag=enrich-policy-api-prereqs]
|
|
|
|
[[esql-create-enrich-source-index]]
|
|
==== Add enrich data
|
|
|
|
include::../ingest/enrich.asciidoc[tag=create-enrich-source-index]
|
|
|
|
[[esql-create-enrich-policy]]
|
|
==== Create an enrich policy
|
|
|
|
include::../ingest/enrich.asciidoc[tag=create-enrich-policy]
|
|
|
|
[[esql-execute-enrich-policy]]
|
|
==== Execute the enrich policy
|
|
|
|
include::../ingest/enrich.asciidoc[tag=execute-enrich-policy1]
|
|
|
|
image::images/esql/esql-enrich-policy.png[align="center"]
|
|
|
|
include::../ingest/enrich.asciidoc[tag=execute-enrich-policy2]
|
|
|
|
[[esql-use-enrich]]
|
|
==== Use the enrich policy
|
|
|
|
After the policy has been executed, you can use the <<esql-enrich,`ENRICH`
|
|
command>> to enrich your data.
|
|
|
|
image::images/esql/esql-enrich-command.png[align="center",width=50%]
|
|
|
|
include::processing-commands/enrich.asciidoc[tag=examples]
|
|
|
|
[[esql-update-enrich-data]]
|
|
==== Update an enrich index
|
|
|
|
include::{es-repo-dir}/ingest/apis/enrich/execute-enrich-policy.asciidoc[tag=update-enrich-index]
|
|
|
|
[[esql-update-enrich-policies]]
|
|
==== Update an enrich policy
|
|
|
|
include::../ingest/enrich.asciidoc[tag=update-enrich-policy]
|