root
/
kafka
mirror of https://github.com/apache/kafka.git
1
0
Fork 0
Code Issues Actions 2 Packages Projects Releases Wiki Activity

KAFKA-16358: Sort transformations by name in documentation; add missing transformations to documentation; add hyperlinks (#15499) 

Reviewers: Yash Mayya <yash.mayya@gmail.com>
This commit is contained in:
Hector Geraldino 2024-03-15 06:56:36 -04:00 committed by GitHub
parent 96bfac4216
commit 7b9f31e35c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 30 additions and 27 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
connect/runtime/src/main/java/org/apache/kafka/connect/tools/TransformationDoc.java
View File

@ -53,22 +53,22 @@ public class TransformationDoc {
} }
private static final List<DocInfo> TRANSFORMATIONS = Arrays.asList( private static final List<DocInfo> TRANSFORMATIONS = Arrays.asList(
new DocInfo(InsertField.class.getName(), InsertField.OVERVIEW_DOC, InsertField.CONFIG_DEF),
new DocInfo(ReplaceField.class.getName(), ReplaceField.OVERVIEW_DOC, ReplaceField.CONFIG_DEF),
new DocInfo(MaskField.class.getName(), MaskField.OVERVIEW_DOC, MaskField.CONFIG_DEF),
new DocInfo(ValueToKey.class.getName(), ValueToKey.OVERVIEW_DOC, ValueToKey.CONFIG_DEF),
new DocInfo(HoistField.class.getName(), HoistField.OVERVIEW_DOC, HoistField.CONFIG_DEF),
new DocInfo(ExtractField.class.getName(), ExtractField.OVERVIEW_DOC, ExtractField.CONFIG_DEF),
new DocInfo(SetSchemaMetadata.class.getName(), SetSchemaMetadata.OVERVIEW_DOC, SetSchemaMetadata.CONFIG_DEF),
new DocInfo(TimestampRouter.class.getName(), TimestampRouter.OVERVIEW_DOC, TimestampRouter.CONFIG_DEF),
new DocInfo(RegexRouter.class.getName(), RegexRouter.OVERVIEW_DOC, RegexRouter.CONFIG_DEF),
new DocInfo(Flatten.class.getName(), Flatten.OVERVIEW_DOC, Flatten.CONFIG_DEF),
new DocInfo(Cast.class.getName(), Cast.OVERVIEW_DOC, Cast.CONFIG_DEF), new DocInfo(Cast.class.getName(), Cast.OVERVIEW_DOC, Cast.CONFIG_DEF),
new DocInfo(TimestampConverter.class.getName(), TimestampConverter.OVERVIEW_DOC, TimestampConverter.CONFIG_DEF),
new DocInfo(Filter.class.getName(), Filter.OVERVIEW_DOC, Filter.CONFIG_DEF),
new DocInfo(InsertHeader.class.getName(), InsertHeader.OVERVIEW_DOC, InsertHeader.CONFIG_DEF),
new DocInfo(DropHeaders.class.getName(), DropHeaders.OVERVIEW_DOC, DropHeaders.CONFIG_DEF), new DocInfo(DropHeaders.class.getName(), DropHeaders.OVERVIEW_DOC, DropHeaders.CONFIG_DEF),
new DocInfo(HeaderFrom.class.getName(), HeaderFrom.OVERVIEW_DOC, HeaderFrom.CONFIG_DEF) new DocInfo(ExtractField.class.getName(), ExtractField.OVERVIEW_DOC, ExtractField.CONFIG_DEF),
new DocInfo(Filter.class.getName(), Filter.OVERVIEW_DOC, Filter.CONFIG_DEF),
new DocInfo(Flatten.class.getName(), Flatten.OVERVIEW_DOC, Flatten.CONFIG_DEF),
new DocInfo(HeaderFrom.class.getName(), HeaderFrom.OVERVIEW_DOC, HeaderFrom.CONFIG_DEF),
new DocInfo(HoistField.class.getName(), HoistField.OVERVIEW_DOC, HoistField.CONFIG_DEF),
new DocInfo(InsertField.class.getName(), InsertField.OVERVIEW_DOC, InsertField.CONFIG_DEF),
new DocInfo(InsertHeader.class.getName(), InsertHeader.OVERVIEW_DOC, InsertHeader.CONFIG_DEF),
new DocInfo(MaskField.class.getName(), MaskField.OVERVIEW_DOC, MaskField.CONFIG_DEF),
new DocInfo(RegexRouter.class.getName(), RegexRouter.OVERVIEW_DOC, RegexRouter.CONFIG_DEF),
new DocInfo(ReplaceField.class.getName(), ReplaceField.OVERVIEW_DOC, ReplaceField.CONFIG_DEF),
new DocInfo(SetSchemaMetadata.class.getName(), SetSchemaMetadata.OVERVIEW_DOC, SetSchemaMetadata.CONFIG_DEF),
new DocInfo(TimestampConverter.class.getName(), TimestampConverter.OVERVIEW_DOC, TimestampConverter.CONFIG_DEF),
new DocInfo(TimestampRouter.class.getName(), TimestampRouter.OVERVIEW_DOC, TimestampRouter.CONFIG_DEF),
new DocInfo(ValueToKey.class.getName(), ValueToKey.OVERVIEW_DOC, ValueToKey.CONFIG_DEF)
); );
private static void printTransformationHtml(PrintStream out, DocInfo docInfo) { private static void printTransformationHtml(PrintStream out, DocInfo docInfo) {

29
docs/connect.html
View File

@ -162,19 +162,22 @@ transforms.InsertSource.static.value=test-file-source</code></pre>
<p>Several widely-applicable data and routing transformations are included with Kafka Connect:</p> <p>Several widely-applicable data and routing transformations are included with Kafka Connect:</p>
<ul> <ul>
<li>InsertField - Add a field using either static data or record metadata</li> <li><a href="#org.apache.kafka.connect.transforms.Cast">Cast</a> - Cast fields or the entire key or value to a specific type</li>
<li>ReplaceField - Filter or rename fields</li> <li><a href="#org.apache.kafka.connect.transforms.DropHeaders">DropHeaders</a> - Remove headers by name</li>
<li>MaskField - Replace field with valid null value for the type (0, empty string, etc) or custom replacement (non-empty string or numeric value only)</li> <li><a href="#org.apache.kafka.connect.transforms.ExtractField">ExtractField</a> - Extract a specific field from Struct and Map and include only this field in results</li>
<li>ValueToKey - Replace the record key with a new key formed from a subset of fields in the record value</li> <li><a href="#org.apache.kafka.connect.transforms.Filter">Filter</a> - Removes messages from all further processing. This is used with a <a href="#connect_predicates">predicate</a> to selectively filter certain messages</li>
<li>HoistField - Wrap the entire event as a single field inside a Struct or a Map</li> <li><a href="#org.apache.kafka.connect.transforms.Flatten">Flatten</a> - Flatten a nested data structure</li>
<li>ExtractField - Extract a specific field from Struct and Map and include only this field in results</li> <li><a href="#org.apache.kafka.connect.transforms.HeaderFrom">HeaderFrom</a> - Copy or move fields in the key or value to the record headers</li>
<li>SetSchemaMetadata - modify the schema name or version</li> <li><a href="#org.apache.kafka.connect.transforms.HoistField">HoistField</a> - Wrap the entire event as a single field inside a Struct or a Map</li>
<li>TimestampRouter - Modify the topic of a record based on original topic and timestamp. Useful when using a sink that needs to write to different tables or indexes based on timestamps</li> <li><a href="#org.apache.kafka.connect.transforms.InsertField">InsertField</a> - Add a field using either static data or record metadata</li>
<li>RegexRouter - modify the topic of a record based on original topic, replacement string and a regular expression</li> <li><a href="#org.apache.kafka.connect.transforms.InsertHeader">InsertHeader</a> - Add a header using static data</li>
<li>Filter - Removes messages from all further processing. This is used with a <a href="#connect_predicates">predicate</a> to selectively filter certain messages.</li> <li><a href="#org.apache.kafka.connect.transforms.MaskField">MaskField</a> - Replace field with valid null value for the type (0, empty string, etc) or custom replacement (non-empty string or numeric value only)</li>
<li>InsertHeader - Add a header using static data</li> <li><a href="#org.apache.kafka.connect.transforms.RegexRouter">RegexRouter</a> - modify the topic of a record based on original topic, replacement string and a regular expression</li>
<li>HeadersFrom - Copy or move fields in the key or value to the record headers</li> <li><a href="#org.apache.kafka.connect.transforms.ReplaceField">ReplaceField</a> - Filter or rename fields</li>
<li>DropHeaders - Remove headers by name</li> <li><a href="#org.apache.kafka.connect.transforms.SetSchemaMetadata">SetSchemaMetadata</a> - modify the schema name or version</li>
<li><a href="#org.apache.kafka.connect.transforms.TimestampConverter">TimestampConverter</a> - Convert timestamps between different formats</li>
<li><a href="#org.apache.kafka.connect.transforms.TimestampRouter">TimestampRouter</a> - Modify the topic of a record based on original topic and timestamp. Useful when using a sink that needs to write to different tables or indexes based on timestamps</li>
<li><a href="#org.apache.kafka.connect.transforms.ValueToKey">ValueToKey</a> - Replace the record key with a new key formed from a subset of fields in the record value</li>
</ul> </ul>
<p>Details on how to configure each transformation are listed below:</p> <p>Details on how to configure each transformation are listed below:</p>