mirror of https://github.com/apache/kafka.git
MINOR: Stream metrics documentation
Author: Eno Thereska <eno@confluent.io> Reviewers: Matthias J. Sax, Guozhang Wang Closes #2542 from enothereska/minor-streams-metrics
This commit is contained in:
Eno Thereska
Guozhang Wang
parent
2c91b324d4
commit
2e1b00f262
130
docs/ops.html
130
docs/ops.html
|
|
@ -14,8 +14,8 @@
|
|||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<script id="ops-template" type="text/x-handlebars-template">
|
||||
|
||||
Here is some information on actually running Kafka as a production system based on usage and experience at LinkedIn. Please send us any additional tips you know of.
|
||||
|
||||
<h3><a id="basic_ops" href="#basic_ops">6.1 Basic Kafka Operations</a></h3>
|
||||
|
|
@ -842,9 +842,9 @@
|
|||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
<h4><a id="selector_monitoring" href="#selector_monitoring">Common monitoring metrics for producer/consumer/connect</a></h4>
|
||||
<h4><a id="selector_monitoring" href="#selector_monitoring">Common monitoring metrics for producer/consumer/connect/streams</a></h4>
|
||||
|
||||
The following metrics are available on producer/consumer/connector instances. For specific metrics, please see following sections.
|
||||
The following metrics are available on producer/consumer/connector/streams instances. For specific metrics, please see following sections.
|
||||
|
||||
<table class="data-table">
|
||||
<tbody>
|
||||
|
|
@ -931,9 +931,9 @@
|
|||
</tbody>
|
||||
</table>
|
||||
|
||||
<h4><a id="common_node_monitoring" href="#common_node_monitoring">Common Per-broker metrics for producer/consumer/connect</a></h4>
|
||||
<h4><a id="common_node_monitoring" href="#common_node_monitoring">Common Per-broker metrics for producer/consumer/connect/streams</a></h4>
|
||||
|
||||
The following metrics are available on producer/consumer/connector instances. For specific metrics, please see following sections.
|
||||
The following metrics are available on producer/consumer/connector/streams instances. For specific metrics, please see following sections.
|
||||
|
||||
<table class="data-table">
|
||||
<tbody>
|
||||
|
|
@ -1314,7 +1314,125 @@
|
|||
</tbody>
|
||||
</table>
|
||||
|
||||
<h5><a id="others_monitoring" href="#others_monitoring">Others</a></h5>
|
||||
|
||||
|
||||
<h4><a id="kafka_streams_monitoring" href="#kafka_streams_monitoring">Streams Monitoring</a></h4>
|
||||
|
||||
A Kafka Streams instance contains all the producer and consumer metrics as well as additional metrics specific to streams. By default Kafka Streams has metrics with two recording levels: debug and info. The debug level records all metrics, while the info level records only the thread-level metrics. Use the following configuration option to specify which metrics you want collected:
|
||||
<pre>metrics.recording.level="info"</pre>
|
||||
|
||||
<h5><a id="kafka_streams_thread_monitoring" href="#kafka_streams_thread_monitoring">Thread Metrics</a></h5>
|
||||
All the following metrics have a recording level of ``info``:
|
||||
<table class="data-table">
|
||||
<tbody>
|
||||
<tr>
|
||||
<th>Metric/Attribute name</th>
|
||||
<th>Description</th>
|
||||
<th>Mbean name</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>[commit | poll | process | punctuate]-latency-[avg | max]</td>
|
||||
<td>The [average | maximum] execution time in ms, for the respective operation, across all running tasks of this thread.</td>
|
||||
<td>kafka.streams:type=stream-metrics,thread.client-id=([-.\w]+)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>[commit | poll | process | punctuate]-rate</td>
|
||||
<td>The average number of respective operations per second across all tasks.</td>
|
||||
<td>kafka.streams:type=stream-metrics,thread.client-id=([-.\w]+)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>task-created-rate</td>
|
||||
<td>The average number of newly created tasks per second.</td>
|
||||
<td>kafka.streams:type=stream-metrics,thread.client-id=([-.\w]+)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>task-closed-rate</td>
|
||||
<td>The average number of tasks closed per second.</td>
|
||||
<td>kafka.streams:type=stream-metrics,thread.client-id=([-.\w]+)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>skipped-records-rate</td>
|
||||
<td>The average number of skipped records per second. </td>
|
||||
<td>kafka.streams:type=stream-metrics,thread.client-id=([-.\w]+)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<h5><a id="kafka_streams_task_monitoring" href="#kafka_streams_task_monitoring">Task Metrics</a></h5>
|
||||
All the following metrics have a recording level of ``debug``:
|
||||
<table class="data-table">
|
||||
<tbody>
|
||||
<tr>
|
||||
<th>Metric/Attribute name</th>
|
||||
<th>Description</th>
|
||||
<th>Mbean name</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>commit-latency-[avg | max]</td>
|
||||
<td>The [average | maximum] commit time in ns for this task. </td>
|
||||
<td>kafka.streams:type=stream-task-metrics,streams-task-id=([-.\w]+)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>commit-rate</td>
|
||||
<td>The average number of commit calls per second. </td>
|
||||
<td>kafka.streams:type=stream-task-metrics,streams-task-id=([-.\w]+)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<h5><a id="kafka_streams_node_monitoring" href="#kafka_streams_node_monitoring">Processor Node Metrics</a></h5>
|
||||
All the following metrics have a recording level of ``debug``:
|
||||
<table class="data-table">
|
||||
<tbody>
|
||||
<tr>
|
||||
<th>Metric/Attribute name</th>
|
||||
<th>Description</th>
|
||||
<th>Mbean name</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>[process | punctuate | create | destroy]-latency-[avg | max]</td>
|
||||
<td>The [average | maximum] execution time in ns, for the respective operation. </td>
|
||||
<td>kafka.streams:type=stream-processor-node-metrics, processor-node-id=([-.\w]+)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>[process | punctuate | create | destroy]-rate</td>
|
||||
<td>The average number of respective operations per second. </td>
|
||||
<td>kafka.streams:type=stream-processor-node-metrics, processor-node-id=([-.\w]+)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>forward-rate</td>
|
||||
<td>The average rate of records being forwarded downstream, from source nodes only, per second. </td>
|
||||
<td>kafka.streams:type=stream-processor-node-metrics, processor-node-id=([-.\w]+)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<h5><a id="kafka_streams_store_monitoring" href="#kafka_streams_store_monitoring">State Store Metrics</a></h5>
|
||||
All the following metrics have a recording level of ``debug``:
|
||||
|
||||
<table class="data-table">
|
||||
<tbody>
|
||||
<tr>
|
||||
<th>Metric/Attribute name</th>
|
||||
<th>Description</th>
|
||||
<th>Mbean name</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>[put | put-if-absent | get | delete | put-all | all | range | flush | restore]-latency-[avg | max]</td>
|
||||
<td>The average execution time in ns, for the respective operation. </td>
|
||||
<td>kafka.streams:type=stream-[store-type]-metrics</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>[put | put-if-absent | get | delete | put-all | all | range | flush | restore]-rate</td>
|
||||
<td>The average rate of respective operations per second for this store.</td>
|
||||
<td>kafka.streams:type=stream-[store-type]-metrics</td>
|
||||
</tr>
|
||||
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
<h4><a id="others_monitoring" href="#others_monitoring">Others</a></h4>
|
||||
|
||||
We recommend monitoring GC time and other stats and various server stats such as CPU utilization, I/O service time, etc.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue