b3313ad21e
Using async commands via Khepri's `async` command option - which corresponds to `ra:pipeline_command/4` - allows Ra to write multiple commands in a batch. This can significantly boost the migration speed since Ra can handle more commands per call to fsync and fsync seems to be the bottleneck during migration. We refactor the callbacks of the converter modules so that they take the overall converter module's (`rabbit_db_m2k_converter`) state record. That state record keeps track of the correlation IDs which are ultimately passed to `ra:pipeline_command/4` and each converter module calls `rabbit_db_m2k_converter:with_correlation_id/2` on the state to get a new correlation ID and add an async request. Each async change is briefly attached to the state record as a function so that requests can be retried in the case that the Ra leader changes. `with_correlation_id/2` also acts as a backpressure mechanism so we avoid flooding the Ra process with messages. I've set the maximum for in-flight requests to 64 to start with. The worker pool for importing definitions has 32 members, so migration tends to be faster than initial definition import. This could be tuned based on memory/CPU usage however. |
||
|---|---|---|
| .. | ||
| include | ||
| src | ||
| test | ||
| .gitignore | ||
| BUILD.bazel | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| LICENSE-MPL-RabbitMQ | ||
| Makefile | ||
| README.md | ||
| app.bzl | ||
README.md
RabbitMQ JMS Topic Exchange Plugin
Overview
This plugin adds server-side support for RabbitMQ JMS client. This plugin provides support for JMS topic routing and selection based on JMS SQL selection rules.
This implementation is based upon the Java Messaging Service Specification Version 1.1.
Project Maturity
RabbitMQ JMS-related projects are several years old and can be considered reasonably mature. They have been first open sourced in June 2016. Some related projects (e.g. a compliance test suite) and documentation are yet to be open sourced.
Supported RabbitMQ Versions
This plugin ships with RabbitMQ.
Installation
Like all other plugins, this plugin must be enabled before it can be used. Enable it with
[sudo] rabbitmq-plugins enable rabbitmq_jms_topic_exchange
Design
The plugin this generates is a user-written exchange type for RabbitMQ
client use. The exchange type name is "x-jms-topic" but this is not
a topic exchange. Instead it works together with a standard topic
exchange to provide the JMS topic selection function.
When JMS Selectors are used on a Topic Destination consumer, the
destination (queue) is bound to an exchange of type x-jms-topic, with
arguments that indicate what the selection criteria are. The
x-jms-topic exchange is, in turn, bound to the standard Topic Exchange
used by JMS messaging (this uses the RabbitMQ exchange-to-exchange
binding extension to the AMQP 0-9-1 protocol).
In this way, normal topic routing can occur, with the overhead of selection only applying when selection is used, and after the routing and filtering implied by the topic name.
Copyright and License
(c) 2007-2023 Broadcom. All Rights Reserved. The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries.
See LICENSE for license information.