History

Michael Davis b3313ad21e Migrate records from mnesia to Khepri with async commands Using async commands via Khepri's `async` command option - which corresponds to `ra:pipeline_command/4` - allows Ra to write multiple commands in a batch. This can significantly boost the migration speed since Ra can handle more commands per call to fsync and fsync seems to be the bottleneck during migration. We refactor the callbacks of the converter modules so that they take the overall converter module's (`rabbit_db_m2k_converter`) state record. That state record keeps track of the correlation IDs which are ultimately passed to `ra:pipeline_command/4` and each converter module calls `rabbit_db_m2k_converter:with_correlation_id/2` on the state to get a new correlation ID and add an async request. Each async change is briefly attached to the state record as a function so that requests can be retried in the case that the Ra leader changes. `with_correlation_id/2` also acts as a backpressure mechanism so we avoid flooding the Ra process with messages. I've set the maximum for in-flight requests to 64 to start with. The worker pool for importing definitions has 32 members, so migration tends to be faster than initial definition import. This could be tuned based on memory/CPU usage however.		2023-12-12 12:01:59 -05:00
..
etc	adds multinode tests	2014-09-23 19:16:25 +02:00
include	(c) year bump: 2022 => 2023	2023-01-01 23:17:36 -05:00
src	Migrate records from mnesia to Khepri with async commands	2023-12-12 12:01:59 -05:00
test	Update (c) according to [1]	2023-11-21 23:18:22 -05:00
.gitignore	Git: Ignore copied CLI	2019-12-12 15:04:35 +01:00
BUILD.bazel	Allow to use Khepri database to store metadata instead of Mnesia	2023-09-29 16:00:11 +02:00
CODE_OF_CONDUCT.md	Replace files with symlinks	2022-04-15 06:04:29 -07:00
CONTRIBUTING.md	Replace files with symlinks	2022-04-15 06:04:29 -07:00
LICENSE	Replace @rabbitmq.com addresses with rabbitmq-core@groups.vmware.com	2023-06-20 15:40:13 +04:00
LICENSE-MPL-RabbitMQ	Revert drop of Exhibit B on MPL 2.0	2020-07-20 17:03:05 +01:00
Makefile	Add khepri to rabbitmq_recent_history_exchange deps	2023-10-16 16:22:11 +02:00
README.md	Recent history exchange: README.md updates	2023-04-15 14:24:02 +04:00
app.bzl	Allow to use Khepri database to store metadata instead of Mnesia	2023-09-29 16:00:11 +02:00

README.md

RabbitMQ Recent History Cache

Keeps track of the last 20 messages that passed through the exchange. Every time a queue is bound to the exchange it delivers that last 20 messages to them. This is useful for implementing a very simple Chat History where clients that join the conversation can get the latest messages.

Exchange yype: x-recent-history.

Installation

This plugin ships with RabbitMQ.

Like all other plugins, it must be enabled before it can be used:

[sudo] rabbitmq-plugins enable rabbitmq_recent_history_exchange

Usage

Creating an exchange

To create a recent history exchange, just declare an exchange providing the type "x-recent-history".

channel.exchangeDeclare("logs", "x-recent-history");

Providing a custom history length

Typically this exchange will store the latest 20 messages sent over the exchange. If you want to set a different cache length, then you can pass a "x-recent-history-length" argument to exchange.declare. The argument must be an integer greater or equal to zero.

For example in Java:

Map<String, Object> args = new HashMap<String, Object>();
args.put("x-recent-history-length", 60);
channel.exchangeDeclare("rh", "x-recent-history", false, false, args);

Preventing some messages from being stored

In case you would like to not store certain messages, just add the header "x-recent-history-no-store" with the value true to the message.

Disabling the Plugin

A future version of RabbitMQ will allow users to disable plugins. When you disable this plugin, it will delete all the cached messages.

License

See LICENSE.