What's Changed
* Allow force shrink to non-voter member
* ra_server_proc: Handle aux_command in all Raft states
* Increase shutdown timeout for segment writer.
* Avoid modification checks when reading sparse entries inside the Ra process
Ra improvements:
* Don't allow a non-voter to start elections
* Register with ra directory before initialising ra server.
* Trigger tick_timeout immediately after entering leader state.
* Set a configurable segment max size
This commit also includes a change to turn the quorum queue
become leader callback to become a noop and instead rely on
the more promptly tick_handler to handle the meta data store
update after a leader election.
This more prompt tick update means there should be a much shorter
gap between the queue metrics being deleted from the old leader
node to them being available again on the new node resulting
in smoother message count metrics.
Fix test that relied on waiting on too simplistic a property
before asserting.
* Add num_segments to Ra counters
* ra_server_proc: Fix handling of local query replies
* Remove Bazel-related files by @mkuratczyk in #520
* Replication bug fixes that could cause replication to stall
* Use infinity timeout for ra_log_ets:mem_table_please
Cowboy 2.13 contains the Websocket optimisations as well
as the ability to set the Websocket max_frame_size option
dynamically, plus plenty of other improvements.
Cowlib was added as a test dep to rabbitmq_mqtt to make
sure emqtt doesn't pull the wrong Cowlib version for Cowboy.
Contains bug fix which would crash at-most-once dead lettering during
node restarts.
Less excessive debug logging around ra log.
Fix issue that could make leader transfers take 5s+ to complete.
This version of Ra contains a substantially refactored Ra log
implementation that provides higher throughput and lower
memory use in serveral scenarios.
New features:
* `log_ext` new effect type that instead of immedately reading
entries from the log it will instead provide a read plan for any
entries only located in segments.
* Machine version upgrades can now be be delayed until all
members are confirmed to support the new version.
This will avoid potential consumption pauses during upgrades.
This version contains bug fixes and a change to use async_dist
when a quorum queue sends a message to a remote node (e.g. a consumer
delivery). Using async_dist will reduce chances of messages not
reaching consumers in a timely manner when the system is loaded
and occasionally fills the distribution buffer.
This release contains a bug fix to an issue that very occasionally
could cause consumers on replica nodes not to be notified about
newly committed offsets in a timely manner.
This osiris release contains a bug fix that would cause an osiris
member to crash during recovery if certain unexpected files
were present in the log directory. (.e.g ".nfsXXXXXXXXXXXX") type
files used by the NFS file system when in use files are deleted.
This is a proof of concept that mostly works but is missing
some tests, such as rabbitmq_mqtt or rabbitmq_cli. It also
doesn't apply to mixed version testing yet.
Otherwise some plugins can't build if we try to run tests
directly after checkout. This is because the plugins
depend on osiris as well as rabbit, but there is no
dep_osiris defined in the plugin itself.
Because of the monorepo most components do not need to be
listed. Only the community plugins and third party dependencies.
Community plugins can now be fetched and acted on from the top
level Makefile by adding COMMUNITY_PLUGINS=1 to the command line
or the environment. This will fetch and build community plugins:
make COMMUNITY_PLUGINS=1
Once fetched they can be targeted directly as usual:
make -C deps/rabbitmq_metronome
This cleanup has a net positive effect on build performance,
especially the performance of the top-level Makefile:
make nope 0,04s user 0,02s system 106% cpu 0,061 total
make nope 0,02s user 0,01s system 104% cpu 0,033 total
But also a minor improvement for application Makefiles:
make -C deps/rabbit nope 0,02s user 0,00s system 98% cpu 0,022 total
make -C deps/rabbit nope 0,01s user 0,00s system 98% cpu 0,020 total
And that improvement adds up when going through dependencies:
make -C deps/rabbitmq_management 0,59s user 0,23s system 100% cpu 0,808 total
make -C deps/rabbitmq_management 0,60s user 0,19s system 101% cpu 0,780 total
Ideally we wouldn't need it, but until applications are
in apps/ it will be necessary for a thing or two.
Note that rabbitmq_server_release is required to be
there for prepare-dist:: to work when building the
generic unix package.
This release contains a few fixes and improvements:
* Add ra:key_metrics/2
* ra_server: Add a new last_applied state query
* Stop checkpoint validation when encountering a valid checkpoint
* Kill snapshot process before deleting everything
This restores the previous pre-init behaviour where an invalid server
will not stop the Ra system from starting. Instead it will log the
errors and continue.
This ensures compatibility with upgraded older systems and systems
where there are historical discrepancies between what is in the
ra_directory and actually on disk.
This contains a fix in the ra_directory module to ensure
names can be deleted even when a Ra server has never been started
during the current node lifetime.
Also contains a small tweak to ensure the ra_directory:unregister_name
is called before deleting a Ra data directory which is less likely
to cause a corrupt state that will stop a Ra system from starting.
This release contains improvements to the checkpointing feature
needed for quorum queues v4 and the following fixes:
* Add read to file:open/2 options in ra_lib:sync_file/1
* Emit the new local_query tuple only if query options are set
* bug fixes for checkpoints
The beam cache allows switching between app and test
builds without having to rebuild everything. Since
the files keep their mtime and other attributes,
rebuilding continues from where it was left off
before, and only the relevant files get rebuilt
if anything changed.
It has largely been superseded by `perf`. It is no longer
generally useful. It can always be added to BUILD_DEPS for
the rare cases it is needed, or installed locally and
pointed to by setting its path to ERL_LIBS.
Michael Davis
Jean-Sébastien Pédron
Karl Nilsson
Michael Klishin
Loïc Hoguin
zhongwencool
Simon Unge
Michael Klishin
Michael Klishin