a934c1da32
The idea is to use a binary instead of lists to to hold messages in a quorum queue's incoming queue. Enqueue = appending to a binary Dequeue = matching the front of a binary Per message memory usage prior to this commit: ``` erts_debug:size([[1|2]]). 4 erts_debug:size([[1|2], [3|4]]). 8 ``` 4 words * 8 bytes (on a 64 bit machine) = 32 bytes Per message memory usage after this commit: 7 bytes to encode the RaIdx + 4 bytes to encode the message size = 11 bytes The message bytes encoding can be further optimised and be below 3 bytes most of the time. Hence, let's say 10 bytes per message. Let's assume we have a 3 node cluster with 500 quorum queues and 200,000 messages each per queue. That's 100 million messages across all queues. Prior to this commit this would require a total of 9.6 GB of memory: 100,000,000 msgs * 32 bytes * 3 nodes = 9.6 GB After this commit this requires only a total of only 3 GB of memory: 100,000,000 msgs * 10 bytes * 3 nodes = 3 GB of memory If there is a message TTL policy set for these queues the savings will be even more significant because prior to this commit the per message memory overhead is 6 words * 8 bytes = 48 bytes: ``` erts_debug:size([[1|[2|5000]]]). 6 erts_debug:size([[1|[2|5000]], [3|[4|5000]]]). 12 ``` Prior to this commit this would require a total of 14.4 GB of memory: 100,000,000 msgs * 48 bytes * 3 nodes = 14.4 GB If we assume addional 6 bytes to encode the expiration timestamp in milliseconds, with a binary encoding, this would require a total of only 4.8 GB of memory: 100,000,000 msgs * 16 bytes * 3 nodes = 4.8 GB The problem with the binary approach is that appending to a binary at a rate of 100,000 msgs/s is catastrophically slow: ``` java -jar target/perf-test.jar -qq -u qq1 -x 1 -y 0 -C 1000000 ``` * sends at 90766 msg/s pior to this commit * sends at 3435 msg/s after this commit After this commit, >80% of CPU time is spent in function `__memmove_evex_unaligned_erms` copying binary data around. Appending frequently to the binary becomes even slower as the binary gets longer. The only practical solution would be a hybrid approach: * lists are used by default * lists will always be used for enqueueing messages (i.e. prepending to the list) * a binary will be created whenever many messages, e.g. 100,000 new messages got accumulated. Such a binary creation will be relatively fast, as explained in https://www.erlang.org/doc/system/binaryhandling.html : "Appending data to a binary as in the example is efficient because it is specially optimized by the runtime system to avoid copying the Acc binary every time." * Once the binary got created, pattern matching at the front and therefore dequeueing messages will be fast. * The binaries themselves will be stored in lists (a list with 10 binaries means ~1 million messages in total in our example). It's questionable though whether such an approach is worth the introduced complexities given that queues are meant to be kept short anyways. |
||
|---|---|---|
| .github | ||
| deps | ||
| doc | ||
| mk | ||
| packaging | ||
| release-notes | ||
| scripts | ||
| selenium | ||
| .dockerignore | ||
| .elp.toml | ||
| .git-blame-ignore-revs | ||
| .gitignore | ||
| .mailmap | ||
| CODE_OF_CONDUCT.md | ||
| COMMUNITY_SUPPORT.md | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| LICENSE-APACHE2 | ||
| LICENSE-MPL-RabbitMQ | ||
| Makefile | ||
| PKG_LINUX.md | ||
| PKG_WINDOWS.md | ||
| README.md | ||
| SERVER_RELEASES.md | ||
| erlang.mk | ||
| erlang_ls.config | ||
| plugins.mk | ||
| rabbitmq-components.mk | ||
| rebar.config | ||
README.md
RabbitMQ Server
RabbitMQ is a feature rich, multi-protocol messaging and streaming broker. It supports:
- AMQP 1.0
- AMQP 0-9-1
- RabbitMQ Stream Protocol
- MQTT 3.1, 3.1.1, and 5.0
- STOMP 1.0 through 1.2
- MQTT over WebSocket
- STOMP over WebSocket
- AMQP 1.0 over WebSocket (supported in VMware Tanzu RabbitMQ)
Installation
- Currently supported released series
- Installation guides for various platforms
- Kubernetes Cluster Operator
- Changelog
- Releases on GitHub
- Community Support Eligibility Policy
- Supported Erlang versions
Tutorials and Documentation
Some key doc guides include
- CLI tools guide
- Clustering and Cluster Formation
- Configuration guide
- Client libraries and tools
- Monitoring and Prometheus/Grafana
- Upgrading
- Kubernetes Cluster Operator
- Production checklist
- Quorum queues: a replicated, data safety- and consistency-oriented queue type
- Streams: a persistent and replicated append-only log with non-destructive consumer semantics
- Runtime Parameters and Policies
- Runnable tutorials
RabbitMQ documentation is also developed on GitHub.
Commercial Features and Support
- Commercial editions of RabbitMQ
- Commercial edition for Kubernetes
- Commercial support from Broadcom for open source RabbitMQ
Getting Help from the Community
Please read the Community Support Eligibility Policy document first.
The recommended community forums are
- GitHub Discussions
- Community Discord server
#rabbitmqon Libera Chat
Contributing
See CONTRIBUTING.md and our development process overview.
Questions about contributing, internals and so on are very welcome in GitHub Discussions
or community Discord server in the core-and-plugin-dev channel.
Licensing
RabbitMQ server is licensed under the MPL 2.0.
Community Support Eligibility Policy document explains the open source RabbitMQ support policy adopted by the RabbitMQ Core Team.
Building From Source and Packaging
Copyright
(c) 2007-2025 Broadcom. All Rights Reserved. The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries.