root
/
rabbitmq-server
mirror of https://github.com/rabbitmq/rabbitmq-server.git
1
0
Fork 0
Code Issues Actions 4 Packages Projects Releases Wiki Activity
rabbitmq-server/release-notes/3.12.0.md

444 lines
19 KiB
Markdown
Raw Permalink Blame History

## RabbitMQ 3.12.0
RabbitMQ `3.12.0` is a new feature release.
This release [goes out of community support](https://www.rabbitmq.com/release-information) on June 1st, 2024.
## Highlights
This release includes several new features, optimizations, and graduates (makes mandatory) a number of feature flags.
The user-facing areas that have seen the biggest improvements in this release are
* [Optimizations](https://blog.rabbitmq.com/posts/2023/05/rabbitmq-3.12-performance-improvements/) for both quorum and classic queues: improved throughput, lower throughput variability, lower latency, lower memory footprint
* More mature and efficient implementation of (non-mirrored) classic queues v2 (CQv2)
* Classic queue lazy and non-lazy modes no longer apply: classic queues v2 always behave very similarly
to the lazy mode in earlier release series: moving data to disk aggressively and only keeping a subset of data in memory
* Significantly [reduced MQTT and Web MQTT memory footprint per connection](https://blog.rabbitmq.com/posts/2023/03/native-mqtt/)
* OAuth 2, OIDC, IDP support
* Even more configurability of the OAuth 2 plugin
This release also features many internal API improvements in preparation to 4.0
with [Khepri](https://www.youtube.com/watch?v=huT-zmXvfuM).
See Compatibility Notes below to learn about breaking or potentially breaking changes in this release.
## Release Artifacts
RabbitMQ releases are distributed via [GitHub](https://github.com/rabbitmq/rabbitmq-server/releases).
[Debian](https://rabbitmq.com/install-debian.html) and [RPM packages](https://rabbitmq.com/install-rpm.html) are available via Cloudsmith mirrors, as well as [PackageCloud](https://packagecloud.io/rabbitmq).
[Community Docker image](https://hub.docker.com/_/rabbitmq/), [Chocolatey package](https://community.chocolatey.org/packages/rabbitmq), and the [Homebrew formula](https://rabbitmq.com/install-homebrew.html)
are other installation options. They are updated with a delay (usually a few days).
## Erlang/OTP Compatibility Notes
This release [requires Erlang 25.0](https://www.rabbitmq.com/which-erlang.html) or later.
This introduces feature parity for x86- and ARM64-based CPUs: Erlang 25 offers the JIT and
[modern Flame Graph profiling](https://blog.rabbitmq.com/posts/2022/05/flame-graphs/) tooling
for both of those major CPU architectures.
[Provisioning Latest Erlang Releases](https://www.rabbitmq.com/which-erlang.html#erlang-repositories) explains
what package repositories and tools can be used to provision latest patch versions of Erlang 25.x.
### Erlang 26
This is the first release that offers Erlang 26 compatibility. Note that Erlang 26 is very new and
is nowhere near as battle tested as Erlang 25.
## Upgrading to 3.12
### Documentation guides on upgrades
See the [Upgrading guide](https://www.rabbitmq.com/upgrade.html) for documentation on upgrades and [RabbitMQ change log](https://www.rabbitmq.com/changelog.html)
for release notes of other releases.
### Required Feature Flags
RabbitMQ 3.12.0 will require [all feature flags from the 3.11.x release series to be enabled](https://www.rabbitmq.com/feature-flags.html#list-of-feature-flags) before upgrading,
similarly to how [3.11.0 required all feature flags introduced before 3.9.0](https://blog.rabbitmq.com/posts/2022/07/required-feature-flags-in-rabbitmq-3.11/).
If the feature flags are not enabled, RabbitMQ 3.12 and later nodes will refuse to start.
### Upgrading to Erlang 26 and RabbitMQ 3.12 at the Same Time
When upgrading from 3.11.x on Erlang 25 to 3.12.x on Erlang 26,
the cluster must be on **at least version 3.11.17** for a [safe upgrade](https://github.com/rabbitmq/ra/pull/369) of quorum queues.
Alternatively, it is possible to upgrade all nodes to 3.12.0 on Erlang 25 from an earlier
3.11.x version, then upgrade the 3.12 cluster to Erlang 26.
### Mixed version cluster compatibility
RabbitMQ 3.12.0 nodes can run alongside `3.11.x` nodes starting with `3.11.17`.
`3.12.x`-specific features can only be made available when all nodes in the cluster
upgrade to 3.12.0 or any other patch release in the new series.
While operating in mixed version mode, some aspects of the system may not behave as expected. The list of known behavior changes is covered below.
Once all nodes are upgraded to 3.12.0, these irregularities will go away.
Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended
periods of time (no more than a few hours).
## Compatibility Notes
### More Feature Flags Gratuate to Core Features ("Always Enabled")
RabbitMQ 3.12.0 will require **all** feature flags from the 3.11.x release series to be enabled before upgrading.
If the feature flags are not enabled, RabbitMQ 3.12 and later nodes will refuse to start.
### MQTT Client Metrics
Some metrics displayed by the management UI are no longer available for MQTT clients.
They are computed for AMQP 0-9-1 channels, which MQTT plugin no longer uses. They may be
reintroduced in future releases when more protocol implementations (e.g. AMQP 1.0) are
refactored the same way the MQTT plugin was for this release.
Connection-level metrics such as ingress and egress data flows are still available,
and so are the metrics of queues used by MQTT subscribers.
### Minimum Supported Erlang Version
Starting with this release, RabbitMQ requires Erlang 25.0 or later versions. Nodes **will fail to start**
on older Erlang releases.
Erlang 25 as our new baseline means much improved performance on ARM64 architectures, [profiling with flame graphs](https://blog.rabbitmq.com/posts/2022/05/flame-graphs/)
across all architectures, and the most recent TLS 1.3 implementation available to all RabbitMQ 3.11 users.
### Client Library Compatibility
Client libraries that were compatible with RabbitMQ `3.11.x` will be compatible with `3.12.0`.
### Getting Help
Any questions about this release, upgrades or RabbitMQ in general are welcome on the [RabbitMQ mailing list](https://groups.google.com/forum/#!forum/rabbitmq-users).
## Changes Worth Mentioning
Release notes are kept under [rabbitmq-server/release-notes](https://github.com/rabbitmq/rabbitmq-server/tree/v3.11.x/release-notes).
### Core Server
#### Enhancements
* Quorum queues now sustain higher throughput with large backlogs. They also offer higher throughput
on average, and in particular when [Single Active Consumer](https://rabbitmq.com/consumers.html#single-active-consumer) is used.
GitHub issue: [#7553](https://github.com/rabbitmq/rabbitmq-server/pull/7553#issuecomment-1463660454)
* Reduced memory footprint, improved memory use predictability and throughput of classic queues.
This particularly benefits classic queues with longer backlogs.
GitHub issues: [#4522](https://github.com/rabbitmq/rabbitmq-server/pull/4522), [#7516](https://github.com/rabbitmq/rabbitmq-server/pull/7516)
* Reduced peak memory footprint of quorum queues.
GitHub issue: [#7175](https://github.com/rabbitmq/rabbitmq-server/pull/7175)
* Improved stream leader distribution efficiency. Mostly relevant to environments with lots of
streams.
GitHub issue: [#6440](https://github.com/rabbitmq/rabbitmq-server/pull/6440)
* All [feature flags](https://www.rabbitmq.com/feature-flags.html) introduced before 3.11.0 are now required to be enabled. Make sure
all feature flags are enabled before upgrading to 3.12.0.
GitHub issue: [#7219](https://github.com/rabbitmq/rabbitmq-server/pull/7219)
* Plugin and core API: all schema data store operations are now available via the `rabbit_db_*` modules.
Plugin maintainers should switch to them for an easier transition to a future 4.0 release.
GitHub issues: [#6430](https://github.com/rabbitmq/rabbitmq-server/pull/6430), [#6821](https://github.com/rabbitmq/rabbitmq-server/pull/6821)
* Plugin and core API: node data directory now should be accessed using `rabbit:data_dir/0`.
GitHub issue: [#6462](https://github.com/rabbitmq/rabbitmq-server/pull/6462)
* Initial (incomplete) Erlang 26 compatibility
GitHub issues: [#7443](https://github.com/rabbitmq/rabbitmq-server/pull/7443), [#7900](https://github.com/rabbitmq/rabbitmq-server/pull/7900),
[#7921](https://github.com/rabbitmq/rabbitmq-server/pull/7921), [#8057](https://github.com/rabbitmq/rabbitmq-server/issues/8057)
* There is now a way to pre-configure users and their permissions for newly created virtual hosts:
``` ini
default_users.qa_user.vhost_pattern = qa.*
default_users.qa_user.tags = policymaker,monitoring
default_users.qa_user.password = fd237824441a78cd922410af4b83f0888186a8d7
default_users.qa_user.read = .*
default_users.qa_user.write = .*
default_users.qa_user.configure = .*
```
This is primarily useful in environments where RabbitMQ is provided as a service but
customers (clients) have control over virtual hosts.
Contributed by @illotum (AWS).
GitHub issue: [#7208](https://github.com/rabbitmq/rabbitmq-server/issues/7208).
* Policies can now be defined to only apply to specific queue types.
For example, it is now possible to have two policies matching all queue names ('.*')
but one applicable only to one queue type, and the other only to different queue type,
without conflicts.
For example, this policy will only apply to quorum queues and not streams or classic queues:
``` bash
rabbitmqctl set_policy at-least-once-dead-lettering ".*" '{"dead-letter-strategy": "at-least-once"}' --apply-to quorum_queues
```
* Nodes with a lot of classic queues now shut down up to 10 times faster.
GitHub issue: [#7745](https://github.com/rabbitmq/rabbitmq-server/pull/7745)
* Reduced metric collection-induced memory use spikes every two minutes in environments with
a lot of mostly idle queues.
GitHub issues: [#7825](https://github.com/rabbitmq/rabbitmq-server/pull/7825), [#7896](https://github.com/rabbitmq/rabbitmq-server/pull/7896)
* [Delivery acknowledgement timeout](https://rabbitmq.com/consumers.html#acknowledgement-timeout) now can be configured per queue (using a policy)
or per consumer (using optional consumer arguments).
GitHub issue: [#5437](https://github.com/rabbitmq/rabbitmq-server/issues/5437)
* It is now possible to limit the maximum number of virtual hosts that can be created
in the cluster.
Contributed by @SimonUnge (AWS).
GitHub issue: [#7777](https://github.com/rabbitmq/rabbitmq-server/issues/7777)
* Nodes with a lot of (non-mirrored) classic queues with v2 storage enabled
by default (using `classic_queue.default_version = 2`) now boot faster.
GitHub issue: [#7676](https://github.com/rabbitmq/rabbitmq-server/pull/7676)
* It is now possible to limit how many shovels or federation links can run on a node
using `rabbitmq.conf`:
``` ini
runtime_parameters.limits.shovel = 10
runtime_parameters.limits.federation = 10
```
Contributed by @illotum (AWS).
GitHub issue: [#7917](https://github.com/rabbitmq/rabbitmq-server/pull/7917)
* Quorum queues: The `x-delivery-count` header will no longer be added to messages the first time they are delivered (`x-delivery-count=0`)
as it has a potential performance impact on consumer deliveries.
GitHub issue: [#7732](https://github.com/rabbitmq/rabbitmq-server/pull/7732)
* Quorum queues will now log if they could not apply policy changes, for example,
because there was no quorum of replicas online, or the queue was going through
a leader election.
GitHub issue: [#7853](https://github.com/rabbitmq/rabbitmq-server/pull/7853)
#### Bug Fixes
This release includes all bug fixes shipped in the `3.11.x` series.
* It is now possible to use at-most-once dead lettering with streams (dead letter
messages to a stream).
GitHub issue: [#7846](https://github.com/rabbitmq/rabbitmq-server/pull/7846)
* It is now possible to use streams in combination with the event exchange.
GitHub issue: [#7846](https://github.com/rabbitmq/rabbitmq-server/pull/7846)
* [Superstream](https://blog.rabbitmq.com/posts/2022/07/rabbitmq-3-11-feature-preview-super-streams/) could fail
to elect a single active consumer (SAC) in certain consumer churn conditions.
GitHub issue: [#7743](https://github.com/rabbitmq/rabbitmq-server/issues/7743)
* Quorum queues now have a mechanism of shrinking down to just one replica.
This is meant to be used **exclusively** for disaster recovery when a majority
of nodes hosting replicas of a queue were permanently lost.
GitHub issue: [#8322](https://github.com/rabbitmq/rabbitmq-server/pull/8322)
* Classic mirrored queues that had a [length limit](https://rabbitmq.com/maxlength.html) defined on them
handled the overflow of messages differently from the current elected leader,
eventually causing exceptions and replica restarts.
GitHub issue: [#7579](https://github.com/rabbitmq/rabbitmq-server/pull/7579)
* When a queue was deleted and instantly re-declared with a different type,
polling consumers (those that use `basic.get`) ran into an exception.
GitHub issue: [#4976](https://github.com/rabbitmq/rabbitmq-server/issues/4976)
### CLI Tools
#### Enhancements
* `rabbitmq-queues rebalance` now supports streams in addition to classic and quorum queues.
GitHub issue: [#6440](https://github.com/rabbitmq/rabbitmq-server/pull/6440)
* `rabbitmq-streams restart_stream` is a new command that can be used to kick off
a new leader election for a stream. This is another option that can be used
to rebalance stream leaders across cluster nodes.
GitHub issue: [#6440](https://github.com/rabbitmq/rabbitmq-server/pull/6440)
* Classic queue metrics are now available via `rabbitmq-diagnostics observer`.
GitHub issue: [#3314](https://github.com/rabbitmq/rabbitmq-server/pull/3314)
* `rabbitmqctl update_vhost_metadata` is a new command that can be used to update
the description, default queue type, or tags of a virtual host:
``` shell
rabbitmqctl update_vhost_metadata vh1 --tags qa,quorum,team3,project2
rabbitmqctl update_vhost_metadata vh1 --description "QA env 1 for issue 37483"
rabbitmqctl update_vhost_metadata vh1 --description "QQs all the way" --default-queue-type "quorum"
rabbitmqctl update_vhost_metadata vh1 --description "streaming my brain out" --default-queue-type "stream"
```
GitHub issue: [#7914](https://github.com/rabbitmq/rabbitmq-server/pull/7914)
* `rabbitmqctl set_permissions_globally` is a new command that sets up user permissions in all existing virtual hosts.
GitHub issue: [#1000](https://github.com/rabbitmq/rabbitmq-server/issues/1000)
* `rabbitmq-diagnostics cluster_status` now lists how many CPU cores are available to individual nodes, plus a total.
GitHub issue: [#7135](https://github.com/rabbitmq/rabbitmq-server/pull/7135)
#### Bug Fixes
* `rabbitmq-stream stream_status` failed with an exception if one of the replicas was offline.
Contributed by @gomoripeti (CloudAMQP)
GitHub issue: [#8270](https://github.com/rabbitmq/rabbitmq-server/pull/8270)
### MQTT Plugin
#### Enhancements
* Significant (up to more than 90% with some workloads) [reduction in memory footprint of MQTT connections](https://blog.rabbitmq.com/posts/2023/03/native-mqtt/),
and a double digit % reduction in memory footprint of MQTT-over-WebSockets (Web MQTT) connections.
GitHub issues: [#5895](https://github.com/rabbitmq/rabbitmq-server/pull/5895), [#7091](https://github.com/rabbitmq/rabbitmq-server/pull/7091), [#7234](https://github.com/rabbitmq/rabbitmq-server/pull/7234)
* The `x-mqtt-dup` header will no longer be present for consumer deliveries as it wasn't used correctly.
### Federation Plugin
#### Bug Fixes
* Fixed a quorum queue-specific issue that could prevent queue federation links from making any progress.
GitHub issue: [#8328](https://github.com/rabbitmq/rabbitmq-server/pull/8328)
### Management Plugin
#### Enhancements
* Support for Identity Provider-initiated Login (IDP).
GitHub issues: [#6015](https://github.com/rabbitmq/rabbitmq-server/pull/6015), [#6201](https://github.com/rabbitmq/rabbitmq-server/pull/6201),
[#6247](https://github.com/rabbitmq/rabbitmq-server/pull/6247)
* **Potentially breaking** change: deprecated UAA-specific configuration keys were removed:
`management.enable_uaa`, `management.uaa_*`.
GitHub issue: [#7887](https://github.com/rabbitmq/rabbitmq-server/pull/7887)
* Classic queues now list their storage implementation version on the details page.
Contributed by @gomoripeti (CloudAMQP)
GitHub issue: [#8244](https://github.com/rabbitmq/rabbitmq-server/pull/8244)
#### Bug Fixes
* Default queue type key used when add a virtual host via the HTTP API has changed from
`defaultqueuetype` to `default_queue_type`. `default_queue_type` will be supported
for a period of time for backwards compatibility.
GitHub issue: [#7738](https://github.com/rabbitmq/rabbitmq-server/pull/7738)
* It was impossible to return to a tab that had a filter expression
that was not a valid regular expressions.
Now such expressions will be used as regular text filters.
GitHub issue: [#8008](https://github.com/rabbitmq/rabbitmq-server/issues/8008)
### OAuth 2 AuthN/AuthZ Backend Plugin
#### Enhancement
* It is now possible to configure (or strip) the scope prefix used by this plugin when translating
token scopes to RabbitMQ permissions.
To do so, override `auth_oauth2.scope_prefix` in `rabbitmq.conf`:
``` ini
# "rabbitmq." is the default prefix
auth_oauth2.scope_prefix = rabbitmq.
```
GitHub issue: [#8001](https://github.com/rabbitmq/rabbitmq-server/pull/8001)
* Several variables (`{username}`, `{vhost}` and JWT claims that are single string values)
now can be used (expanded) in topic operation authorization.
GitHub issue: [#7178](https://github.com/rabbitmq/rabbitmq-server/issues/7178)
### HTTPS AuthN/AuthZ Backend Plugin
#### Bug Fixes
* The authorization backend could run into an exception when used in
combination with other backends.
GitHub issue: [#7864](https://github.com/rabbitmq/rabbitmq-server/issues/7864)
### Consul Peer Discovery Plugin
#### Bug Fixes
* Consul peer discovery now correctly determines if the node uses long or short names.
Contributed by @SimonUnge (AWS).
GitHub issue: [#4230](https://github.com/rabbitmq/rabbitmq-server/issues/4230)
### Dependency Changes
* [`ra`](https://github.com/rabbitmq/ra) was upgraded [to `2.6.1`](https://github.com/rabbitmq/ra/releases)
* [`osiris`](https://github.com/rabbitmq/osiris) was upgraded to `1.5.1`
* [`cowlib`](https://github.com/ninenines/cowlib) was upgraded to `2.12.1`
## Source Code Archives
To obtain source code of the entire distribution, please download the archive named `rabbitmq-server-3.12.0.tar.xz`
instead of the source tarball produced by GitHub.
Reference in New Issue View Git Blame Copy Permalink