root
/
rabbitmq-server
mirror of https://github.com/rabbitmq/rabbitmq-server.git
1
0
Fork 0
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Initial 3.11 release notes

This commit is contained in:
Michael Klishin 2022-09-02 03:12:55 +04:00
parent b1897e7e58
commit 994a71321c
1 changed files with 87 additions and 253 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

340
release-notes/3.11.0.md
View File

@ -44,17 +44,38 @@ periods of time (no more than a few hours).
## Compatibility Notes
###
RabbitMQ 3.11.0 will [require all feature flags from the 3.8.x release series to be enabled](https://blog.rabbitmq.com/posts/2022/07/required-feature-flags-in-rabbitmq-3.11/) before upgrading.
If the feature flags are not enabled, RabbitMQ 3.11 and later nodes will refuse to start.
### Minimum Supported Erlang Version
Starting with this release, RabbitMQ requires Erlang 25.0 or later versions. Nodes **will fail to start**
on older Erlang releases.
Erlang 25 as our new baseline means much improved performance on ARM64 architectures, [profiling with flame graphs](https://blog.rabbitmq.com/posts/2022/05/flame-graphs/)
across all architectures, and the most recent TLS 1.3 implementation available to all RabbitMQ 3.11 users.
### Open File Handle Limit is Capped by Default
Nodes now default to 65536 concurrent client connections instead of using the effective kernel open file handle limit.
This change was introduced because of a change in several Linux distributions: they now use a default open file handle limit so high,
they cause a significant (say, 1.5 GiB) memory preallocated the Erlang runtime.
For example, [RHEL 9 and CentOS Stream 9](https://access.redhat.com/solutions/1479623) are examples of such distributions.
The new default [comes from modern systemd](https://github.com/systemd/systemd/commit/a8b627aaed409a15260c25988970c795bf963812).
See [rabbitmq/rabbitmq-server#5684](https://github.com/rabbitmq/rabbitmq-server/pull/5684) and [docker-library/rabbitmq#545](https://github.com/docker-library/rabbitmq/issues/545#issuecomment-1224977154)
for details.
### Client Library Compatibility
Client libraries that were compatible with RabbitMQ `3.10.x` will be compatible with `3.11.0`.
### Upgrading to Erlang 24.x
RabbitMQ `3.9.x` versions up to `3.9.22` require Erlang `23.2` and support Erlang `24` (recommended).
Users of RabbitMQ versions that [do not support Erlang 23.2](https://www.rabbitmq.com/which-erlang.html) should first upgrade to the latest `3.9.22` on Erlang 23.2,
then upgrade to `3.10.0` on Erlang 24 or (ideally) 25.
### Getting Help
@ -64,287 +85,100 @@ Any questions about this release, upgrades or RabbitMQ in general are welcome on
## Changes Worth Mentioning
Release notes are kept under [rabbitmq-server/release-notes](https://github.com/rabbitmq/rabbitmq-server/tree/v3.10.x/release-notes).
Release notes are kept under [rabbitmq-server/release-notes](https://github.com/rabbitmq/rabbitmq-server/tree/v3.11.x/release-notes).
### Core Server
#### Enhancements
* Reduced memory footprint of quorum queues. Quorum queues now move enqueues messages to disk as quickly as
possible, ignoring the `x-max-in-memory-length` and `x-max-in-memory-bytes` policy keys. This reduces per-queue memory footprint
and makes throughput and latency less variable.
* Some features flags have [graduated to always enabled](https://blog.rabbitmq.com/posts/2022/07/required-feature-flags-in-rabbitmq-3.11/).
This means that they must be enabled before upgrading to 3.11. If that is not the case, the upgrade process will not proceed.
GitHub issue: [#3898](https://github.com/rabbitmq/rabbitmq-server/issues/3898)
* More efficient connection tracking. This means less CPU load in environments where high [connection churn](https://www.rabbitmq.com/connections.html#high-connection-churn)
cannot be avoided.
* New classic queue index and message store implementations. This makes memory footprint of classic queues less variable
and on average, lower than before. Consumer delivery is more efficient, potentially increasing throughput for workloads
where consumers do keep up with producers.
In part contributed by @gomoripeti.
This implementation is not enabled by default. It can be enabled using a policy:
GitHub issue: [#5610](https://github.com/rabbitmq/rabbitmq-server/pull/5610)
``` shell
rabbitmqctl set_policy cq_version "^cq\." '{"queue-version": 2}' --priority 1 --apply-to queues
```
or set via `rabbitmq.conf`:
``` ini
classic_queue.default_version = 2
```
GitHub issue: [#3029](https://github.com/rabbitmq/rabbitmq-server/pull/3029)
* Quorum queues now support [message TTL](https://rabbitmq.com/ttl.html).
GitHub issue: [#3121](https://github.com/rabbitmq/rabbitmq-server/pull/3121)
* Reduced per-queue garbage collection.
GitHub issue: [#4110](https://github.com/rabbitmq/rabbitmq-server/pull/4110)
* Stream replication connections now adopt [TLS settings configured for inter-node communication links](https://rabbitmq.com/clustering-ssl.html).
GitHub issue: [#3553](https://github.com/rabbitmq/rabbitmq-server/pull/3553)
* Quorum queues now support two [dead lettering](https://rabbitmq.com/dlx.html) strategies, at-most-once (the 3.9 quorum queue and classic queue behavior)
and at-least-once.
To use at-least-once the user must opt-in.
GitHub issue: [#3100](https://github.com/rabbitmq/rabbitmq-server/issues/3100)
* Messages delivered by a quorum queue and negatively acknowledged with a requeue will be added to the back of the queue
until the queue has redelivery limit set. With a redelivery limit, requeueing will use the original position of the message
if possible.
This improves operational safety: consumers that get stuck or requeue deliveries at a high rate will not indefinitely
grow quorum queue Raft log, potentially driving the node out of disk space.
![Breaking change](https://img.shields.io/badge/-Breaking%20change-red) This is a **potentially breaking change**.
Applications can make sure quorum queues have a redelivery limit
set if the original behavior is important to keep.
GitHub issue: [#3776](https://github.com/rabbitmq/rabbitmq-server/issues/3776)
* [Boot time definition sync](https://rabbitmq.com/definitions.html#import-on-boot) now can be skipped if definition source (e.g. file or directory) contents
have not changed:
``` ini
definitions.skip_if_unchanged = true
definitions.import_backend = local_filesystem
definitions.local.path = /path/to/definitions/defs.json
```
GitHub issue: [#4062](https://github.com/rabbitmq/rabbitmq-server/pull/4062)
* Classic mirrored queues peak eager sync throughput now can be limited.
Contributed by @thuandb (AWS).
GitHub issue: [#3925](https://github.com/rabbitmq/rabbitmq-server/pull/3925)
* [Optional queue arguments](https://rabbitmq.com/queues.html#optional-arguments) are now more strictly validated at queue
declaration time, as are optional consumer arguments when new consumers subscribe.
GitHub issue: [#4572](https://github.com/rabbitmq/rabbitmq-server/pull/4572)
* Headers exchanges (AMQP 0-9-1) now support a new matching strategy, `"any-with-x"`,
which works the same way as `"any"` but also considers headers that begin with
an `"x-"`.
GitHub issue: [#4143](https://github.com/rabbitmq/rabbitmq-server/pull/4143)
* Queues can no longer be created in a vhost while deletion of that vhost is in progress
Contributed by @tomyouyou.
GitHub issue: [#4260](https://github.com/rabbitmq/rabbitmq-server/pull/4260)
* rabbitmqctl encode/decode can now accept more values via standard input
GitHub issue: [#4258](https://github.com/rabbitmq/rabbitmq-server/pull/4258)
* rabbitmq_tracing now uses local time for timestamps
GitHub issue: [#4277](https://github.com/rabbitmq/rabbitmq-server/pull/4277)
#### Internal API and Other Changes
* Two rarely used classic queue metrics have been removed. Their removal reduces pressure
on internal table writes and boosts classic queue v1 throughput by 10 to 20 percent.
Contributed by @felixhuettner.
GitHub issue: [#4431](https://github.com/rabbitmq/rabbitmq-server/pull/4431)
* Tracing and `idle_since` timestamps now reflect local time in RFC 3339 format. Previously
a custom format was used for these timestamps and they always reflected UTC time.
GitHub issue: [#4276](https://github.com/rabbitmq/rabbitmq-server/pull/4276)
* Inter-node message transfer is optimized for the case where a message is only routed to one queue.
Contributed by @tomyouyou.
GitHub issue: [#3865](https://github.com/rabbitmq/rabbitmq-server/pull/3865)
* Definition import now can be extended to support arbitrary sources. This is done
by providing a module that implements three functions: `is_enabled/0`, `load/1`, `load_with_hashing/3`.
The module can then should be specified as the `definitions.import_backend` value:
``` ini
definitions.import_backend = rabbit_definitions_import_our_custom_backend
```
GitHub issue: [#3311](https://github.com/rabbitmq/rabbitmq-server/pull/3311)
* Elixir 1.13 is now supported at build time.
GitHub issue: [#4108](https://github.com/rabbitmq/rabbitmq-server/pull/4108)
* 10% to 20% throughput gain for publishers that use the default exchange.
GitHub issue: [#4606](https://github.com/rabbitmq/rabbitmq-server/pull/4606)
#### Bug Fixes
This release includes all applicable [bug fixes that shipped in `3.9.x` releases](https://www.rabbitmq.com/changelog.html).
* Durable classic queues could get "lost" under certain conditions when a node failed.
* Addressed a performance regression for large fanouts when streams were not enabled.
GitHub issue: [#4563](https://github.com/rabbitmq/rabbitmq-server/pull/4563)
GitHub issue: [#4073](https://github.com/rabbitmq/rabbitmq-server/pull/4073)
* Nodes could get stuck when performing a rolling restart on Kubernetes where DNS hostname
resolution was not available early on node boot.
For example, this is the case with a [popular default caching setting of CoreDNS](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#stable-network-id).
* Stream delivery rate could drop if concurrent stream consumers consumed in a way
that made them reach the end of the stream often.
GitHub issue: [#4371](https://github.com/rabbitmq/rabbitmq-server/pull/4371)
* Channels could confirm a subset of messages published as mandatory in some cases.
GitHub issue: [#3870](https://github.com/rabbitmq/rabbitmq-server/pull/3870)
* Direct Erlang client initialisation had a race condition with earlier boot steps,
which could lead plugins that depend on it to fail to open connections.
GitHub issue: [#3685](https://github.com/rabbitmq/rabbitmq-server/pull/3685)
* Exclusive queues are always placed on local node.
[Exclusive queues](https://rabbitmq.com/queues.html#exclusive-queues) are by definition node-local because they are tied to the lifecycle of
their declaring connection, which is node-local.
GitHub issue: [#3341](https://github.com/rabbitmq/rabbitmq-server/pull/3341)
GitHub issue: [#5438](https://github.com/rabbitmq/rabbitmq-server/pull/5438)
### Prometheus Plugin
### Stream Plugin
#### Bug Fixes
#### Enhancement
* Prometheus collector could run into an exception early on node's
schema database sync.
* RabbitMQ now supports [partitioned streams](https://blog.rabbitmq.com/posts/2022/07/rabbitmq-3-11-feature-preview-super-streams/) (super streams).
Super streams are a way to scale out by partitioning a large stream into smaller streams.
GitHub issue: [#4376](https://github.com/rabbitmq/rabbitmq-server/pull/4376)
* Stream protocol clients and RabbitMQ nodes now have a mechanism of exchanging
their capabilities (sets of supported commands).
GitHub issue: [#5308](https://github.com/rabbitmq/rabbitmq-server/issues/5308)
* New stream protocol command that allows certain stream metadata to be inspected by
applications.
GitHub issue: [#5412](https://github.com/rabbitmq/rabbitmq-server/issues/5412)
* [Single Active Consumer support for streams](https://blog.rabbitmq.com/posts/2022/07/rabbitmq-3-11-feature-preview-single-active-consumer-for-streams/).
GitHub issue: [#3754](https://github.com/rabbitmq/rabbitmq-server/pull/3754)
* Consumers now have access to their committed offset information.
GitHub issue: [#5307](https://github.com/rabbitmq/rabbitmq-server/issues/5307)
### OAuth 2 AuthN/AuthZ Backend Plugin
#### Enhancements
* OAuth 2 plugin now [supports more identity providers and OpenID Connect](https://blog.rabbitmq.com/posts/2022/07/oidc-integration/).
GitHub issue: [#4749](https://github.com/rabbitmq/rabbitmq-server/pull/4749)
* OAuth 2 plugin now supports [Rich Authorization Requests](https://oauth.net/2/rich-authorization-requests/).
GitHub issue: [#5234](https://github.com/rabbitmq/rabbitmq-server/pull/5234)
### Management Plugin
#### Enhancements
* `idle_since` field now uses RFC 3339 format.
* Management UI OAuth 2 integration supports more identity providers and [OpenID Connect](https://openid.net/connect/).
![Breaking change](https://img.shields.io/badge/-Breaking%20change-red) This is a **potentially breaking change**.
Sample value with the previous format: `2022-03-22 11:39:37`. With the new format: `2022-03-22T11:39:37.908+01:00`.
Clients of the REST API should be able to parse values in the new format.
GitHub issue: [#4749](https://github.com/rabbitmq/rabbitmq-server/pull/4749)
GitHub issue: [#4276](https://github.com/rabbitmq/rabbitmq-server/issues/4276)
* HTTP API responses that render a large rest sets are now more efficient thanks to a more efficient JSON serialiser library used internally.
Peak memory footprint of such queries is also double digit percent lower compared to v3.10.x.
#### Bug Fixes
GitHub issue: [#5356](https://github.com/rabbitmq/rabbitmq-server/pull/5356)
* `rabbitmqadmin` now correctly loads TLS-related keys from its configuration file.
* New endpoint for listing connections of a specific user.
Contributed by @fwolfsjaeger.
Contributed by @NuwanSameera.
GitHub issue: [#4408](https://github.com/rabbitmq/rabbitmq-server/pull/4408)
### MQTT Plugin
#### Enhancements
* Durable (QoS 1) subscriptions now can be backed by [quorum queues](https://www.rabbitmq.com/quorum-queues.html) if configured so.
This setting **can only be applied for new clusters**: existing durable classic
queues will not be converted. Enabling this setting in a cluster with existing
durable subscription data of MQTT clients will make it impossible for them to
subscribe without **explicit queue deletion** by the cluster operator.
To opt in to use quorum queues for durable MQTT subscriptions, use the `mqtt.durable_queue_type`
configuration key in `rabbitmq.conf`:
``` ini
mqtt.durable_queue_type = quorum
```
GitHub issue: [#4401](https://github.com/rabbitmq/rabbitmq-server/pull/4401)
### OAuth 2 AuthN and AuthZ Backend Plugin
#### Enhancements
* The plugin now supports scope aliases. In some environments, it's unrealistic to
adopt JWTs that follow the `scope` convention assumed by the plugin. Instead,
identity services fill `scope` or `claims` field with a "role name" or "role alias"
that implicitly maps to a set of scopes/permissions.
With this feature, RabbitMQ operators can map those values to a set of
scopes that can be translated to RabbitMQ permissions.
GitHub issue: [#4588](https://github.com/rabbitmq/rabbitmq-server/issues/4588)
* Improvements to JKW support and new HTTPS settings.
Contributed by @anhanhnguyen (Erlang Solutions).
GitHub issue: [#3887](https://github.com/rabbitmq/rabbitmq-server/pull/3887)
### Shovel Plugin
#### Enhancements
* Dynamic Shovels can be deleted or restarted from any cluster node
Contributed by @kostakal.
GitHub issue: [#4242](https://github.com/rabbitmq/rabbitmq-server/pull/4242)
#### Bug Fixes
* Shovels with automatic deletion threshold of `0` are now correctly deleted immediately.
Contribited by @luos (Erlang Solutions).
GitHub issue: [#4098](https://github.com/rabbitmq/rabbitmq-server/pull/4098)
### Consul Peer Discovery Plugin
#### Bug Fixes
* Handle a response code change in Consul 1.11.
Contributed by @maestroes.
GitHub issue: [#4149](https://github.com/rabbitmq/rabbitmq-server/issues/4149)
GitHub issue: [#5319](https://github.com/rabbitmq/rabbitmq-server/pull/5319)
### Dependency Changes
* `ra` upgraded from [`2.0.0` to `2.0.9`](https://github.com/rabbitmq/ra/compare/v2.0.0...v2.0.9)
* `osiris` upgraded from [`1.0.0` to `1.2.6`](https://github.com/rabbitmq/osiris/compare/v1.0.0...v1.2.6)
* `ranch` upgraded from [`2.0.0` to `2.1.0`](https://github.com/ninenines/ranch/compare/2.0.0...2.1.0)
* `prometheus` upgraded from [`4.8.1` to `4.8.2`](https://github.com/deadtrickster/prometheus.erl/compare/06425c21a39c1564164f1cc3fe5bdfa8b23b1f78...v4.8.2)
* `eetcd` upgraded from [`0.3.3` to `0.3.5`](https://github.com/zhongwencool/eetcd/compare/v0.3.3...v0.3.5)
* `observer_cli` upgraded from [`1.6.2` to `1.7.3`](https://github.com/zhongwencool/observer_cli/compare/1.6.2...1.7.3)
* `gen_batch_server` upgraded from [`0.8.6` to `0.8.7`](https://github.com/rabbitmq/gen-batch-server/compare/38191672ee0f22a8d5291c6c09f8c07178d565ca...v0.8.7)
* `ra` upgraded to [`2.3.0`](https://github.com/rabbitmq/ra/releases)
* `osiris` upgraded to [`1.3.0`](https://github.com/rabbitmq/osiris/tags)
* `prometheus` upgraded to [`4.9.0`](https://github.com/deadtrickster/prometheus.erl/tags)
* `jsx` was replaced with [`thoas`](https://github.com/lpil/thoas)