rabbitmq-server/release-notes/4.1.0.md

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

378 lines
16 KiB
Markdown
Raw Permalink Normal View History

## RabbitMQ 4.1.0-beta.4
RabbitMQ 4.1.0-beta.4 is a preview release (in development) of a new feature release.
See Compatibility Notes below to learn about **breaking or potentially breaking changes** in this release.
2024-10-07 23:55:58 +08:00
## Highlights
Some key improvements in this release are listed below.
### Initial Support for AMQP 1.0 Filter Expressions
Support for the `properties` and `appliation-properties` filters of [AMQP Filter Expressions Version 1.0 Working Draft 09](https://groups.oasis-open.org/higherlogic/ws/public/document?document_id=66227).
### Feature Flags Quality of Life Improvements
Graduated (mandatory) [feature flags](https://www.rabbitmq.com/docs/feature-flags) several minors ago has proven that they could use some user experience improvements.
For example, certain required feature flags will now be enabled on node boot when all nodes in the cluster support them.
See core server changes below as well as the [GitHub project dedicated to feature flags improvements](https://github.com/orgs/rabbitmq/projects/4/views/1)
for the complete list of related changes.
## Breaking Changes and Compatibility Notes
### MQTT
* The default MQTT [Maximum Packet Size](https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901086) changed from 256 MiB to 16 MiB.
This default can be overridden by [configuring](https://www.rabbitmq.com/docs/configure#config-file) `mqtt.max_packet_size_authenticated`.
Note that this value must not be greater than `max_message_size` (which also defaults to 16 MiB).
### etcd Peer Discovery
The following `rabbitmq.conf` settings are unsupported:
* `cluster_formation.etcd.ssl_options.fail_if_no_peer_cert`
* `cluster_formation.etcd.ssl_options.dh`
* `cluster_formation.etcd.ssl_options.dhfile`
## Erlang/OTP Compatibility Notes
2024-12-11 10:50:18 +08:00
This release [requires Erlang 26.2](https://www.rabbitmq.com/docs/which-erlang) and supports Erlang 27.x.
[Provisioning Latest Erlang Releases](https://www.rabbitmq.com/docs/which-erlang#erlang-repositories) explains
2024-12-11 10:50:18 +08:00
what package repositories and tools can be used to provision latest patch versions of Erlang 26.x and 27.x.
## Release Artifacts
2024-12-11 08:57:44 +08:00
Artifacts for preview releases are distributed via GitHub releases:
* In main repository, [`rabbitmq/rabbitmq-server`](https://github.com/rabbitmq/rabbitmq-server/releases)
* In the development builds repository, [`rabbitmq/server-packages`](https://github.com/rabbitmq/server-packages/releases)
There is a `4.1.0` preview version of the [community RabbitMQ image](https://github.com/docker-library/rabbitmq).
## Upgrading to 4.1.0
### Documentation guides on upgrades
See the [Upgrading guide](https://www.rabbitmq.com/docs/upgrade) for documentation on upgrades and [GitHub releases](https://github.com/rabbitmq/rabbitmq-server/releases)
for release notes of individual releases.
This release series only supports upgrades from `4.0.x`.
[Blue/Green Deployment](https://www.rabbitmq.com/docs/blue-green-upgrade)-style upgrades are avaialble for migrations from 3.12.x and 3.13.x series
to `4.1.x`.
### Required Feature Flags
None/TBD.
### Mixed version cluster compatibility
RabbitMQ 4.1.0 nodes can run alongside `4.0.x` nodes. `4.1.x`-specific features can only be made available when all nodes in the cluster
upgrade to 4.0.0 or a later patch release in the new series.
While operating in mixed version mode, some aspects of the system may not behave as expected. The list of known behavior changes will be covered in future updates.
Once all nodes are upgraded to 4.1.0, these irregularities will go away.
Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended
periods of time (no more than a few hours).
### Recommended Post-upgrade Procedures
2024-12-11 08:57:44 +08:00
This version does not require any additional post-upgrade procedures
compared to other versions.
## Changes Worth Mentioning
2024-12-11 08:57:44 +08:00
This section can be incomplete and will be expanded as 4.1 approaches its release candidate stage.
### Core Server
#### Enhancements
* Feature flag quality of live improvements.
Certain required feature flags will now be automatically required on node boot
and do not have to be explicitly enabled before an upgrade.
This does not apply to all feature flags, however.
GitHub project: [#4](https://github.com/orgs/rabbitmq/projects/4/views/1).
GitHub issues: [#12466](https://github.com/rabbitmq/rabbitmq-server/pull/12466), [#12444](https://github.com/rabbitmq/rabbitmq-server/pull/12444),
[#12447](https://github.com/rabbitmq/rabbitmq-server/pull/12447)
* `properties` and `appliation-properties` filters of [AMQP Filter Expressions Version 1.0 Working Draft 09](https://groups.oasis-open.org/higherlogic/ws/public/document?document_id=66227)
when consuming from a stream via AMQP 1.0. String prefix and suffix matching is also supported.
This feature adds the ability to RabbitMQ to have multiple concurrent clients each consuming only a subset of messages while maintaining message order.
It also reduces network traffic between RabbitMQ and clients by only dispatching those messages that the clients are actually interested in.
GitHub issue: [#12415](https://github.com/rabbitmq/rabbitmq-server/pull/12415)
* AMQP 1.0 connections that use OAuth 2.0 now can renew their JWT tokens
This allows clients to set a new token proactively before the current one [expires](/docs/oauth2#token-expiration), ensuring uninterrupted connectivity.
If a client does not set a new token before the existing one expires, RabbitMQ will automatically close the AMQP 1.0 connection.
GitHub issue: [#12599](https://github.com/rabbitmq/rabbitmq-server/pull/12599)
* Nodes will now fall back to system CA certificate list (if available) when no CA certificate
is explicitly configured.
Contributed by @LoisSotoLopez.
GitHub issue: [#10519](https://github.com/rabbitmq/rabbitmq-server/issues/10519), [#12564](https://github.com/rabbitmq/rabbitmq-server/pull/12564)
* Support for Multiple Routing Keys in AMQP 1.0 via `x-cc` Message Annotation.
AMQP 1.0 publishers now can set multiple routing keys by using the `x-cc` message annotation.
This annotation allows publishers to specify a [list](https://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-types-v1.0-os.html#type-list)
of routing keys ([strings](https://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-types-v1.0-os.html#type-string)) for more flexible message distribution,
similar to the [CC](https://www.rabbitmq.com/docs/sender-selected) header in AMQP 0.9.1.
GitHub issue: [#12559](https://github.com/rabbitmq/rabbitmq-server/pull/12559)
2024-12-11 08:57:44 +08:00
* Peer discovery resilience improvements.
GitHub issues: [#12801](https://github.com/rabbitmq/rabbitmq-server/pull/12801), [#12809](https://github.com/rabbitmq/rabbitmq-server/pull/12809)
#### Bug Fixes
* AMQP 0-9-1 channel exception generator could not handle entity names (say, queue or stream names)
that contained non-ASCII characters.
This affected applications that use passive queue declarations, such as the Shovel plugin.
Contributed by @bpint.
GitHub issue: [#12888](https://github.com/rabbitmq/rabbitmq-server/pull/12888)
* Reintroduced transient flow control between classic queue replicas and AMQP 0-9-1 channels,
MQTT connections.
Flow control between these specific parts of the core were unintentionally
removed in `4.0.0` together with classic queue mirroring.
Contributed by @gomoripeti.
GitHub issue: [#12907](https://github.com/rabbitmq/rabbitmq-server/pull/12907)
* AMQP 1.0 connections with a higher consumption rate could set the incoming window field
on the flow frame to a negative value, which resulted in an exception that affected the consumer.
GitHub issues: [#12816](https://github.com/rabbitmq/rabbitmq-server/issues/12816)
* In rare cases quorum queue could end up without an elected leader because
chosen candidate replica was not verified for aliveness.
Contributed by @Ayanda-D.
GitHub issues: [#12727](https://github.com/rabbitmq/rabbitmq-server/pull/12727), [#10423](https://github.com/rabbitmq/rabbitmq-server/discussions/10423), [#12701](https://github.com/rabbitmq/rabbitmq-server/discussions/12701)
* When a new replica is added to a quorum queue, the node that handles this request will now wait
the operation to complete. Previously an early return could result in confusing `cluster_change_not_permitted`
errors for subsequent operations, for example, an addition of another replica.
GitHub issue: [#12837](https://github.com/rabbitmq/rabbitmq-server/pull/12837)
* In very rare cases, RabbitMQ could fail to notify stream consumers connected to follower replicas
about newly committed offsets as quickly as it usually happens for consumers connected to the stream leader.
GitHub issue: [#12785](https://github.com/rabbitmq/rabbitmq-server/pull/12785)
### MQTT Plugin
#### Enhancements
* The default MQTT [Maximum Packet Size](https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901086) changed from 256 MiB to 16 MiB.
This default can be overridden by [configuring](https://www.rabbitmq.com/docs/configure#config-file) `mqtt.max_packet_size_authenticated`.
Note that this value must not be greater than `max_message_size` (which also defaults to 16 MiB).
2024-12-11 08:57:44 +08:00
### CLI Tools
#### Enhancements
* New major version of `rabbitmqadmin`, a CLI tool that targets RabbitMQ's HTTP API, is maturing.
Unlike its predecessor, the tool is distirbuted via GitHub as as a standalone native binary.
There are minor command line interface changes and a slightly different configuration file
format ([TOML](https://toml.io/en/) instead of `ini`)
GitHub repository: [`rabbitmq/rabbitmqadmin-ng`](https://github.com/rabbitmq/rabbitmqadmin-ng)
* `rabbitmq-diagnostics check_if_any_deprecated_features_are_used` implementation is now more complete
(checks for a more deprecated features).
GitHub issue: [#12675](https://github.com/rabbitmq/rabbitmq-server/pull/12675)
#### Bug Fixes
* Fixes a false positive that incorrectly reported deprecated feature use, specifically
the use of non-exclusive transient classic queues.
GitHub issue: [#12840](https://github.com/rabbitmq/rabbitmq-server/pull/12840)
### Prometheus Plugin
#### Enhancements
* RabbitMQ nodes now provide a Prometheus histogram for message sizes published by applications.
2024-10-07 23:55:58 +08:00
This feature allows operators to gain insights into the message sizes being published to RabbitMQ,
such as average message size, number of messages per pre-defined bucket (which can both be computed accurately), and percentiles (which will be approximated).
Each metric is labelled by protocol (AMQP 1.0, AMQP 0.9.1, MQTT 5.0, MQTT 3.1.1, and MQTT 3.1).
2024-10-07 23:55:58 +08:00
GitHub issue: [#12342](https://github.com/rabbitmq/rabbitmq-server/pull/12342)
2024-10-07 23:55:58 +08:00
2024-12-11 08:57:44 +08:00
* Two new stream metrics for streams.
Contributed by @gomoripeti and @markus812498.
GitHub issue: [#12765](https://github.com/rabbitmq/rabbitmq-server/pull/12765)
### Grafana Dashboards
#### Bug Fixes
* Grafana 11.3.x compatibility.
Contributed by @anhanhnguyen.
GitHub issue: [#12720](https://github.com/rabbitmq/rabbitmq-server/pull/12720)
### Management UI
#### Enhancements
2024-12-11 08:57:44 +08:00
* Static assets served by the management UI now have a `control-cache` header set
to make sure that browsers reload them between upgrades.
UI code now tries to track version changes to perform a forced reload of the entire page.
GitHub issue: [#12749](https://github.com/rabbitmq/rabbitmq-server/pull/12749)
* Connection pages now display detailed AMQP 1.0 session and link information:
1. Link names
2. Link target and source addresses
3. Link flow control state
4. Session flow control state
5. Number of unconfirmed and unacknowledged messages
GitHub issue: [#12670](https://github.com/rabbitmq/rabbitmq-server/pull/12670)
* The management UI now shows if a feature flag has a migration function (in other words, it may take time to be enabled),
if it is experimental and whether it is supported or not. To enable an experimental feature flag,
a user must to tick checkboxes to confirm they know what they are doing.
2024-12-11 08:57:44 +08:00
GitHub issue: [#12643](https://github.com/rabbitmq/rabbitmq-server/pull/12643)
2024-12-11 08:57:44 +08:00
* Feature flags are now enabled using asynchronous requests in the management UI.
This means that feature flags that perform data migrations (which can take some time)
won't block the browser tab.
Support publishing AMQP 1.0 to Event Exchange ## What? Prior to this commit, the `rabbitmq_event_exchange` internally published always AMQP 0.9.1 messages to the `amq.rabbitmq.event` topic exchange. This commit allows users to configure the plugin to publish AMQP 1.0 messages instead. ## Why? Prior to this commit, when an AMQP 1.0 client consumed events, event properties that are lists were omitted. For example property `client_properties` of event `connection.created` or property `arguments` of event `queue.created` were omitted because of the following sequence: 1. The event exchange plugins listens for all kind of internal events. 2. The event exchange plugin re-publishes all events as AMQP 0.9.1 message to the event exchange. 3. Later, when an AMQP 1.0 client consumes this message, the broker must translate the message from AMQP 0.9.1 to AMQP 1.0. 4. This translation follows the rules outlined in https://www.rabbitmq.com/docs/conversions#amqpl-amqp 5. Specifically, in this table the row before the last one describes the rule we're hitting here. It says that if the AMQP 0.9.1 header value is not an `x-` prefixed header and its value is an array or table, then this header is not converted. That's because AMQP 1.0 application-properties must be simple types as mandated in https://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-messaging-v1.0-os.html#type-application-properties ## How? The user can configure the plugin as follows to have the plugin internally publish AMQP 1.0 messages: ``` event_exchange.protocol = amqp_1_0 ``` To support complex types such as lists, the plugin sets all event properties as AMQP 1.0 message-annotations. The plugin prefixes all message annotation keys with `x-opt-` to comply with the AMQP 1.0 spec. ## Alternative Design An alternative design would have been to format all event properties e.g. as JSON within the message body. However, this breaks routing on specific event property values via a headers exchange. ## Documentation https://github.com/rabbitmq/rabbitmq-website/pull/2129
2024-11-12 19:16:15 +08:00
2024-12-11 08:57:44 +08:00
GitHub issue: [#12643](https://github.com/rabbitmq/rabbitmq-server/pull/12643)
Support publishing AMQP 1.0 to Event Exchange ## What? Prior to this commit, the `rabbitmq_event_exchange` internally published always AMQP 0.9.1 messages to the `amq.rabbitmq.event` topic exchange. This commit allows users to configure the plugin to publish AMQP 1.0 messages instead. ## Why? Prior to this commit, when an AMQP 1.0 client consumed events, event properties that are lists were omitted. For example property `client_properties` of event `connection.created` or property `arguments` of event `queue.created` were omitted because of the following sequence: 1. The event exchange plugins listens for all kind of internal events. 2. The event exchange plugin re-publishes all events as AMQP 0.9.1 message to the event exchange. 3. Later, when an AMQP 1.0 client consumes this message, the broker must translate the message from AMQP 0.9.1 to AMQP 1.0. 4. This translation follows the rules outlined in https://www.rabbitmq.com/docs/conversions#amqpl-amqp 5. Specifically, in this table the row before the last one describes the rule we're hitting here. It says that if the AMQP 0.9.1 header value is not an `x-` prefixed header and its value is an array or table, then this header is not converted. That's because AMQP 1.0 application-properties must be simple types as mandated in https://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-messaging-v1.0-os.html#type-application-properties ## How? The user can configure the plugin as follows to have the plugin internally publish AMQP 1.0 messages: ``` event_exchange.protocol = amqp_1_0 ``` To support complex types such as lists, the plugin sets all event properties as AMQP 1.0 message-annotations. The plugin prefixes all message annotation keys with `x-opt-` to comply with the AMQP 1.0 spec. ## Alternative Design An alternative design would have been to format all event properties e.g. as JSON within the message body. However, this breaks routing on specific event property values via a headers exchange. ## Documentation https://github.com/rabbitmq/rabbitmq-website/pull/2129
2024-11-12 19:16:15 +08:00
2024-12-11 08:57:44 +08:00
#### Bug Fixes
2024-10-07 23:55:58 +08:00
2024-12-11 08:57:44 +08:00
* Fixes a false positive that incorrectly reported deprecated feature use, specifically
the use of non-exclusive transient classic queues.
2024-10-07 23:55:58 +08:00
2024-12-11 08:57:44 +08:00
GitHub issue: [#12840](https://github.com/rabbitmq/rabbitmq-server/pull/12840)
* When a logged in user's JWT token was refreshed, the user identity displayed in the UI was changed.
GitHub issue: [#12818](https://github.com/rabbitmq/rabbitmq-server/pull/12818)
### Shovel Plugin
#### Bug Fixes
* AMQP 0-9-1 channel exception generator could not handle entity names (say, queue or stream names)
that contained non-ASCII characters.
This affected applications that use passive queue declarations, such as the Shovel plugin.
Contributed by @bpint.
GitHub issue: [#12888](https://github.com/rabbitmq/rabbitmq-server/pull/12888)
2024-10-07 23:55:58 +08:00
### Event Exchange Plugin
#### Enhancements
* The `rabbitmq_event_exchange` plugin now can be configured to internally publish AMQP 1.0 instead of AMQP 0.9.1 messages to the `amq.rabbitmq.event` topic exchange.
This allows AMQP 1.0 consumers to receive event properties containing complex types such as [lists](https://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-types-v1.0-os.html#type-list)
or [maps](https://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-types-v1.0-os.html#type-map), for example queue arguments for the `queue.created`
event or client provided properties for the `connection.created` event.
GitHub issue: [#12714](https://github.com/rabbitmq/rabbitmq-server/pull/12714)
2024-12-11 08:57:44 +08:00
### AWS Peer Discovery Plugin
#### Bug Fixes
* Avoids an exception during automatic removal of cluster members that are
no longer returned by peer discovery (an [opt-in feature](https://www.rabbitmq.com/docs/cluster-formation#node-health-checks-and-cleanup)).
GitHub issue: [#12809](https://github.com/rabbitmq/rabbitmq-server/pull/12809)
### Kubernetes Peer Discovery Plugin
#### Bug Fixes
* Avoids an exception during automatic removal of cluster members that are
no longer returned by peer discovery (an [opt-in feature](https://www.rabbitmq.com/docs/cluster-formation#node-health-checks-and-cleanup)).
GitHub issue: [#12809](https://github.com/rabbitmq/rabbitmq-server/pull/12809)
### Consul Peer Discovery Plugin
#### Bug Fixes
* Avoids an exception during automatic removal of cluster members that are
no longer returned by peer discovery (an [opt-in feature](https://www.rabbitmq.com/docs/cluster-formation#node-health-checks-and-cleanup)).
GitHub issue: [#12809](https://github.com/rabbitmq/rabbitmq-server/pull/12809)
### etcd Peer Discovery Plugin
#### Bug Fixes
* Avoids an exception during automatic removal of cluster members that are
no longer returned by peer discovery (an [opt-in feature](https://www.rabbitmq.com/docs/cluster-formation#node-health-checks-and-cleanup)).
GitHub issue: [#12809](https://github.com/rabbitmq/rabbitmq-server/pull/12809)
### Dependency Changes
2024-12-11 08:57:44 +08:00
* `osiris` was upgraded to [`1.8.5`](https://github.com/rabbitmq/osiris/releases)
## Source Code Archives
To obtain source code of the entire distribution, please download the archive named `rabbitmq-server-4.1.0.tar.xz`
instead of the source tarball produced by GitHub.