Trigger a 4.3.x alpha release build / trigger_alpha_build (push) Waiting to runDetails
Test (make) / Build and Xref (1.18, 26) (push) Waiting to runDetails
Test (make) / Build and Xref (1.18, 27) (push) Waiting to runDetails
Test (make) / Build and Xref (1.18, 28) (push) Waiting to runDetails
Test (make) / Test (1.18, 28, khepri) (push) Waiting to runDetails
Test (make) / Test (1.18, 28, mnesia) (push) Waiting to runDetails
Test (make) / Test mixed clusters (1.18, 28, khepri) (push) Waiting to runDetails
Test (make) / Test mixed clusters (1.18, 28, mnesia) (push) Waiting to runDetails
Test (make) / Type check (1.18, 28) (push) Waiting to runDetails
`rabbit_amqqueue:declare/7` consults the `is_enabled/0` callback of
`rabbit_queue_type` before attempting to create a queue.
`rabbit_stream_manager` emulates `rabbit_amqqueue:declare/7` rather
than calling it directly. To align with the `is_enabled/0` check in
`rabbit_amqqueue:declare/7`, `rabbit_stream_manager` needs to check
the callback during stream creation.
* Add comprehensive tests for the `rabbitmq_aws` cuttlefish schema.
* Move `aws_prefer_imdsv2` setting to the `rabbitmq_aws` application.
* Use AWS session token when present in env or config file. It was only used with IMDSv2 previously.
* Add rabbitmq_aws:api_post_request/4, README cleanup
A quorum queue tries to repair its record in a tick handler. This can
happen during a network partition and the metadata store may itself be
unavailable, making the update likely to time out.
The default metadata store timeout is usually higher than the tick
interval, so the tick handler may be stuck during several ticks. The
record takes some time to be updated (timeout + tick interval, 30 + 5
seconds by default), significantly longer than it takes the metadata
store to trigger an election and recover.
Client applications may rely on the quorum queue topology to connect to
an appropriate node, so making the system reflect the actual topology
faster is important to them.
This commit makes the record update operations use a timeout 1-second
lower than the tick interval. The tick handler process should finish
earlier in case of metadata datastore unavailability and it should not
take more than a couple of ticks once the datastore is available to
update the record.
... instead of as part of a parallel CT group.
[Why]
This testsuite takes quite a lot of time. It affects the time to run the
parallel CT set 2 significantly, especially if it has to be restarted.
Previously direct 0-9-1 connections did not notice when memory or disk
alarms were set. This could allow an 0-9-1 shovel where the destination
is a direct connection to completely overload a broker which is already
in alarm. With this change, direct connections register the connection
process with `rabbit_alarm` and emit `connection.blocked` and
`connection.unblocked` to the blocked handler if one is registered.
`rabbit_amqp091_shovel` already respects the `connection.blocked`, so
the destination will not receive any messages.
This change also refactors them to use two unclustered nodes. This is
a prerequisite for the child change which will remove the workaround of
using a direct connection to be able to publish while a node is in
alarm.
[Why]
The `*_queue_after_partition_recovery_1` testcases rely on the fact that
the queue process retries to delete its queue record in its terminate
function.
RabbitMQ 4.1.x and before don't have that. Thus, depending on the timing
of the election of the Khepri leader after a network partition, the test
might fail.
As mentioned in discussion #14426, the way that `cacerts` is handled by
cuttlefish schemas simply will not work if set.
If `cacerts` were set to a string value containing one X509 certificate,
it would eventually result in a crash because the `cacerts` ssl option
must be of [this type](https://www.erlang.org/doc/apps/ssl/ssl.html#t:client_option_cert/0):
```
{cacerts, CACerts :: [public_key:der_encoded()] | [public_key:combined_cert()]}
```
Neither of those are strings, of course.
This PR removes all use of `cacerts` in cuttlefish schemas. In addition,
it filters out `cacerts` and `certs_keys` from being JSON-encoded by an
HTTP API call to `/api/overview`. It _is_ technically possible to set
`cacerts` via `advanced.config`, so, if set, it would crash this API
call, as would `certs_keys`.
[Why]
With Mnesia, when the network partition strategy is set to
`pause_minority`, nodes on the "minority side" are stopped.
Thus, the exclusive queues that were hosted by nodes on that minority
side are lost:
* Consumers connected on these nodes are disconnected because the nodes
are stopped.
* Queue records on the majority side are deleted from the metadata
store.
This was ok with Mnesia and how this network partition handling strategy
is implemented. However, it does not work with Khepri because the nodes
on the "minority side" continue to run and serve clients. Therefore the
cluster ends up in a weird situation:
1. The "majority side" deleted the queue records.
2. When the network partition is solved, the "minority side" gets the
record deletion, but the queue processes continue to run.
This was similar for auto-delete queues.
[How]
With Khepri, we stop to delete transient queue records in general, just
because there is a node going down. Thanks to this, an exclusive or an
auto-delete queue and its consumer(s) are not affected by a network
partition: they continue to work.
However, if a node is really lost, we need to clean up dead queue
records. This was already done for durable queues with both Mnesia and
Khepri. But with Khepri, transient queue records persist in the store
like durable queue records (unlike with Mnesia).
That's why this commit changes the clean-up function,
`rabbit_amqqueue:forget_all_durable/1` into
`rabbit_amqqueue:forget_all/1` which deletes all queue records of queues
that were hosted on the given node, regardless if they are transient or
durable.
In addition to this, the queue process will spawn a temporary process
who will try to delete the underlying record indefinitely if no other
processes are waiting for a reply from the queue process. That's the
case for queues that are deleted because of an internal event (like the
exclusive/auto-delete conditions). The queue process will exit, which
will notify connections that the queue is gone.
Thanks to this, the temporary process will do its best to delete the
record in case of a network partition, whether the consumers go away
during or after that partition. That said, the node monitor drives some
failsafe code that cleans up record if the queue process was killed
before it could delete its own record.
Fixes#12949, #12597, #14527.
[Why]
So far, when there was a network partition with Mnesia, the most popular
partition handling strategies restarted RabbitMQ nodes. Therefore,
`rabbit` would execute the boot steps and one of them would notify other
members of the cluster that "this RabbitMQ node is live".
With Khepri, nodes are not restarted anymore and thus, boot steps are
not executed at the end of a network partition. As a consequence, other
members are not notified that a member is back online.
[How]
When the node monitor receives the `nodeup` message (managed by Erlang,
meaning that "a remote Erlang node just connected to this node through
Erlang distribution"), a `node_up` message is sent to all cluster
members (meaning "RabbitMQ is now running on the originating node").
Yeah, very poor naming...
This lets the RabbitMQ node monitor know when other nodes running
RabbitMQ are back online and react accordingly.
If a node is restarted, it means that another node could receive the
`node_up` message twice. The actions behind it must be idempotent.
Peer Discovery AWS Integration Test / Integration Test (push) Has been cancelledDetails
Previously if the peer discovery backend returned an error from failing
to discover nodes, the `service_discovery_nodes/0` helper returned an
empty list. During cleanup this would mean that any nodes unreachable
during a partition would have destructive action taken against them:
`rabbit_db_cluster:forget_member/2` and `rabbit_quorum_queue:shrink_all/1`.
The `list_nodes/0` callback can fail transiently, though, and a failure
shouldn't mean that the cluster is empty. It's safer to avoid cleaning
up any nodes when the peer discovery backend fails to return the
intended set of nodes.
This change adds config options for controlling the `is_enabled/0`
queue type callback for classic queues, quorum queues and stream
queues. This is useful for operators to be able to control whether
users can declare these queues - i.e. to prevent users from declaring
queue types which cannot be supported operationally.
Setting
queue_types.stream.enabled = false
In a config file for example prevents stream queues from being declared.
'user_provided_plugins_data_dir' make it immediately
obvious that the data directory is for 3rd party
plugins (user-provided plugins), and it is
not set up or used internally by RabbitMQ.
Michael Klishin
Luke Bakken
Diana Parra Corbacho
Michael Davis
Sunny Katkuri
Arnaud Cogoluègnes
dependabot[bot]
Jean-Sébastien Pédron
Michael Klishin
markus812498