Commit Graph

57202 Commits

Author SHA1 Message Date
Jean-Sébastien Pédron 0276c7b11f
[Remove this change before creating a pull reques] 2024-08-13 13:55:19 +02:00
Jean-Sébastien Pédron 3e63cfcb40
Delete child resources before parent 2024-08-13 13:55:19 +02:00
Jean-Sébastien Pédron b7f217a88f
Experiment with #exchange records stored as maps 2024-08-02 12:14:09 +02:00
Jean-Sébastien Pédron b934dc597c
rabbit_db_queue: Use other path functions to extract the name of a queue
[Why]

The function was previously hard-coding the path. Therefore, it was
defined in multiple places. With the upcoming reorganization of the
data, it's best to not do that.
2024-08-01 18:44:29 +02:00
Jean-Sébastien Pédron f745a57ccc
rabbt_db: Unify Khepri paths API
[Why]

Currently, `rabbit_db_*` modules use and export the following kind of
functions to return the path to the resources they manage:

    khepri_db_thing:khepri_things_path(),
    khepri_db_thing:khepri_thing_path(Identifier).

Internally, `khepri_db_thing:khepri_thing_path(Identifier)` appends
`Identifier` to the list returned by
`khepri_db_thing:khepri_things_path()`. This works for the organization
of the records we have today in Khepri:

    |-- thing
    |   |-- <<"identifier1">>
    |   |   <<"identifier2">>
    `-- other_thing
	`-- <<"other_identifier1">>

However, with the upcoming organization that leverages the tree in
Khepri, identifiers may be in the middle of the path instead of a leaf
component. We may also put `other_thing` under `thing` in the tree.

That's why, we can't really expose a parent directory for `thing` and
`other_thing`. Therefore, `khepri_db_thing:khepri_things_path/0` needs
to go away. Only `khepri_db_thing:khepri_thing_path/1` should be
exported and used.

[How]

The patch does exactly that. Uses of
`khepri_db_thing:khepri_things_path()` are generally replaced by
`rabbit_db_thing:khepri_thing_path(?KHEPRI_WILDCARD_STAR)`.

In the future, for a resource that depends on another one, the
corresponding module will call the `rabbit_db_thing:khepri_thing_path/1`
for that other resource and build its path on top of that.
2024-08-01 18:44:29 +02:00
Jean-Sébastien Pédron cd5522cccb
Update Khepri from 0.14.0 to 0.15.0
Release notes:
https://github.com/rabbitmq/khepri/releases/tag/v0.15.0

The `favor` default value is now `low_latency`. Therefore, we don't need
to specify it explicitly anymore.
2024-08-01 18:43:31 +02:00
Michael Klishin 0525ab06a0 rabbitmq.conf.example: mention log.file.rotation.* keys 2024-08-01 01:11:46 -04:00
David Ansari 0397035669 Add test for AMQP 1.0 clients using OAuth token 2024-07-31 12:05:22 +02:00
Michael Klishin f91498284e
Merge pull request #11867 from rabbitmq/mqtt-credential-expiration
Disconnect MQTT client when its credential expires
2024-07-30 21:10:12 -04:00
David Ansari d7f29426a8 Fix test flake
Sometimes in CI under Khepri, the test case errored with:
```
receiver_attached flushed: {amqp10_event,
                            {session,<0.396.0>,
                             {ended,
                              {'v1_0.error',
                               {symbol,<<"amqp:internal-error">>},
                               {utf8,
                                <<"stream queue 'leader_transfer_stream_credit_single' in vhost '/' does not have a running replica on the local node">>},
                               undefined}}}}
```
2024-07-30 21:05:25 +02:00
David Ansari 7fb78338c6 Disconnect MQTT client when its credential expires
Fixes https://github.com/rabbitmq/rabbitmq-server/discussions/11854
Fixes https://github.com/rabbitmq/rabbitmq-server/issues/11862

This commit uses the same approach as implemented for AMQP 1.0 and
Streams: When a token expires, RabbitMQ will close the connection.
2024-07-30 19:55:46 +02:00
David Ansari 4e3ff2c8ef Delete leftover code
This code should have been deleted as part of
https://github.com/rabbitmq/rabbitmq-server/pull/11642
2024-07-30 18:47:09 +02:00
David Ansari 9d9a69aed9 Make AMQP flow control configurable
Make the following AMQP 1.0 flow control variables configurable via
`advanced.config`:
* `max_incoming_window` (session flow control)
* `max_link_credit` (link flow control)
* `max_queue_credit` (link flow control)
2024-07-30 16:40:52 +02:00
Michael Klishin 03885faabf
Merge pull request #11857 from rabbitmq/classic_queue_consumer_unsent_message_limit
Make classic_queue_consumer_unsent_message_limit configurable
2024-07-29 23:16:08 -04:00
David Ansari c771b2422a Make classic_queue_consumer_unsent_message_limit configurable
Similar to other RabbitMQ internal credit flow configurations such as
`credit_flow_default_credit` and `msg_store_credit_disc_bound`, this
commit makes the `classic_queue_consumer_unsent_message_limit`
configurable via `advanced.config`.

See https://github.com/rabbitmq/rabbitmq-server/pull/11822 for the
original motivation to make this setting configurable.
2024-07-29 22:48:48 +02:00
Michael Klishin 777fd6fc80
Merge pull request #11846 from rabbitmq/credit-reply-crash
Fix quorum queue credit reply crash in AMQP session
2024-07-28 19:52:36 -04:00
David Ansari ce915ae05a Fix quorum queue credit reply crash in AMQP session
Fixes #11841

PR #11307 introduced the invariant that at most one credit request between
session proc and quorum queue proc can be in flight at any given time.
This is not the case when rabbit_fifo_client re-sends credit
requests on behalf of the session proc when the quorum queue leader changes.

This commit therefore removes assertions which assumed only a single credit
request to be in flight.

This commit also removes field queue_flow_ctl.desired_credit
since it is redundant to field client_flow_ctl.credit
2024-07-28 12:34:41 +02:00
David Ansari d3109e9f09 Remove max_frame_size from AMQP writer
because the session process already splits frames that are too large
into smaller frames
2024-07-26 16:35:36 +02:00
David Ansari dde8e699a1 Report frame_max as integer
Resolves https://github.com/rabbitmq/rabbitmq-server/issues/11838
2024-07-26 16:35:36 +02:00
Michael Klishin 963c5d6c8f
Merge pull request #11839 from rabbitmq/gazelle-main
bazel run gazelle
2024-07-26 00:22:44 -04:00
GitHub f011b54767 bazel run gazelle 2024-07-26 04:02:38 +00:00
Michael Klishin 4aaa1c410e
Merge pull request #11664 from rabbitmq/khepri-node-added-event
rabbit_node_monitor: use a leader query for cluster members on node_added event
2024-07-25 15:41:28 -04:00
Michael Klishin 29251a0a54
Merge pull request #11706 from rabbitmq/md/khepri-minority-errors/rabbit_db_vhost
Handle timeouts possible in Khepri minority in `rabbit_db_vhost`
2024-07-25 15:40:40 -04:00
Michael Klishin 5a56e326d9
Merge pull request #11741 from rabbitmq/retry-register-projections-during-boot
rabbit_khepri: Retry register_projections during boot
2024-07-25 15:39:15 -04:00
Michael Klishin b605f78602
Merge pull request #11831 from rabbitmq/stream-plugin-close-with-delay-on-authentication-failure
Close stream connection with delay in case of authentication failure
2024-07-25 12:11:01 -04:00
Arnaud Cogoluègnes c9951ec1f4
Close stream connection with delay in case of authentication failure
For consistency with other protocols (to protect from potential DoS
attacks). Wrong credentials and virtual host access errors trigger
the delay.
2024-07-25 17:00:36 +02:00
Michael Klishin 8a17043045
Merge pull request #11793 from rabbitmq/shovel-predeclared-destinations
Configure shovels to wait until all sources and destinations are declared
2024-07-24 18:27:10 -04:00
Michael Klishin 5161aea393 Shovel dynamic_SUITE: ignore one more harmless exception
when a socket is closed before a writer tries
to flush it.
2024-07-24 17:50:45 -04:00
Marcial Rosales 61c9cf2ce2 More clean up 2024-07-24 17:50:45 -04:00
Marcial Rosales ae17c6c86a Clean up 2024-07-24 17:50:45 -04:00
Marcial Rosales 86a0ebe1af First wait until shovel is terminated 2024-07-24 17:50:45 -04:00
Marcial Rosales 17e470e6eb Fix test 2024-07-24 17:50:45 -04:00
Marcial Rosales e2e92d3214 Support predeclared feature in static shovels 2024-07-24 17:50:45 -04:00
Marcial Rosales 55bc5a2920 Remove unnecessary function 2024-07-24 17:50:44 -04:00
Marcial Rosales c24756e505 Static shovels remain as they are 2024-07-24 17:50:44 -04:00
Marcial Rosales 48f1bc7507 Clean up 2024-07-24 17:50:44 -04:00
Marcial Rosales 42e42219b7 Configure plugin with topology.predeclared 2024-07-24 17:50:44 -04:00
Marcial Rosales a888c7b576 Test predeclared dest queue 2024-07-24 17:50:44 -04:00
Marcial Rosales 04164df6cf Test predeclared for queues in shovels 2024-07-24 17:50:44 -04:00
Marcial Rosales 5465b44332 Test predeclared for src 2024-07-24 17:50:44 -04:00
Michael Davis 4207faf433
Merge pull request #11785 from rabbitmq/md/khepri-minority-errors/rabbit_db_exchange
Handle timeouts possible in Khepri minority in `rabbit_db_exchange`
2024-07-24 12:11:17 -05:00
Michael Davis b56abeec12
Use `rabbit_misc:rs/1` on exchange resource records
This fixes a potential crash in `rabbit_amqp_amanegment` where we tried
to format the exchange resource as a string (`~ts`). The other changes
are cosmetic.
2024-07-24 11:32:33 -04:00
Michael Davis fb3154ba82
rabbit_channel: Fix formatting of error message for exchange deletion
Co-authored-by: David Ansari <david.ansari@gmx.de>
2024-07-24 11:19:31 -04:00
Karl Nilsson 1a9da90153
Merge pull request #11809 from rabbitmq/qq-system-recovery
QQ: use a dedicated function for queue recovery after Ra system restart.
2024-07-24 16:14:05 +01:00
Michael Davis 98616a0037
rabbit_amqp_management: Use HTTP code 503 for timeout errors
`rabbit_amqp_management` returns HTTP status codes to the client. 503
means that a service is unavailable (which Khepri is while it is in a
minority) so it's a more appropriate code than the generic 500
internal server error.
2024-07-24 11:13:17 -04:00
Michal Kuratczyk ae41f65c64
Fix rabbit_priority_queue:update_rates bug (#11814)
updates_rates fails after publishing a message to a queue
with priorities enabled.
2024-07-24 16:34:56 +02:00
Karl Nilsson 4863bc3b8f QQ: use a dedicated function for queue recovery after Ra system restart.
Previously we used the `registered` approach where all Ra servers that
have a registered name would be recovered. This could have unintended
side effects for queues that e.g. were deleted when not all members of
a quorum queueu were running when the queue was deleted. In this case
the Ra system would have recovered the members that were not deleted
which is not ideal as a dangling member would just sit and loop in
pre vote state and a future declaration of the queue may partially
fail.

Instead we rely on the meta data store for the truth about which
members should be restarted after a ra system restart.
2024-07-24 14:24:42 +01:00
David Ansari be6a7fec95 Fix test flake
Sometimes on Khepri the test failed with:
```
=== Ended at 2024-07-24 10:07:15
=== Location: [{gen_server,call,419},
              {amqpl_direct_reply_to_SUITE,rpc,226},
              {test_server,ts_tc,1793},
              {test_server,run_test_case_eval1,1302},
              {test_server,run_test_case_eval,1234}]
=== === Reason: {{shutdown,
                      {server_initiated_close,404,
                          <<"NOT_FOUND - no queue 'tests.amqpl_direct_reply_to.rpc.requests' in vhost '/'">>}},
                  {gen_server,call,
                      [<0.272.0>,
                       {call,
                           {'basic.get',0,
                               <<"tests.amqpl_direct_reply_to.rpc.requests">>,
                               false},
                           none,<0.246.0>},
                       infinity]}}
```

https://github.com/rabbitmq/rabbitmq-server/actions/runs/10074558971/job/27851173817?pr=11809
shows an instance of this flake.
2024-07-24 13:42:20 +02:00
Karl Nilsson 665a9dd39a
Merge pull request #11808 from rabbitmq/fix-consistent-hash-exchange
Fix crash in consistent hash exchange
2024-07-24 11:48:14 +01:00
David Ansari cdc5b886f8 Fix crash in consistent hash exchange
Prior to this commit, a crash occurred when a consistent hash exchange
got declared with a `hash-header` argument, but the publishing client
didn't set that header on the message.

This bug is present in RabbitMQ 3.13.0 - 3.13.6.

Fixes https://github.com/rabbitmq/rabbitmq-server/discussions/11671
2024-07-24 11:42:59 +02:00