Commit Graph

58467 Commits

Author SHA1 Message Date
Michael Klishin d224d01cca
Alpha workflows: use a short SHA here 2024-11-11 00:57:02 -05:00
Michael Klishin df7fac06d8
Alpha workflows: fixes 2024-11-11 00:33:14 -05:00
Michael Klishin 07b694f1e3
4.1.x alpha workflows: syntax correction 2024-11-11 00:27:54 -05:00
Michael Klishin 89ce948e5b
Alpha workflows: generate and set prerelease_identifier (again) 2024-11-11 00:26:37 -05:00
Michael Klishin 10a7706e83
release-4.1.x-alphas workflow: trigger a run 2024-11-10 23:04:41 -05:00
Michael Klishin eaaacbeb99
release-4.0.x-alphas workflow: sync with its 4.1.x counterpart 2024-11-10 22:57:02 -05:00
Michael Klishin 4d961fefcc
4.1.x alpha release workflow: rely on github.event.repository.pushed_at 2024-11-10 22:52:53 -05:00
Michael Klishin 440de4673e
4.1.x alpha release workflow: WIP 2024-11-10 22:50:51 -05:00
Michael Klishin 6d8ea93fc5
Trigger a new 4.1.x alpha build workflow 2024-11-10 22:18:30 -05:00
Michael Klishin 4f4ab35790
Trigger a new 4.1.x alpha build workflow 2024-11-10 21:51:36 -05:00
Michael Klishin 0810e2832d
Trigger a new 4.1.x alpha build workflow 2024-11-10 21:48:21 -05:00
Michael Klishin aff0bc814a
4.1.x alpha workflow: try release-kit/unix-timestamp@v1 2024-11-10 21:44:20 -05:00
Michael Klishin e988b7e660
Trigger a new 4.1.x alpha build workflow 2024-11-10 21:26:45 -05:00
Michael Klishin 2acc51ddc5
Trigger a new 4.1.x alpha build workflow 2024-11-10 20:52:13 -05:00
Michael Klishin 074f38d16f
Trigger a new 4.1.x alpha build workflow 2024-11-10 20:29:53 -05:00
Michael Klishin b43a7263f5
List cluster_tags in rabbitmq.conf.example #12552 #12659 #12699 2024-11-10 20:26:24 -05:00
Michael Klishin f5801be6db
Merge pull request #12659 from rabbitmq/su_aws/cluster_tag
Make it possible to set some cluster metadata besides the name using tags
2024-11-10 19:17:53 -05:00
Michael Klishin 7c66fba0c3
Make it possible to clear cluster_tags via rabbitmq.conf 2024-11-10 14:38:34 -05:00
Michael Klishin 9e649aefc0
We no longer use 'maybe' in this module 2024-11-10 14:35:14 -05:00
Michael Klishin e5d805ea6d
Cluster tags: set unconditionally
Otherwise once set, it would not be possible
to change them by updating rabbitmq.conf
2024-11-10 14:30:51 -05:00
Michael Klishin 6b614fc879
rabbitmq.conf.example: add management.http.max_body_size 2024-11-09 18:02:16 -05:00
Michael Klishin 961e5c5a21
Undo the Bazel-related change from #12696
(cherry picked from commit a66c926985)
2024-11-09 17:47:06 -05:00
Michael Klishin 673826425a
Merge pull request #12696 from rabbitmq/mk-http-api-lower-body-length-limit-for-binding-creation
HTTP API: reduce body size limit for the endpoint used to bind queues/streams/exchanges
2024-11-09 17:13:03 -05:00
Michael Klishin 3dc5c463a4
Pass Dialyzer 2024-11-09 16:53:45 -05:00
Michael Klishin b0abf88aa8
rabbit_mgmt_util: minor refactoring 2024-11-09 16:38:48 -05:00
Michael Klishin fb300d2a4b
HTTP API: limit default body size for binding creation
It does not need to use the "worst case scenario"
default HTTP request body size limit that
is primarily necessary because definition imports
can be large (MiBs in size, for example).

Since exchange, queue names and routing key
have limits of 255 bytes and optional arguments
can practically be expected to be short, we
can lower the limit to < 10 KiB.
2024-11-09 16:16:35 -05:00
Simon Unge eeea517da5 Store tags in global parameters 2024-11-08 21:41:49 +00:00
Simon Unge f5ef64ad06 Add cluster tag config that is exposed via HTTP /api/overview and CTL cluster_status 2024-11-08 21:05:02 +00:00
Michael Klishin 8b554474a6
Simplify 4.1.x and 4.0.x alpha release workflows
timestamping is now performed in rabbitmq/server-packages
2024-11-08 11:22:06 -05:00
David Ansari d309ec9afd
Merge pull request #12694 from rabbitmq/mqtt_flake_will_delay_session_takeover
Fix flake will_delay_session_takeover
2024-11-08 16:26:32 +01:00
David Ansari ae423721ad Fix flake will_delay_session_takeover
Prior to this commit, the following flake occurred in CI for
```
make -C deps/rabbitmq_mqtt ct-v5 t=cluster_size_1:will_delay_session_takeover
```

```
=== Location: [{v5_SUITE,will_delay_session_takeover,1473},
              {test_server,ts_tc,1793},
              {test_server,run_test_case_eval1,1302},
              {test_server,run_test_case_eval,1234}]
=== === Reason: {test_case_failed,"Received unexpected PUBLISH payload. Expected: <<\"will-3a\">> Got: <<\"will-4a\">>"}
```

The RabbitMQ logs for this single node test show:
```
2024-11-04 14:43:35.039196+00:00 [debug] <0.1334.0> MQTT accepting TCP connection <0.1334.0> (127.0.0.1:42576 -> 127.0.0.1:27005)
2024-11-04 14:43:35.039336+00:00 [debug] <0.1334.0> Received a CONNECT, client ID: c3, username: undefined, clean start: true, protocol version: 5, keepalive: 60, property names: []
2024-11-04 14:43:35.039438+00:00 [debug] <0.1334.0> MQTT connection 127.0.0.1:42576 -> 127.0.0.1:27005 picked vhost using plugin_configuration_or_default_vhost
2024-11-04 14:43:35.039537+00:00 [debug] <0.1334.0> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal
2024-11-04 14:43:35.039729+00:00 [info] <0.1334.0> Accepted MQTT connection 127.0.0.1:42576 -> 127.0.0.1:27005 for client ID c3
2024-11-04 14:43:35.040297+00:00 [debug] <0.1337.0> MQTT accepting TCP connection <0.1337.0> (127.0.0.1:42580 -> 127.0.0.1:27005)
2024-11-04 14:43:35.040442+00:00 [debug] <0.1337.0> Received a CONNECT, client ID: c4, username: undefined, clean start: true, protocol version: 5, keepalive: 60, property names: []
2024-11-04 14:43:35.040534+00:00 [debug] <0.1337.0> MQTT connection 127.0.0.1:42580 -> 127.0.0.1:27005 picked vhost using plugin_configuration_or_default_vhost
2024-11-04 14:43:35.040597+00:00 [debug] <0.1337.0> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal
2024-11-04 14:43:35.040793+00:00 [info] <0.1337.0> Accepted MQTT connection 127.0.0.1:42580 -> 127.0.0.1:27005 for client ID c4
2024-11-04 14:43:35.041463+00:00 [debug] <0.1340.0> MQTT accepting TCP connection <0.1340.0> (127.0.0.1:42596 -> 127.0.0.1:27005)
2024-11-04 14:43:35.041715+00:00 [debug] <0.1340.0> Received a CONNECT, client ID: c1, username: undefined, clean start: false, protocol version: 5, keepalive: 60, property names: ['Session-Expiry-Interval']
2024-11-04 14:43:35.041806+00:00 [debug] <0.1340.0> MQTT connection 127.0.0.1:42596 -> 127.0.0.1:27005 picked vhost using plugin_configuration_or_default_vhost
2024-11-04 14:43:35.041881+00:00 [debug] <0.1340.0> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal
2024-11-04 14:43:35.041982+00:00 [warning] <0.1328.0> MQTT disconnecting client <<"127.0.0.1:42560 -> 127.0.0.1:27005">> with duplicate id 'c1'
2024-11-04 14:43:35.042062+00:00 [info] <0.1340.0> Accepted MQTT connection 127.0.0.1:42596 -> 127.0.0.1:27005 for client ID c1
2024-11-04 14:43:35.045624+00:00 [debug] <0.1345.0> MQTT accepting TCP connection <0.1345.0> (127.0.0.1:42602 -> 127.0.0.1:27005)
2024-11-04 14:43:35.045781+00:00 [debug] <0.1345.0> Received a CONNECT, client ID: c2, username: undefined, clean start: false, protocol version: 5, keepalive: 60, property names: ['Session-Expiry-Interval']
2024-11-04 14:43:35.045874+00:00 [debug] <0.1345.0> MQTT connection 127.0.0.1:42602 -> 127.0.0.1:27005 picked vhost using plugin_configuration_or_default_vhost
2024-11-04 14:43:35.045943+00:00 [debug] <0.1345.0> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal
2024-11-04 14:43:35.046032+00:00 [warning] <0.1331.0> MQTT disconnecting client <<"127.0.0.1:42566 -> 127.0.0.1:27005">> with duplicate id 'c2'
2024-11-04 14:43:35.046281+00:00 [info] <0.1345.0> Accepted MQTT connection 127.0.0.1:42602 -> 127.0.0.1:27005 for client ID c2
2024-11-04 14:43:35.047063+00:00 [debug] <0.1350.0> MQTT accepting TCP connection <0.1350.0> (127.0.0.1:42614 -> 127.0.0.1:27005)
2024-11-04 14:43:35.047702+00:00 [debug] <0.1350.0> Received a CONNECT, client ID: c3, username: undefined, clean start: true, protocol version: 5, keepalive: 60, property names: ['Session-Expiry-Interval']
2024-11-04 14:43:35.047910+00:00 [debug] <0.1350.0> MQTT connection 127.0.0.1:42614 -> 127.0.0.1:27005 picked vhost using plugin_configuration_or_default_vhost
2024-11-04 14:43:35.048467+00:00 [debug] <0.1350.0> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal
2024-11-04 14:43:35.049701+00:00 [info] <0.1350.0> Accepted MQTT connection 127.0.0.1:42614 -> 127.0.0.1:27005 for client ID c3
2024-11-04 14:43:35.050907+00:00 [warning] <0.1334.0> MQTT disconnecting client <<"127.0.0.1:42576 -> 127.0.0.1:27005">> with duplicate id 'c3'
2024-11-04 14:43:35.051248+00:00 [debug] <0.1353.0> MQTT accepting TCP connection <0.1353.0> (127.0.0.1:42626 -> 127.0.0.1:27005)
2024-11-04 14:43:35.051395+00:00 [debug] <0.1353.0> Received a CONNECT, client ID: c4, username: undefined, clean start: false, protocol version: 5, keepalive: 60, property names: ['Session-Expiry-Interval']
2024-11-04 14:43:35.051519+00:00 [debug] <0.1353.0> MQTT connection 127.0.0.1:42626 -> 127.0.0.1:27005 picked vhost using plugin_configuration_or_default_vhost
2024-11-04 14:43:35.051590+00:00 [debug] <0.1353.0> User 'guest' authenticated successfully by backend rabbit_auth_backend_internal
2024-11-04 14:43:35.051871+00:00 [info] <0.1353.0> Accepted MQTT connection 127.0.0.1:42626 -> 127.0.0.1:27005 for client ID c4
2024-11-04 14:43:35.051960+00:00 [warning] <0.1337.0> MQTT disconnecting client <<"127.0.0.1:42580 -> 127.0.0.1:27005">> with duplicate id 'c4'
2024-11-04 14:43:35.052689+00:00 [debug] <0.1337.0> sent Will Message to topic my/topic for MQTT client ID c4
2024-11-04 14:43:35.054119+00:00 [debug] <0.1334.0> sent Will Message to topic my/topic for MQTT client ID c3
```

We see nicely how RabbitMQ sends the will message for both c3 and c4.
However, the order in which RabbitMQ sends is not guaranteed.
Hence, we adapt the test expectation to not depend on the order of Will
messages being received.
2024-11-08 16:12:52 +01:00
Arnaud Cogoluègnes ca70f20bca
Merge pull request #12686 from rabbitmq/stream-coordinator-ra-local-query-infinity-timeout
Use infinity timout for RA local query in stream coordinator
2024-11-08 15:08:16 +01:00
David Ansari c839409599 Fix test flake properties_section
This test flaked in CI with the following error:
```
=== === Reason: no match of right hand side value {error,half_attached}
  in function  amqp_utils:detach_link_sync/1 (amqp_utils.erl, line 100)
  in call from amqp_filtex_SUITE:properties_section/1 (amqp_filtex_SUITE.erl, line 187)
  in call from test_server:ts_tc/3 (test_server.erl, line 1793)
  in call from test_server:run_test_case_eval1/6 (test_server.erl, line 1302)
  in call from test_server:run_test_case_eval/9 (test_server.erl, line 1234)
```
2024-11-08 11:46:19 +01:00
David Ansari fa61266979
Merge pull request #12689 from rabbitmq/mqtt_flake_management_plugin_connection
Fix MQTT test flake management_plugin_connection
2024-11-08 11:45:16 +01:00
David Ansari 9095f7d961 Fix test flake
Increase waiting for credit being applied as described in commit
aeedad7b51 since this test case still flakes rarely with:
```
=== === Reason: {assertEqual,[{module,amqp_client_SUITE},
                               {line,3030},
                               {expression,"amqp10_msg : body ( Msg1 )"},
                               {expected,[<<"1">>]},
                               {value,[<<"2">>]}]}
  in function  amqp_client_SUITE:detach_requeues_two_connections/2 (amqp_client_SUITE.erl, line 3030)
  in call from test_server:ts_tc/3 (test_server.erl, line 1793)
  in call from test_server:run_test_case_eval1/6 (test_server.erl, line 1302)
  in call from test_server:run_test_case_eval/9 (test_server.erl, line 1234)
```
2024-11-08 11:35:04 +01:00
David Ansari 090384fe37 Fix MQTT test flake management_plugin_connection
Prior to this commit this test flaked in CI:
```
=== === Reason: {assertEqual,
                     [{module,mqtt_shared_SUITE},
                      {line,1222},
                      {expression,"http_get ( Config , \"/connections\" )"},
                      {expected,[]},
                      {value,
                          [#{timeout => 99,
                             name => <<"127.0.0.1:58712 -> 127.0.0.1:29005">>,
                             node =>
                                 <<"rmq-ct-mqtt-cluster_size_1-1-29000@localhost">>,
                             port => 29005,user => <<"guest">>,ssl => false,
                             protocol => <<"MQTT 5-0">>,
                             host => <<"127.0.0.1">>,
                             client_properties =>
                                 #{client_id =>
                                       <<"management_plugin_connection">>},
                             vhost => <<"/">>,peer_host => <<"127.0.0.1">>,
                             peer_port => 58712,frame_max => 0,
                             channel_max => 0,auth_mechanism => <<"none">>,
                             connected_at => 1730797370048,
                             ssl_protocol => null,ssl_key_exchange => null,
                             ssl_cipher => null,ssl_hash => null,
                             peer_cert_issuer => null,
                             peer_cert_subject => null,
                             peer_cert_validity => null,
                             user_who_performed_action => <<"guest">>}]}]}
  in function  mqtt_shared_SUITE:management_plugin_connection/1 (mqtt_shared_SUITE.erl, line 1222)
  in call from test_server:ts_tc/3 (test_server.erl, line 1793)
  in call from test_server:run_test_case_eval1/6 (test_server.erl, line 1302)
  in call from test_server:run_test_case_eval/9 (test_server.erl, line 1234)
```
2024-11-08 11:12:04 +01:00
David Ansari db6401d480
Merge pull request #12687 from rabbitmq/mqtt-flake-consumer
Fix MQTT test flake
2024-11-08 11:11:46 +01:00
David Ansari 40bf778e89 Fix MQTT test flake
Prior to this commit, test
```
make -C deps/rabbitmq_mqtt ct-mqtt_shared t=[mqtt,cluster_size_1,v4]:non_clean_sess_reconnect_qos0_and_qos1
```

flaked in CI with error:
```
{mqtt_shared_SUITE,non_clean_sess_reconnect_qos0_and_qos1,972}
{badmatch,{publish_not_received,<<"msg-0">>}}
```

The problem was the following race condition:
* The MQTT v4 client sends an async DISCONNECT
* The global MQTT consumer metric got decremented. However, the classic
  queue still has the MQTT connection proc registered as consumer.
* The test case sends a message
* The classic queue checks out the message to the old connection instead
  of checking out the message to the new connection.

The solution in this commit is to check the consumer count of the
classic queue before proceeding to send the message after disconnection.
2024-11-08 10:35:36 +01:00
Arnaud Cogoluègnes 1634adbff3
Use infinity timout for RA local query in stream coordinator
The 5-second default timeout is too short.
2024-11-08 09:26:24 +01:00
Michael Klishin 1872ce981a
Alpha release workflows: cosmetics 2024-11-07 17:48:04 -05:00
Michael Klishin f582b15587
Rename a workflow file 2024-11-07 17:45:12 -05:00
Michael Klishin 6d448639e5
4.1.x, 4.0.x alpha release workflows: move base versions to repository-scoped variables 2024-11-07 15:43:35 -05:00
Michael Klishin bcfca2be98
Merge pull request #12670 from rabbitmq/amqp-connection-sessions
Show session and link details for AMQP 1.0 connection
2024-11-07 14:39:47 -05:00
Arnaud Cogoluègnes 430ed9e65f
Merge pull request #12684 from rabbitmq/simplify-stream-manager
Remove gen_server behaviour from stream manager
2024-11-07 18:36:54 +01:00
Arnaud Cogoluègnes 26f941b815
Squash dialyzer warning 2024-11-07 18:17:47 +01:00
Arnaud Cogoluègnes 1554b74fc7
Use 4-space indent in stream manager 2024-11-07 17:03:50 +01:00
Arnaud Cogoluègnes 5107fd48ba
Remove gen_server behaviour from stream manager
The stream manager does not need to be a gen_server (no cast, no state)
and the gen_server can create contention for large stream deployments
(some functions make cluster-wide calls that can take some time).
2024-11-07 16:50:31 +01:00
David Ansari 124ef694bc Fix crashes
This commit fixes two different bugs/crashes.

To repro, prior to this commit:
1. Create an AMQP 1.0 connection on node-1.
2. Open the Management UI on node-2 and open the connection page of this
   single AMQP 1.0 connection.

The first crash was the following:
```
[error] <0.1297.0>   crasher:
[error] <0.1297.0>     initial call: cowboy_stream_h:request_process/3
[error] <0.1297.0>     pid: <0.1297.0>
[error] <0.1297.0>     registered_name: []
[error] <0.1297.0>     exception error: no case clause matching
[error] <0.1297.0>                      {badrpc,
[error] <0.1297.0>                          {'EXIT',
[error] <0.1297.0>                              {undef,
[error] <0.1297.0>                                  [{rabbit_connection_tracking,lookup,
[error] <0.1297.0>                                       [<<"[::1]:51729 -> [::1]:5672">>,
[error] <0.1297.0>                                        ['rabbit-1@ABCDDDEEAA']],
[error] <0.1297.0>                                       []}]}}}
[error] <0.1297.0>       in function  rabbit_connection_tracking:lookup/2 (rabbit_connection_tracking.erl, line 235)
[error] <0.1297.0>       in call from rabbit_mgmt_wm_connection_sessions:conn/1 (rabbit_mgmt_wm_connection_sessions.erl, line 72)
[error] <0.1297.0>       in call from rabbit_mgmt_wm_connection_sessions:is_authorized/2 (rabbit_mgmt_wm_connection_sessions.erl, line 63)
[error] <0.1297.0>       in call from cowboy_rest:call/3 (src/cowboy_rest.erl, line 1590)
[error] <0.1297.0>       in call from cowboy_rest:is_authorized/2 (src/cowboy_rest.erl, line 368)
[error] <0.1297.0>       in call from cowboy_rest:upgrade/4 (src/cowboy_rest.erl, line 284)
[error] <0.1297.0>       in call from cowboy_stream_h:execute/3 (src/cowboy_stream_h.erl, line 306)
[error] <0.1297.0>       in call from cowboy_stream_h:request_process/3 (src/cowboy_stream_h.erl, line 295)
```

The second crash was the following:
```
[error] <0.1132.0>   crasher:
[error] <0.1132.0>     initial call: cowboy_stream_h:request_process/3
[error] <0.1132.0>     pid: <0.1132.0>
[error] <0.1132.0>     registered_name: []
[error] <0.1132.0>     exception error: no case clause matching
[error] <0.1132.0>                      {tracked_connection,
[error] <0.1132.0>                          {'rabbit-1@ABCDDDEEAA',
[error] <0.1132.0>                              <<"[::1]:65505 -> [::1]:5672">>},
[error] <0.1132.0>                          'rabbit-1@ABCDDDEEAA',<<"/">>,
[error] <0.1132.0>                          <<"[::1]:65505 -> [::1]:5672">>,<13661.1110.0>,
[error] <0.1132.0>                          {1,0},
[error] <0.1132.0>                          network,
[error] <0.1132.0>                          {0,0,0,0,0,0,0,1},
[error] <0.1132.0>                          65505,<<"guest">>,1730908606089}
[error] <0.1132.0>       in function  rabbit_connection_tracking:lookup/2 (rabbit_connection_tracking.erl, line 235)
[error] <0.1132.0>       in call from rabbit_mgmt_wm_connection_sessions:conn/1 (rabbit_mgmt_wm_connection_sessions.erl, line 72)
[error] <0.1132.0>       in call from rabbit_mgmt_wm_connection_sessions:is_authorized/2 (rabbit_mgmt_wm_connection_sessions.erl, line 63)
[error] <0.1132.0>       in call from cowboy_rest:call/3 (src/cowboy_rest.erl, line 1590)
[error] <0.1132.0>       in call from cowboy_rest:is_authorized/2 (src/cowboy_rest.erl, line 368)
[error] <0.1132.0>       in call from cowboy_rest:upgrade/4 (src/cowboy_rest.erl, line 284)
[error] <0.1132.0>       in call from cowboy_stream_h:execute/3 (src/cowboy_stream_h.erl, line 306)
[error] <0.1132.0>       in call from cowboy_stream_h:request_process/3 (src/cowboy_stream_h.erl, line 295)
2024-11-07 15:11:42 +01:00
David Ansari 9d0c851df2 Show session and link details for AMQP 1.0 connection
## What?

On the connection page in the Management UI, display detailed session and
link information including:
* Link names
* Link target and source addresses
* Link flow control state
* Session flow control state
* Number of unconfirmed and unacknowledged messages

 ## How?

A new HTTP API endpoint is added:
```
/connections/:connection_name/sessions
```

The HTTP handler first queries the Erlang connection process to find out about
all session Pids. The handler then queries each Erlang session process
of this connection.

(The table auto-refreshes by default every 5 seconds. The handler querying a single
connection with 60 idle sessions with each 250 links takes ~100 ms.)

For better user experience in the Management UI, this commit also makes the
session process store and expose link names as well as source/target addresses.
2024-11-07 15:11:42 +01:00
Jean-Sébastien Pédron 8dd49a532d
Merge pull request #12679 from rabbitmq/link-to-deprecated-features-docs-from-management-ui
rabbitmq_management: Link from the deprecated features panel to docs
2024-11-07 09:59:47 +01:00