Per discussion in #10415, this introduces a new module,
rabbit_mgmt_nodes, which provides a couple of helpers
that can be used to implement Cowboy REST's
resource_exists/2 in the modules that return
information about cluster members.
`rabbit_mgmt_util:direct_request/6` is always called with an
`ErrorMsg` which expects one format argument as a string. Convert the
arbitrary reason term into a string to avoid a crash like the below:
```
warning: FORMATTER CRASH: {"Delete exchange error: ~ts",[{'EXIT',{{badmatch,{error,...
```
This prevents the below harmless crash when multiple parallel API
requests arrive soon after starting the node.
```
exception error: no match of right hand side value
{error,{already_started,<0.1593.0>}}
in function rabbit_mgmt_db_cache:fetch/4 (rabbit_mgmt_db_cache.erl, line 68)
in call from rabbit_mgmt_db:submit_cached/4 (rabbit_mgmt_db.erl, line 756)
in call from rabbit_mgmt_util:augment/2 (rabbit_mgmt_util.erl, line 412)
in call from rabbit_mgmt_util:run_augmentation/2 (rabbit_mgmt_util.erl, line 389)
in call from rabbit_mgmt_util:augment_resources0/6 (rabbit_mgmt_util.erl, line 378)
in call from rabbit_mgmt_util:with_valid_pagination/3 (rabbit_mgmt_util.erl, line 302)
in call from rabbit_mgmt_wm_queues:to_json/2 (rabbit_mgmt_wm_queues.erl, line 44)
in call from cowboy_rest:call/3 (src/cowboy_rest.erl, line 1583)
```
Avoid the following crash
```
** Reason for termination ==
** {mqtt_unexpected_cast,{shutdown,"Closed via management plugin"}}
crasher:
initial call: rabbit_mqtt_reader:init/1
pid: <0.1096.0>
registered_name: []
exception exit: {mqtt_unexpected_cast,
{shutdown,"Closed via management plugin"}}
in function gen_server:handle_common_reply/8 (gen_server.erl, line 1208)
```
when closing MQTT or Stream connections via HTTP API endpoint
```
/connections/username/:username
```
Scan queues, exchanges and bindings before attempting
to import anything on boot. If they miss the virtual
host field, fail early and log a sensible message.
Listing queues with the HTTP API when there are many (1000s) of
quorum queues could be excessively slow compared to the same scenario
with classic queues.
This optimises various aspects of HTTP API queue listings.
For QQs it removes the expensive cluster wide rpcs used to get the
"online" status of each quorum queue. This was previously done _before_
paging and thus would perform a cluster-wide query for _each_ quorum queue in
the vhost/system. This accounted for most of the slowness compared to
classic queues.
Secondly the query to separate the running from the down queues
consisted of two separate queries that later were combined when a single
query would have sufficed.
This commit also includes a variety of other improvements and minor
fixes discovered during testing and optimisation.
MINOR BREAKING CHANGE: quorum queues would previously only display one
of two states: running or down. Now there is a new state called minority
which is emitted when the queue has at least one member running but
cannot commit entries due to lack of quorum.
Also the quorum queue may transiently enter the down state when a node
goes down and before its elected a new leader.
Introduce GET /api/queues/detailed endpoint
Just removed garbage_collection, idle_since and any 'null' value
/api/queues with 10k classic queues returns 7.4MB of data
/api/queues/detailed with 10k classic queues returns 11MB of data
This sits behind a new feature flag, required to collect data from
all nodes: detailed_queues_endpoint
Diana Parra Corbacho
Michael Klishin
Michael Klishin
Loïc Hoguin
Ariel Otilibili
Marcial Rosales
Péter Gömöri
David Ansari
Diana Parra Corbacho
Johan Rhodin
Johan Rhodin
Ayanda Dube
Karl Nilsson
Emerson Almeida