Commit Graph

58467 Commits

Author SHA1 Message Date
Michael Klishin ea58fb1b48
crashing_queues_SUITE: squash a compiler warning 2024-11-17 17:23:00 -05:00
Michael Klishin 9f026f7a4b
Merge pull request #12727 from rabbitmq/rabbitmq-server-12709
By @Ayanda-D: Ensure only alive leaders and followers when fetching QQ replica states
2024-11-15 13:53:41 -05:00
Ayanda Dube 53cc8f8f2b
Update unit_quorum_queue_SUITE to use temporary alive & registered
test queue processes (since we now check/return only alive members
when fetching replica states)

(cherry picked from commit ebc0387b81)
2024-11-15 12:49:55 -05:00
David Ansari 6e8b566323 Deduplicate AMQP type inference
Introduce a single place in the AMQP 1.0 Erlang client that infers the AMQP 1.0 type.

Erlang integers are inferred to be AMQP type `long` to avoid overflow surprises.
2024-11-15 17:40:36 +01:00
Jean-Sébastien Pédron 44e74ceb96
Merge pull request #12739 from rabbitmq/fix-khepri-reset
Fix RabbitMQ reset when using Khepri
2024-11-15 17:40:16 +01:00
Jean-Sébastien Pédron 2938338182
rabbit_khepri: Do not hard-code `coordination`, use the constant instead 2024-11-15 16:41:16 +01:00
Jean-Sébastien Pédron 05717ccccf
rabbit_khepri: Remove serial file during reset 2024-11-15 16:40:50 +01:00
Jean-Sébastien Pédron e41d766b29
rabbit_khepri: Ensure RabbitMQ is stopped before resetting with Khepri 2024-11-15 16:40:45 +01:00
Jean-Sébastien Pédron 248d7a9d22
Merge pull request #12736 from rabbitmq/support-relative-forced-feature-flags-in-conf
rabbit_feature_flags: Support relative setting in `forced_feature_flags_on_init`
2024-11-15 15:17:18 +01:00
Jean-Sébastien Pédron 7e2e7b79f2
rabbit_feature_flags: Support relative setting in `forced_feature_flags_on_init`
[Why]
We already support that from the environment variable, it is easy to add
to the configuration setting.
2024-11-15 14:50:35 +01:00
Michael Klishin 3846e5e22c
Minor Discussion template updates 2024-11-15 02:31:53 -05:00
Michael Klishin 4766d91dfe
bazel run gazelle 2024-11-15 02:29:56 -05:00
Michael Klishin 8df2deb36e
4.1.0 release notes: 4.1.0-beta.1 is out 2024-11-15 00:54:00 -05:00
Michael Klishin 1b05ab46bb
gitgnore CT spec files for parallel test runs 2024-11-15 00:52:26 -05:00
Michael Klishin 3e509c9f30
Merge pull request #12714 from rabbitmq/amqp-event-exchange
Support publishing AMQP 1.0 to Event Exchange
2024-11-14 18:09:19 -05:00
Michael Klishin 89b15cc2ec
Merge pull request #12662 from rabbitmq/test-tls-auth-mqtt
Selenium suites: Test TLS-based authentication via messaging protocols
2024-11-14 14:18:08 -05:00
Ayanda Dube 3ecb3b61d4
Use whereis/1 instead of rabbit_process helper, and lists:filtermap/2 in
rabbit_quorum_queue:all_replica_states/0

(cherry picked from commit 19cc2d0608)
2024-11-14 14:05:17 -05:00
Ayanda Dube 6bb4c89c71
Add test for rabbit_quorum_queue:all_replica_states/0
and ensure non-existent/inactive/noproc QQ members are
not reported.

(cherry picked from commit 4e2c62b6af)
2024-11-14 14:05:12 -05:00
Ayanda Dube 9070e394d3
Ensure only alive QQ replica states are reported
when checking replica states to help avoid missing
inactive replicas e.g. on QQ checks from cli tools

(cherry picked from commit 491485092c)
2024-11-14 14:05:08 -05:00
Michael Klishin d5063c7076
Merge pull request #12675 from rabbitmq/fix-check_if_any_deprecated_features_are_used-cli-command
CLI: Finish `check_if_any_deprecated_features_are_used` implementation
2024-11-14 13:57:40 -05:00
Michael Klishin 15d3d5a8a1
Merge pull request #12674 from rabbitmq/add-is_feature_used-callback-to-transient_nonexcl_queues-depr-feature
rabbit_amqqueue: Add `is_feature_used` callback to `transient_nonexcl_queues` depr. feature
2024-11-14 13:57:32 -05:00
Marcial Rosales 6bf27a212f Use tls in oauth providers and rabbitmq 2024-11-14 19:39:06 +01:00
Michael Klishin c888689cca
Merge pull request #12722 from rabbitmq/fix-flakes
Fix flakes
2024-11-14 13:36:17 -05:00
Michael Klishin 79f712897d
Merge pull request #12720 from anhanhnguyen/update/grafana-erlang-dashboards
Grafana Dashboards: Fix Query Errors and Improve Instance Filtering in Erlang Distribution and BEAM Dashboards
2024-11-14 13:18:02 -05:00
Michael Klishin 4ebfd7f5e8
Merge pull request #12721 from rabbitmq/loic-fix-cq-flake
CQ: Fix flakes in the store file scan test
2024-11-14 13:15:53 -05:00
Loïc Hoguin db50739ad8
CQ: Fix flakes in the store file scan test
We don't expect random bytes to be there in the current
version of the message store as we overwrite empty spaces
with zeroes when moving messages around.

We also don't expect messages to be false flagged when
the broker is running because it checks for message
validity in the index. Therefore make sure message bodies
in the tests don't contain byte 255.
2024-11-14 15:04:49 +01:00
Diana Parra Corbacho 6e7269994d Tests: per_node_limit_SUITE cleanup
Catch exceptions when closing connections during cleanup
2024-11-14 15:02:47 +01:00
Diana Parra Corbacho db78f9b812 Tests: mqtt_shared_SUITE match expected connection 2024-11-14 15:02:47 +01:00
Diana Parra Corbacho 5ef4fba851 tests: amqp_client_SUITE longer wait on receive for CI 2024-11-14 15:02:47 +01:00
Diana Parra Corbacho 9054b122fd tests: clustering_SUITE wait for stats 2024-11-14 15:02:47 +01:00
Diana Parra Corbacho 2d025b579b Tests: amqpl_consumer_ack use unmanaged connection 2024-11-14 15:02:47 +01:00
Diana Parra Corbacho e9a365b20e tests: clustering_prop_SUITE force stats on every wait 2024-11-14 15:02:47 +01:00
Diana Parra Corbacho 067a54aa40 tests: clustering_SUITE wait for metrics 2024-11-14 15:02:47 +01:00
David Ansari de804d1fa7 Support publishing AMQP 1.0 to Event Exchange
## What?

Prior to this commit, the `rabbitmq_event_exchange` internally published
always AMQP 0.9.1 messages to the `amq.rabbitmq.event` topic exchange.
This commit allows users to configure the plugin to publish AMQP 1.0
messages instead.

 ## Why?

Prior to this commit, when an AMQP 1.0 client consumed events,
event properties that are lists were omitted. For example property
`client_properties` of event `connection.created` or property
`arguments` of event `queue.created` were omitted because of the following sequence:
1. The event exchange plugins listens for all kind of internal events.
2. The event exchange plugin re-publishes all events as AMQP 0.9.1 message to the event exchange.
3. Later, when an AMQP 1.0 client consumes this message, the broker must translate the message from AMQP 0.9.1 to AMQP 1.0.
4. This translation follows the rules outlined in https://www.rabbitmq.com/docs/conversions#amqpl-amqp
5. Specifically, in this table the row before the last one describes the rule we're hitting here. It says that if the AMQP 0.9.1
header value is not an `x-` prefixed header and its value is an array or table, then this header is not converted.
That's because AMQP 1.0 application-properties must be simple types as mandated in https://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-messaging-v1.0-os.html#type-application-properties

 ## How?

The user can configure the plugin as follows to have the plugin
internally publish AMQP 1.0 messages:
```
event_exchange.protocol = amqp_1_0
```

To support complex types such as lists, the plugin sets all event
properties as AMQP 1.0 message-annotations. The plugin prefixes all message
annotation keys with `x-opt-` to comply with the AMQP 1.0 spec.

 ## Alternative Design

An alternative design would have been to format all event properties
e.g. as JSON within the message body. However, this breaks routing on
specific event property values via a headers exchange.

 ## Documentation
https://github.com/rabbitmq/rabbitmq-website/pull/2129
2024-11-14 12:52:09 +01:00
Anh Nguyen dc9311a561 Update Erlang Distribution dashboard panel and instance filtering
- Modified metric expression and legend format in State of distribution links
- Changed panel type from 'flant-statusmap-panel' to 'status-history' for Process state
2024-11-14 11:04:07 +07:00
Michael Klishin 27952938f8
Merge pull request #12712 from rabbitmq/gh_12608
QQ: reduce memory use when dropping many messages at once.
2024-11-13 11:42:20 -05:00
Anh Nguyen b9dc0ea3b4 Add instance filtering to Erlang BEAM Grafana dashboard metrics
- Updated metric expressions to include instance filtering with {instance=\"$node\"}
  for the following metrics:
  - erlang_vm_statistics_run_queues_length
  - erlang_vm_statistics_dirty_io_run_queue_length
  - erlang_vm_statistics_dirty_cpu_run_queue_length
- Added 'DS_PROMETHEUS' as a templated data source variable
2024-11-13 20:20:02 +07:00
Karl Nilsson bfa293ab3b QQ: reduce memory use when dropping many messages at once.
As may happen when a max_length configuration change is made
when there are many messages on the queue.
2024-11-13 09:07:40 +00:00
Michael Klishin c78bc8a9c3
4.1: Avoid an exception when an AMQP 0-9-1-originating message with expiration set is converted for an MQTT consumer (#12710)
* MQTT: avoid an exception

when an AMQP 0-9-1 publisher publishes a message
that has expiration set.

Stack trace was contributed in #12707 by @rdsilio.

* mc_mqtt_SUITE test for #12707 #12710

* MQTT protocol_interop_SUITE: new test for #12710 #12707

* Simplify tests

---------

Co-authored-by: David Ansari <david.ansari@gmx.de>
2024-11-13 09:20:43 +01:00
Michael Klishin 6c16b4dd64
4.0.1 release notes: clarify that Blue/Green deployments are an option for 3.13.x clusters with Khepri enabled 2024-11-12 19:26:21 -05:00
Michael Klishin 62a4c0a922
Continuous build workflow: trigger when rabbitmq-components.mk changes 2024-11-12 11:46:18 -05:00
Michael Klishin e09b02fa46
Merge pull request #12704 from rabbitmq/osiris-v1.8.4
Osiris v1.8.4
2024-11-12 11:26:48 -05:00
Karl Nilsson 5f4715979c Osiris v1.8.4
This osiris release contains a bug fix that would cause an osiris
member to crash during recovery if certain unexpected files
were present in the log directory. (.e.g ".nfsXXXXXXXXXXXX") type
files used by the NFS file system when in use files are deleted.
2024-11-12 10:20:38 +00:00
Michael Klishin aba62b9d12
Mention node_tags #12702 in rabbitmq.conf.example 2024-11-11 22:56:47 -05:00
Michael Klishin 859fd03d0d
Merge pull request #12702 from rabbitmq/su_aws/node_tags
Make it possible to set some node-specific metadata using tags
2024-11-11 18:34:26 -05:00
Michael Klishin 94c8f01699
rabbitmq-diagnostics status: handle output of 3.13.x and previously released 4.0.x nodes
In a mixed cluster environment,
'rabbitmq-diagnostics status' can hit a node
that does not return any node tags.

Be more defensive and handle such cases
by simply displaying "(none)" for such
values.
2024-11-11 17:41:33 -05:00
Simon Unge 3d35416635 Node tags local to broker, add to /api/overview output and ctl status command 2024-11-11 20:49:21 +00:00
Jean-Sébastien Pédron 638e3a4b08
rabbit_amqqueue: Add `is_feature_used` callback to `transient_nonexcl_queues` depr. feature
[Why]
Without this callback, the deprecated features subsystem can't report if
the feature is used or not.

This reduces the usefulness of the HTTP API endpoint or the CLI command
that help verify if a cluster is using deprecated features.

[How]
The callback counts transient non-exclusive queues and return `true` if
there are one or more of them.

References #12619.
2024-11-11 15:57:52 +01:00
Jean-Sébastien Pédron ddaea6facb
CLI: Finish `check_if_any_deprecated_features_are_used` implementation
[Why]
The previous implementation bypassed the deprecated features subsystem.
It only cared about classic mirrored queues and called some
queue-related code directly to determine if this specific feature was
used.

[How]
The command code is simplified by calling the deprecated subsystem to
list used deprecated features instead.

References #12619.
2024-11-11 10:13:25 +01:00
Michael Klishin 38bc831ad9
Alpha workflows: drop values that are no longer used 2024-11-11 01:13:32 -05:00