Commit Graph

59868 Commits

Author SHA1 Message Date
Diana Parra Corbacho 07a085365e Local shovels: optimisations 2025-08-13 12:59:40 +02:00
Diana Parra Corbacho e9d767b84b Local shovels: fix credit flow 2025-08-13 12:19:36 +02:00
Diana Parra Corbacho 0b1aefd4ae Local shovels: move unacked_message_q inside source config 2025-08-13 11:49:44 +02:00
David Ansari b6dcca0ccb
Merge pull request #14326 from lukebakken/lukebakken/make-deterministic-optional-again
Use `YRL_ERLC_OPTS` instead of `ERL_COMPILER_OPTIONS`
2025-08-13 09:16:10 +02:00
Michael Klishin 0c35f0fec3
Workflows: bump expected IBM MQ tag to 9.4.0.12 2025-08-12 20:31:14 -04:00
Michael Klishin be05f9d8b8
Merge pull request #14377 from rabbitmq/dependabot/github_actions/main/actions/checkout-5
build(deps): bump actions/checkout from 4 to 5
2025-08-12 20:00:17 -04:00
dependabot[bot] 8c53b69b1a
build(deps): bump actions/checkout from 4 to 5
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-08-12 23:05:28 +00:00
Luke Bakken a87445cda6 Use `YRL_ERLC_OPTS` instead of `ERL_COMPILER_OPTIONS`
This is a follow-up to commit 93db480bc4

`erlang.mk` supports the `YRL_ERLC_OPTS` variable to set `erlc`-specific
compiler options when processing `.yrl` and `.xrl` files. By using this
variable, it allows `make RMQ_ERLC_OPTS=` to disable the
`+deterministic` option. This allows using `c()` in the erl shell to
recompile modules on the fly when a cluster is running.

You can see that, when `make RMQ_ERLC_OPTS=` is run, these generated
files were produced with the `+deterministic` option, because their
`-file` directives use only basenames.

* `deps/rabbit/src/rabbit_amqp_sql_lexer.erl`
* `deps/rabbit/src/rabbit_amqp_sql_parser.erl`

```
-file("rabbit_amqp_sql_parser.yrl", 0).
-module(rabbit_amqp_sql_parser).
-file("rabbit_amqp_sql_parser.erl", 3).
-export([parse/1, parse_and_scan/1, format_error/1]).
-file("rabbit_amqp_sql_parser.yrl", 122).
```

This commit also ignores those two files, as they will always be
auto-generated.
2025-08-12 12:00:54 -07:00
Michael Klishin 64e5a6a7e5
Merge pull request #14373 from rabbitmq/mk-reduce-rabbitmq-plugins-log-rate
Do not log every transient (or Erlang/OTP-provided) dependency when listing plugins
2025-08-12 15:00:42 -04:00
Michael Klishin cf3fd1e7c7
Merge pull request #14359 from rabbitmq/verify-auth-plugin-is-enabled
rabbit_access_control: Check configured auth backends are enabled at boot time
2025-08-12 14:38:51 -04:00
Jean-Sébastien Pédron 23588b665a
rabbit_access_control: Check configured auth backends are enabled at boot time
[Why]
If a user configures an auth backend module, but doesn't enabled the
plugin that provides it, it will get a crash and a stacktrace when
authentication is performed. The error is not helpful to understand what
the problem is.

[How]
We add a boot step that go through the configured auth backends and
query the core of RabbitMQ and the plugins. If an auth backend is
provided by a plugin, the plugin must be enabled to consider the auth
backend to be valid.

In the end, at least one auth backend must be valid, otherwise the boot
is aborted.

If only some of the configured auth backends were filtered out, but
there are still some valid auth backends, we store the filtered list in
the application environment variable so that
authentication/authorization doesn't try to use them later.

We also report invalid auth backends in the logs:

* Info message for a single invalid auth backend:

    [info] <0.213.0> The `rabbit_auth_backend_ldap` auth backend module is configured. However, the `rabbitmq_auth_backend_ldap` plugin must be enabled in order to use this auth backend. Until then it will be skipped during authentication/authorization

* Warning message when some auth backends were filtered out:

    [warning] <0.213.0> Some configured backends were dropped because their corresponding plugins are disabled. Please look at the info messages above to learn which plugin(s) should be enabled. Here is the list of auth backends kept after filering:
    [warning] <0.213.0> [rabbit_auth_backend_internal]

* Error message when no auth backends are valid:

    [error] <0.213.0> None of the configured auth backends are usable because their corresponding plugins were not enabled. Please look at the info messages above to learn which plugin(s) should be enabled.

V2: In fact, `rabbit_plugins:is_enabled/1` indicates if a plugin is
    running, not if it is enabled... The new check runs as a boot step
    and thus is executed before plugins are started. Therefore we can't
    use this API. Instead, we use `rabbit_plugins:enabled_plugins/0'
    which lists explicitly enabled plugins. The drawback is that in the
    auth backend is enabled implicitly because it is a dependency of
    another explicitly enabled plugin, the check will still consider it
    is disabled and thus abort the boot.

Fixes #13783.
2025-08-12 18:38:28 +02:00
Jean-Sébastien Pédron a8bef770a5
rabbit_plugins: Add `which_plugin/1` to query which plugin provides a module
[Why]
This will be used in a later commit to find the auth backend plugin that
provides a configured auth backend module.

[How]
We go through the list of available plugins, regardless if they are
enabled or not, then look up the given module in the list of modules
associated with each plugin's application.
2025-08-12 18:38:28 +02:00
Jean-Sébastien Pédron 6d3d297598
rabbit_plugins: Add `list/0` to get the list of plugins
... without having to pass a plugins path.

[Why]
It's painful to have to get the plugins path, then pass it to `list/1`
every time. It's also more difficult to discover how to use
`rabbit_plugins` to get that list of plugins.
2025-08-12 18:38:28 +02:00
Michael Klishin 87c6615ba6
Update SERVER_RELEASES.md 2025-08-12 12:14:07 -04:00
Michael Klishin 97a3df5f29
Update PKG_WINDOWS.md 2025-08-12 12:13:39 -04:00
Michael Klishin 9ae9477010
Update PKG_LINUX.md 2025-08-12 12:12:46 -04:00
Michael Klishin 30d78a490b
Do not log every transient dependency when listing plugins
The code path in question is executed every time
rabbit_plugins:list/2 (e.g. rabbit_plugins:is_enabled/1)
is used, which with some distributed plugins can
happen once or several times a minute.

Given the maturity of the plugins subsystem, we
arguably can drop those messages.
2025-08-12 12:05:48 -04:00
Michal Kuratczyk c34c803754
Remove flake in prometheus_http_SUITE (#14367)
Sometimes the metrics for streams created by `stream_pub_sub_metrics`
would be returned when the next test starts, breaking the assertions.
2025-08-12 15:03:20 +02:00
Diana Parra Corbacho 8c9f79fb36 Shovel management: add local shovels 2025-08-11 21:37:05 +02:00
Diana Parra Corbacho bb5e1d95d9 Local shovels: handle credit on sender side 2025-08-11 21:37:05 +02:00
Diana Parra Corbacho 78167f0fb7 Local shovel: more tests 2025-08-11 21:37:05 +02:00
Michael Klishin 414da5d146 local_dynamic_SUITE: ignore two expected crash reports in the logs 2025-08-11 21:37:05 +02:00
Diana Parra Corbacho be061cd54b Local shovel: fix credit handling order 2025-08-11 21:37:05 +02:00
Diana Parra Corbacho a0d42ea31d Local shovels: remove unused prefetch count 2025-08-11 21:37:05 +02:00
Diana Parra Corbacho a855146b99 Local shovel: handle destination queue events 2025-08-11 21:37:05 +02:00
Michael Klishin 30f67e0f7a local_dynamic_SUITE: await credit for publishing links
This elimiantes a race condition between the destination
granting the sender link credit and the rest of what
the test does.

Note: the amqp_utils module in server core cannot be easily
moved to, say, rabbit_ct_helpers because it combines
two kinds of helpers that belong to two of our
CT helper subprojects.

So we've copied two small functions from it for
the needs of this suite.
2025-08-11 21:37:05 +02:00
Diana Parra Corbacho b5a2dac758 Local shovels: grant credit after confirming 2025-08-11 21:37:04 +02:00
Diana Parra Corbacho 2e2dc4b2dd Shovel tests: ignore nodename
CI uses a different hostname
2025-08-11 21:37:04 +02:00
Diana Parra Corbacho ae6a36a7b2 Local shovels: remove unused parameter
Fixes a dialyzer failure
2025-08-11 21:37:04 +02:00
Diana Parra Corbacho 085c0ccc70 Local shovels: remove rabbit_log and switch to LOG_ macros 2025-08-11 21:37:04 +02:00
Diana Parra Corbacho c3ec85fafb Local shovel: finish credit flow handling
This implementation intentionally ignores the
drain property of AMQP 1.0 flow control because
local shovels does not use this feature, and no
external component can "enable" it, so we can
simply ignore it.

Pair: @dcorbacho.

Co-authored-by: Diana Parra Corbacho <diana.parra-corbacho@broadcom.com>
Co-authored-by: Michael Klishin <michaelklishin@icloud.com>
2025-08-11 21:37:04 +02:00
Diana Parra Corbacho f381993db0 Local shovels: renew credit 2025-08-11 21:37:04 +02:00
Diana Parra Corbacho a112aa5214 Local shovels: set link credit 2025-08-11 21:37:04 +02:00
Diana Parra Corbacho 860eb6c499 Local shovel: fix initial delivery count and state handling 2025-08-11 21:37:04 +02:00
Diana Parra Corbacho c8127ea3ab Local shovel: Initialise delivery count for credit 2025-08-11 21:37:04 +02:00
Diana Parra Corbacho acbd4b465b Local shovel: use queue name to filter source/dest messages 2025-08-11 21:37:04 +02:00
Diana Parra Corbacho 6fce51ddff WIP 2025-08-11 21:37:04 +02:00
Diana Parra Corbacho 444b5644b2 Local shovels 2025-08-11 21:37:04 +02:00
Diana Parra Corbacho c8840785d9 AMQP10 shovel: make bare message inmutable 2025-08-11 21:37:04 +02:00
Diana Parra Corbacho ccd3bde810 amqp10_msg: type spec & code refactor 2025-08-11 21:37:04 +02:00
Diana Parra Corbacho 6e6a33ec0b Shovel: use message containers 2025-08-11 21:37:03 +02:00
Michael Klishin 5f69116a64
Merge pull request #14364 from rabbitmq/mk-bump-cuttlefish
Cuttlefish 3.5.0
2025-08-11 14:15:13 -04:00
Michael Klishin 41c25cfa04
Merge pull request #14360 from rabbitmq/fix-amqp10
Shovel: AMQP1.0 use prefetch-count as credit on delete-after
2025-08-11 13:54:52 -04:00
Michael Klishin 7413511195
Cuttlefish 3.5.0
This version forces prefixed binaries
(such as encrypted:TkQbjiVWtUJw3Ed/hkJ5JIsFIyhruKII6uKPXogfvDyMXGH1qQK3hVqshFolLN0S)
to have alphanumeric prefixes ([a-zA-Z0-9_]+).

This allows us to tell a generated password value
with a colon from an tagged binary.

If a value of, say, default_pass or ssl_options.password
cannot be parsed as a tagged value, it will be
parsed as a regular binary, because rabbit.schema
specifies multiple types as supported.

References #14233.
2025-08-11 13:25:36 -04:00
Michael Klishin 2024a4bc77
Merge pull request #14348 from udeeksha30-netizen/local_random_exchange
Add config option for enabling local_random_exchange
2025-08-11 13:23:56 -04:00
David Ansari 2f49f9da08
Permit amqp_filter_set_bug by default (#14361)
This partially reverts
https://github.com/rabbitmq/rabbitmq-server/pull/14245.
This makes 4.2 <-> 3.13 mixed version tests succeed.
We can set this flag to `denied_by_default` in 4.3.
2025-08-11 19:09:56 +02:00
udeeksha30-netizen 781c14035e Addressed requested changes 2025-08-11 09:51:20 -07:00
Diana Parra Corbacho fa66b4eb4c Shovel: AMQP1.0 use prefetch-count as credit on delete-after 2025-08-11 18:32:11 +02:00
udeeksha30-netizen 93cffe72e0 Removed extra comments 2025-08-11 08:12:33 -07:00
udeeksha30-netizen 2afbc5eeb6 Add config option for enabling local_random_exchange 2025-08-11 07:47:08 -07:00