Commit Graph

59216 Commits

Author SHA1 Message Date
Loïc Hoguin ee0c6a69b9
CQ shared store: Delete from index on remove or roll over
It was expensive to delete files because we had clean up
the index and to get the messages in the file we have to
scan it.

Instead of cleaning up the index on file delete this
commit deletes from the index as soon as possible.
There are two scenarios: messages that are removed
from the current write file, and messages that are
removed from other files. In the latter case, we
can just delete the index entry on remove. For messages
in the current write file, we want to keep the entry
in case fanout is used, because we don't want to write
the fanout message multiple times if we can avoid it.
So we keep track of removes in the current write file
and do a cleanup of these entries on file roll over.

Compared to the previous implementation we will no
longer increase the ref_count of messages that are
not in the current write file, meaning we may do more
writes in fanout scenarios. But at the same time the
file delete operation is much cheaper.

Additionally, we prioritise delete calls in rabbit_msg_store_gc.
Without that change, if the compaction was lagging behind,
we could have file deletion requests queued behind many compaction
requests, leading to many unnecessary compactions of files
that could already be deleted.
2025-05-30 13:14:16 +02:00
Michael Klishin 9f150dbb4a
Merge pull request #13980 from rabbitmq/mergify/bp/v4.1.x/pr-13977
Trigger a 4.1.x alpha release build / trigger_alpha_build (push) Waiting to run Details
Test (make) / Build and Xref (1.17, 26) (push) Waiting to run Details
Test (make) / Build and Xref (1.17, 27) (push) Waiting to run Details
Test (make) / Test (1.17, 27, khepri) (push) Waiting to run Details
Test (make) / Test (1.17, 27, mnesia) (push) Waiting to run Details
Test (make) / Test mixed clusters (1.17, 27, khepri) (push) Waiting to run Details
Test (make) / Test mixed clusters (1.17, 27, mnesia) (push) Waiting to run Details
Test (make) / Type check (1.17, 27) (push) Waiting to run Details
[skip ci] Fix plugin version warning formatting (backport #13977)
2025-05-29 18:13:03 +04:00
Michal Kuratczyk 972c4c7278 [skip ci] Fix plugin version warning formatting
This doesn't fail the test, but shows up
as a end_per_testcase failure.

https://github.com/rabbitmq/rabbitmq-server/actions/runs/15322789846/job/43110071803?pr=13959
(cherry picked from commit b9c38560c1)
2025-05-29 14:09:37 +00:00
Michael Klishin 906e554584
Merge pull request #13975 from rabbitmq/mergify/bp/v4.1.x/pr-13972
Trigger a 4.1.x alpha release build / trigger_alpha_build (push) Waiting to run Details
Test (make) / Build and Xref (1.17, 26) (push) Waiting to run Details
Test (make) / Build and Xref (1.17, 27) (push) Waiting to run Details
Test (make) / Test (1.17, 27, khepri) (push) Waiting to run Details
Test (make) / Test (1.17, 27, mnesia) (push) Waiting to run Details
Test (make) / Test mixed clusters (1.17, 27, khepri) (push) Waiting to run Details
Test (make) / Test mixed clusters (1.17, 27, mnesia) (push) Waiting to run Details
Test (make) / Type check (1.17, 27) (push) Waiting to run Details
Test Management UI with Selenium / selenium (chrome, 1.17.3, 27.3) (push) Has been cancelled Details
Clear management auth storage when redirecting to login (backport #13972)
2025-05-29 09:59:03 +04:00
Michael Klishin e880904124
Merge pull request #13974 from rabbitmq/mergify/bp/v4.1.x/pr-13963
Selenium: minor refactoring (backport #13963)
2025-05-29 09:58:23 +04:00
Michael Davis ed2e1a11a4 Clear management auth storage when redirecting to login
This branch redirects the client to the login page when the cookie
expires. To complete the logout process we should also clear any auth
data stored in local storage: local storage has no built-in expiration
mechanism.

To test this locally you can use `make run-broker`, set the session
timeout to one minute for quick testing:

    application:set_env(rabbitmq_management, login_session_timeout, 1)

go to the management page (`http://localhost:15672/#/`), login with
default credentials and wait a minute. After this change the local
storage only contains info like `rabbitmq.vhost` and `rabbitmq.version`.

(cherry picked from commit 2a1b65df17)
2025-05-29 05:58:10 +00:00
Marcial Rosales c36a4668b6 Relocate temp folder from /tmp to /var
(cherry picked from commit 94cba43e75)
2025-05-28 18:14:42 +00:00
Michael Klishin 938a2f5f96
Merge pull request #13968 from rabbitmq/mergify/bp/v4.1.x/pr-13950
Trigger a 4.1.x alpha release build / trigger_alpha_build (push) Waiting to run Details
Test (make) / Build and Xref (1.17, 26) (push) Waiting to run Details
Test (make) / Build and Xref (1.17, 27) (push) Waiting to run Details
Test (make) / Test (1.17, 27, khepri) (push) Waiting to run Details
Test (make) / Test (1.17, 27, mnesia) (push) Waiting to run Details
Test (make) / Test mixed clusters (1.17, 27, khepri) (push) Waiting to run Details
Test (make) / Test mixed clusters (1.17, 27, mnesia) (push) Waiting to run Details
Test (make) / Type check (1.17, 27) (push) Waiting to run Details
Test Management UI with Selenium / selenium (chrome, 1.17.3, 27.3) (push) Waiting to run Details
Selenium: Verify consumer state when using SAC (backport #13950)
2025-05-28 15:01:30 +04:00
Marcial Rosales ce3c5b9551 Fix location of queue consumer stats 2025-05-28 11:15:20 +02:00
Marcial Rosales 13f7438fa1 Fix credentials used to create vhost
(cherry picked from commit 09fc5357a7)
2025-05-28 07:53:11 +00:00
Marcial Rosales c9a5ea5c90 Stop page refresh
To prevent state element exception

(cherry picked from commit a92242918b)
2025-05-28 07:53:11 +00:00
Marcial Rosales 4d37daa303 Fix typo
(cherry picked from commit 98061c8e96)
2025-05-28 07:53:11 +00:00
Marcial Rosales 3ebbad1dce Test SAC with consumres w/o priorities
(cherry picked from commit 23eb9854ee)
2025-05-28 07:53:11 +00:00
Marcial Rosales 148debc550 Use different way to parse tables for consuers
(cherry picked from commit 870c66734b)
2025-05-28 07:53:10 +00:00
Marcial Rosales b48600251f Apply patch that addresses the issue with SAC
And improve how to parse a html table to
extract its rows

(cherry picked from commit 8960d19492)
2025-05-28 07:53:10 +00:00
Marcial Rosales 93e5aefa9a Reproduce issue
(cherry picked from commit d2767983dc)
2025-05-28 07:53:10 +00:00
Marcial Rosales 9464a202fd Add amqplib
Required to subscribe with a priority in the arguments
which is only possible in AMQP 0.9.1

(cherry picked from commit 4fdbcb33e1)
2025-05-28 07:53:10 +00:00
Marcial Rosales 19c45cb08a Remove log statements
(cherry picked from commit 586a946213)
2025-05-28 07:53:09 +00:00
Marcial Rosales 3a4ea0ae21 Selenium test with SAC
(cherry picked from commit 70ec8dffdd)

# Conflicts:
#	deps/rabbitmq_management/priv/www/js/tmpl/quorum-queue-stats.ejs
2025-05-28 07:53:09 +00:00
Michael Klishin 5d2a54d92b
Merge pull request #13945 from rabbitmq/dependabot/maven/deps/rabbitmq_auth_backend_http/examples/rabbitmq_auth_backend_spring_boot/v4.1.x/prod-deps-801db9dc1e
Test (make) / Build and Xref (1.17, 26) (push) Has been cancelled Details
Test (make) / Build and Xref (1.17, 27) (push) Has been cancelled Details
Test (make) / Test (1.17, 27, khepri) (push) Has been cancelled Details
Test (make) / Test (1.17, 27, mnesia) (push) Has been cancelled Details
Test (make) / Test mixed clusters (1.17, 27, khepri) (push) Has been cancelled Details
Test (make) / Test mixed clusters (1.17, 27, mnesia) (push) Has been cancelled Details
Test (make) / Type check (1.17, 27) (push) Has been cancelled Details
[skip ci] Bump the prod-deps group across 2 directories with 1 update
2025-05-25 13:29:39 +04:00
dependabot[bot] 5a9f2c0251
[skip ci] Bump the prod-deps group across 2 directories with 1 update
Bumps the prod-deps group with 1 update in the /deps/rabbitmq_auth_backend_http/examples/rabbitmq_auth_backend_spring_boot directory: [org.springframework.boot:spring-boot-starter-parent](https://github.com/spring-projects/spring-boot).
Bumps the prod-deps group with 1 update in the /deps/rabbitmq_auth_backend_http/examples/rabbitmq_auth_backend_spring_boot_kotlin directory: [org.springframework.boot:spring-boot-starter-parent](https://github.com/spring-projects/spring-boot).


Updates `org.springframework.boot:spring-boot-starter-parent` from 3.4.5 to 3.5.0
- [Release notes](https://github.com/spring-projects/spring-boot/releases)
- [Commits](https://github.com/spring-projects/spring-boot/compare/v3.4.5...v3.5.0)

Updates `org.springframework.boot:spring-boot-starter-parent` from 3.4.5 to 3.5.0
- [Release notes](https://github.com/spring-projects/spring-boot/releases)
- [Commits](https://github.com/spring-projects/spring-boot/compare/v3.4.5...v3.5.0)

---
updated-dependencies:
- dependency-name: org.springframework.boot:spring-boot-starter-parent
  dependency-version: 3.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-deps
- dependency-name: org.springframework.boot:spring-boot-starter-parent
  dependency-version: 3.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-05-24 19:00:32 +00:00
Iliia Khaprov - VMware by Broadcom eb7a6c70a7
Merge pull request #13938 from rabbitmq/mergify/bp/v4.1.x/pr-13548
Trigger a 4.1.x alpha release build / trigger_alpha_build (push) Has been cancelled Details
Test (make) / Build and Xref (1.17, 26) (push) Has been cancelled Details
Test (make) / Build and Xref (1.17, 27) (push) Has been cancelled Details
Test (make) / Test (1.17, 27, khepri) (push) Has been cancelled Details
Test (make) / Test (1.17, 27, mnesia) (push) Has been cancelled Details
Test (make) / Test mixed clusters (1.17, 27, khepri) (push) Has been cancelled Details
Test (make) / Test mixed clusters (1.17, 27, mnesia) (push) Has been cancelled Details
Test (make) / Type check (1.17, 27) (push) Has been cancelled Details
For 4.1.x, by @aaron-seo: introduce a command that would force QQs to take a checkpoint and truncate its segments (backport #13548)
2025-05-23 10:24:01 +02:00
Michael Klishin 4574b3822b
Resolve a conflict, adapt to v4.1.x #13548 #13938 2025-05-23 01:10:34 -04:00
Diana Parra Corbacho 79aa0bba47 Force checkpoint in all members
(cherry picked from commit 0c2b6a1cb3)
2025-05-23 03:12:37 +00:00
Michael Klishin 901343d78f quorum_queue_SUITE: keep Raft state logging in force_checkpoint_on_queue
(cherry picked from commit 7d3292cedd)
2025-05-23 03:12:36 +00:00
Michael Klishin 1a998f8887 Quorum queue machine: do not publish certain state records
we can use log state in tests.

(cherry picked from commit d17b085653)
2025-05-23 03:12:36 +00:00
Michael Klishin e7e1fb76a4 Update a #13175 test to not use private Ra machine state
(cherry picked from commit e49acf956c806849068f543f282683978ca5a385)
(cherry picked from commit d54fee2e7a)
2025-05-23 03:12:36 +00:00
Aaron Seo 9bc9276e18 Add timeout to rpc call for force_checkpoint
(cherry picked from commit 4439150e50)
(cherry picked from commit 6a78e9f7ba)
2025-05-23 03:12:36 +00:00
Aaron Seo 66d4245ddf Fix force_checkpoint tests and CLI command
(cherry picked from commit 12bf3e094e)

Conflicts:
	deps/rabbit/src/rabbit_quorum_queue.erl
(cherry picked from commit fa310864d7)
2025-05-23 03:12:35 +00:00
Aaron Seo d2dcc61b96 Add force checkpoint functions for quorum queues and command line tool
(cherry picked from commit b54ab1d5e5)
(cherry picked from commit 0d3dfd9695)

# Conflicts:
#	deps/rabbit/src/rabbit_quorum_queue.erl
2025-05-23 03:12:35 +00:00
Michael Klishin 413ad94ccc
Merge pull request #13937 from rabbitmq/mergify/bp/v4.1.x/pr-13932
Trigger a 4.1.x alpha release build / trigger_alpha_build (push) Waiting to run Details
Test (make) / Build and Xref (1.17, 26) (push) Waiting to run Details
Test (make) / Build and Xref (1.17, 27) (push) Waiting to run Details
Test (make) / Test (1.17, 27, khepri) (push) Waiting to run Details
Test (make) / Test (1.17, 27, mnesia) (push) Waiting to run Details
Test (make) / Test mixed clusters (1.17, 27, khepri) (push) Waiting to run Details
Test (make) / Test mixed clusters (1.17, 27, mnesia) (push) Waiting to run Details
Test (make) / Type check (1.17, 27) (push) Waiting to run Details
Test Management UI with Selenium / selenium (chrome, 1.17.3, 27.3) (push) Has been cancelled Details
Add tests for rabbit_classic_queue_index_v2:bounds/2 (backport #13932)
2025-05-23 06:08:52 +04:00
Michael Klishin 5a2081e0b8
Merge pull request #13935 from rabbitmq/mergify/bp/v4.1.x/pr-13931
Fix selenium issue closing popup warning dialog since recent upgrade  (backport #13931)
2025-05-23 06:08:27 +04:00
Péter Gömöri 383818e27f Fix comment about CQ v1->v2 index recovery
(cherry picked from commit ec455d5cff)
2025-05-23 02:06:12 +00:00
Péter Gömöri ea68a41ba5 Add tests for rabbit_classic_queue_index_v2:bounds/2
(cherry picked from commit 55e3c458c2)
2025-05-23 02:06:12 +00:00
Marcial Rosales 94e4aa9412 Fix issue related to popup warning
And in particular locating the span#close button

(cherry picked from commit 27b3e21554)
2025-05-22 23:23:47 +00:00
Michael Klishin 4553f8ae2f
Merge pull request #13927 from rabbitmq/mergify/bp/v4.1.x/pr-13925
Test Management UI with Selenium / selenium (chrome, 1.17.3, 27.3) (push) Has been cancelled Details
Bump up chrome driver (backport #13925)
2025-05-21 21:50:27 +04:00
Marcial Rosales 4054f6572e Briefly explain how to start a second rabbitmq server
interactively

(cherry picked from commit a028db8156)
2025-05-21 14:22:42 +00:00
Marcial Rosales c9da9e15ef Eliminate flake around listing live amqp connections
(cherry picked from commit 44dd282ed4)
2025-05-21 14:22:41 +00:00
Marcial Rosales d84424f6aa Bump up chrome driver
This is needed when running tests interactively.
The OS updates the local chrome binary and this
node.js library has to be upgraded too.

(cherry picked from commit 6578c83a0e)
2025-05-21 14:22:41 +00:00
Michael Klishin c1086b98ff
Merge pull request #13922 from rabbitmq/mergify/bp/v4.1.x/pr-13920
Trigger a 4.1.x alpha release build / trigger_alpha_build (push) Has been cancelled Details
Test (make) / Build and Xref (1.17, 26) (push) Has been cancelled Details
Test (make) / Build and Xref (1.17, 27) (push) Has been cancelled Details
Test (make) / Test (1.17, 27, khepri) (push) Has been cancelled Details
Test (make) / Test (1.17, 27, mnesia) (push) Has been cancelled Details
Test (make) / Test mixed clusters (1.17, 27, khepri) (push) Has been cancelled Details
Test (make) / Test mixed clusters (1.17, 27, mnesia) (push) Has been cancelled Details
Test (make) / Type check (1.17, 27) (push) Has been cancelled Details
Add a proptest checking `ra_indexes` indices (backport #13920)
2025-05-21 05:20:29 +04:00
Michael Davis fafb51f9c2 Add a proptest checking `ra_indexes` indices
This is mostly the same as the `messages_total` property test but checks
that the Raft indexes in `ra_indexes` are the set of the indexes checked
out by all consumers union any indexes in the `returns` queue. This is
the intended state of `ra_indexes` and failing this condition could
cause bugs that would prevent snapshotting.

(cherry picked from commit 01b4051b03)
2025-05-20 20:56:36 +00:00
Michael Klishin 39060ca57b
Merge pull request #13917 from rabbitmq/mergify/bp/v4.1.x/pr-13916
Trigger a 4.1.x alpha release build / trigger_alpha_build (push) Has been cancelled Details
Test (make) / Build and Xref (1.17, 26) (push) Has been cancelled Details
Test (make) / Build and Xref (1.17, 27) (push) Has been cancelled Details
Test (make) / Test (1.17, 27, khepri) (push) Has been cancelled Details
Test (make) / Test (1.17, 27, mnesia) (push) Has been cancelled Details
Test (make) / Test mixed clusters (1.17, 27, khepri) (push) Has been cancelled Details
Test (make) / Test mixed clusters (1.17, 27, mnesia) (push) Has been cancelled Details
Test (make) / Type check (1.17, 27) (push) Has been cancelled Details
OTP28: re:split change; street-address macro (backport #13916)
2025-05-19 17:33:41 +04:00
Michal Kuratczyk 22f1374f21 Remove rabbitmq_mqtt from DEPS (it's a TEST_DEP)
(cherry picked from commit 9fefcc4827)
2025-05-19 12:19:45 +00:00
Michal Kuratczyk b07dcf8bf5 OTP28: re:split change; street-address macro
https://github.com/erlang/otp/issues/9739

In OTP28+, splitting an empty string returns an empty list, not an empty
string (the input).

Additionally `street-address` macro was removed in OTP28 - replace with
the value it used to be.

Lastly, rabbitmq_auth_backend_oauth2 has an MQTT test, so add
rabbitmq_mqtt to TEST_DEPS

(cherry picked from commit 637a2bc8cc)
2025-05-19 12:19:45 +00:00
Michael Klishin 2db9a615ea
Merge pull request #13915 from rabbitmq/mergify/bp/v4.1.x/pr-13913
Trigger a 4.1.x alpha release build / trigger_alpha_build (push) Waiting to run Details
Test (make) / Build and Xref (1.17, 26) (push) Waiting to run Details
Test (make) / Build and Xref (1.17, 27) (push) Waiting to run Details
Test (make) / Test (1.17, 27, khepri) (push) Waiting to run Details
Test (make) / Test (1.17, 27, mnesia) (push) Waiting to run Details
Test (make) / Test mixed clusters (1.17, 27, khepri) (push) Waiting to run Details
Test (make) / Test mixed clusters (1.17, 27, mnesia) (push) Waiting to run Details
Test (make) / Type check (1.17, 27) (push) Waiting to run Details
Bump Ra to 2.16.9 (backport #13913)
2025-05-19 09:35:52 +04:00
Michael Klishin 7feb4463ad Bump Ra to 2.16.9
(cherry picked from commit c2d6dee8e8)
2025-05-19 04:12:24 +00:00
Michael Klishin be1b83f05d
Merge pull request #13914 from rabbitmq/mergify/bp/v4.1.x/pr-13912
Bump Osiris to 1.8.8 (backport #13912)
2025-05-19 08:11:36 +04:00
Michael Klishin 876b17d1c8 Bump Osiris to 1.8.8
(cherry picked from commit 94575bc76d)
2025-05-19 03:30:59 +00:00
Michael Klishin 25f81e4081
Merge pull request #13902 from rabbitmq/dependabot/maven/deps/rabbitmq_auth_backend_http/examples/rabbitmq_auth_backend_spring_boot_kotlin/v4.1.x/dev-deps-5ac378ddf0
[skip ci] Bump the dev-deps group across 1 directory with 2 updates
2025-05-19 06:22:20 +04:00
dependabot[bot] 88015c037a
[skip ci] Bump the dev-deps group across 1 directory with 2 updates
Bumps the dev-deps group with 2 updates in the /deps/rabbitmq_auth_backend_http/examples/rabbitmq_auth_backend_spring_boot_kotlin directory: [org.jetbrains.kotlin:kotlin-test](https://github.com/JetBrains/kotlin) and org.jetbrains.kotlin:kotlin-maven-allopen.


Updates `org.jetbrains.kotlin:kotlin-test` from 2.1.20 to 2.1.21
- [Release notes](https://github.com/JetBrains/kotlin/releases)
- [Changelog](https://github.com/JetBrains/kotlin/blob/master/ChangeLog.md)
- [Commits](https://github.com/JetBrains/kotlin/compare/v2.1.20...v2.1.21)

Updates `org.jetbrains.kotlin:kotlin-maven-allopen` from 2.1.20 to 2.1.21

---
updated-dependencies:
- dependency-name: org.jetbrains.kotlin:kotlin-test
  dependency-version: 2.1.21
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: dev-deps
- dependency-name: org.jetbrains.kotlin:kotlin-maven-allopen
  dependency-version: 2.1.21
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-05-17 18:15:04 +00:00