Commit Graph

834 Commits

Author SHA1 Message Date
Michael Klishin 28cc12177e
Merge pull request #12655 from rabbitmq/mk-management-ui-selenium-suite-workflow-update
Selenium management UI suite workflow: updates for 2024
2024-11-04 15:07:54 -05:00
Michael Klishin 003f7a9f93 Selenium management UI suite workflow: updates for 2024
1. Use Elixir 1.17.x
2. Only run this suite when the management plugin
   or web_dispatch source code changes
3. Target relevant branches: main and v4.0.x
2024-11-04 14:14:26 -05:00
Michael Klishin 3ca95d103b 'deps/rabbitmq_management/selenium/**' is relevant for this workflow 2024-11-04 14:03:10 -05:00
Michael Klishin 376cff0676 Selenium suite workflow: updates for 2024
1. Use Elixir 1.17.x
2. Run only when several relevant code paths change
3. Do not run when unrelated core server tests change
2024-11-04 13:55:32 -05:00
Michael Klishin af0d8206c8 Actions/alpha build: cosmetics 2024-11-03 23:03:33 -05:00
Michael Klishin 15b924b05e Revert "Actions, alpha build: try passing in a different prerelease_identifier"
This reverts commit de0d8cf70b.
2024-11-03 22:56:47 -05:00
Michael Klishin de0d8cf70b Actions, alpha build: try passing in a different prerelease_identifier 2024-11-03 22:56:22 -05:00
Michael Klishin a1e78dc2df Actions: trigger alpha build workflow run when workflow itself changes 2024-11-03 22:53:04 -05:00
Michael Klishin 4fb989c99d Actions: try a using short commit SHA for alpha identifier 2024-11-03 22:52:07 -05:00
Michael Klishin b063a9910e Use a known repository_dispatch event type 2024-11-03 22:45:42 -05:00
Michael Klishin 2278e4cee8 New workflow for triggering alpha releases
in rabbitmq/server-packages, an Actions-only
repo dedicated to open source RabbitMQ release
automation.
2024-11-03 22:23:35 -05:00
Michael Klishin 92a40463f8
Update SECURITY.md 2024-11-03 21:52:30 -05:00
Michael Klishin 030380a4c1
Merge pull request #12629 from rabbitmq/mk-run-aws-peer-discovery-workflow-on-a-schedule
Reduce AWS peer discovery workflow run rate
2024-10-31 14:25:50 -04:00
Michael Klishin df0b76705b Actions deps: manually apply #12630 #12631 2024-10-31 14:22:59 -04:00
Michael Klishin 893a349f44 Reduce AWS peer discovery workflow run rate
By running it

 * On push, when relevant code paths change
 * Every Monday morning

The peer discovery subsystem does not change
particularly often, and this plugin in particular
does not. Nonetheless, we currently run it for
every push unconditionally.
2024-10-31 13:50:19 -04:00
Loïc Hoguin e89e1fa2e8
Make CI: Enable khepri mixed clusters testing 2024-10-25 14:41:51 +02:00
Loïc Hoguin f68fc8bb94
Make CI: Add mixed version testing
This is enabled on main and for pull requests. Bazel remains
used in previous branches.
2024-10-25 13:50:05 +02:00
Marcial Rosales aea7b82bd6 Fix docker image to 9.4.0.5 2024-10-23 12:54:24 +02:00
Marcial Rosales 3dc9b1f441 Correct tag version 2024-10-23 12:14:22 +02:00
Marcial Rosales b8a3abaa0d Correct version 2024-10-23 12:00:52 +02:00
Marcial Rosales e1132edeec Bump up to 9.4.0.5 2024-10-23 11:55:15 +02:00
Loïc Hoguin 05b75bd419
Make CI: Always upload log_private CT logs
All test runs produce artifacts of less than 12MB in size which
is acceptable as it is fast to produce, upload and download.
Most test runs are actually below 1MB.
2024-10-17 11:52:26 +02:00
Loïc Hoguin 469c3a0791
Make CI: Check that CI knows about all CT_SUITES in CI
Instead of every time we run Make for these applications.

This means that during development we are free to modify
these values or create new test suites without having to
worry about the check. If we forget to then add the test
suites in PARALLEL_CT the workflow will tell us.
2024-10-17 10:52:28 +02:00
Michael Klishin 48acb6aec5 Discussion template tweaks 2024-10-16 22:16:31 -04:00
David Ansari 8c0cd1b78c Bump dotnet
This commit fixes the CI error on `main` branch where
amqp_system_SUITE failed with the following error:
```
Process terminated. Couldn't find a valid ICU package installed on the system. Set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support.
   at System.Environment.FailFast(System.String)
   at System.Globalization.GlobalizationMode.GetGlobalizationInvariantMode()
   at System.Globalization.GlobalizationMode..cctor()
   at System.Globalization.CultureData.CreateCultureWithInvariantData()
   at System.Globalization.CultureData.get_Invariant()
   at System.Globalization.CultureInfo..cctor()
   at System.String.ToLowerInvariant()
   at Microsoft.DotNet.PlatformAbstractions.RuntimeEnvironment.GetArch()
   at Microsoft.DotNet.PlatformAbstractions.RuntimeEnvironment..cctor()
   at Microsoft.DotNet.PlatformAbstractions.RuntimeEnvironment.GetRuntimeIdentifier()
   at Microsoft.DotNet.Cli.MulticoreJitProfilePathCalculator.CalculateProfileRootPath()
   at Microsoft.DotNet.Cli.MulticoreJitActivator.StartCliProfileOptimization()
   at Microsoft.DotNet.Cli.MulticoreJitActivator.TryActivateMulticoreJit()
   at Microsoft.DotNet.Cli.Program.Main(System.String[])

Exit code: 134 (pid <0.1533.0>)
```
2024-10-16 16:35:37 +02:00
Michael Klishin 507b07ebe5
Actions: use sudo with 'service' 2024-10-14 14:06:31 -04:00
Michael Klishin d851982872
Actions: disable AppArmor entirely
(cherry picked from commit 92a4562c10390fbb9d7a0eb907a693ad94b810df)
2024-10-14 13:31:10 -04:00
Michael Klishin ba8d7a82dc
Actions: try disabling AppArmor 2024-10-14 12:59:09 -04:00
Loïc Hoguin 299373fa17
Make CI: Temporarily disable tests with OTP-27
There are two known OTP-27 bugs making tests fail.
Until they are fixed (OTP-27.1.2 most likely) we
disable OTP-27 for tests.
2024-10-09 13:55:37 +02:00
Marcial Rosales d98eb17c2c Move also selenium authz tests to Makefile 2024-10-08 08:17:48 +02:00
Marcial Rosales e1e101db9b Fix issue with docker image name 2024-10-08 08:17:48 +02:00
Marcial Rosales 21a4a250d5 Run selenium job with Makefile 2024-10-08 08:17:48 +02:00
Arnaud Cogoluègnes ac15ee3c19
Set broker version to 4.1.0-alpha.1 in dev Docker image 2024-10-07 16:43:06 +02:00
Michael Klishin 0b2d4d78bb Actions: align google-github-actions/auth versions 2024-10-02 16:06:51 -04:00
dependabot[bot] a20c1ff42c
build(deps): bump google-github-actions/auth from 2.1.5 to 2.1.6
Bumps [google-github-actions/auth](https://github.com/google-github-actions/auth) from 2.1.5 to 2.1.6.
- [Release notes](https://github.com/google-github-actions/auth/releases)
- [Changelog](https://github.com/google-github-actions/auth/blob/main/CHANGELOG.md)
- [Commits](https://github.com/google-github-actions/auth/compare/v2.1.5...v2.1.6)

---
updated-dependencies:
- dependency-name: google-github-actions/auth
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-10-02 18:28:46 +00:00
Luke Bakken ae8a35554b
Use `<details>` tag when collecting text 2024-10-02 08:22:44 -07:00
Loïc Hoguin 994abc1a8c
Bazel CI: Fix workflow templates 2024-09-30 12:35:44 +02:00
Loïc Hoguin aee0cd0079
make & make CI: Small cleanups 2024-09-30 12:35:43 +02:00
Loïc Hoguin 4c80dde233
make CI: Use strings instead of numbers for OTP/Ex versions
As recommended by erlef/setup-beam.
2024-09-30 12:35:43 +02:00
Loïc Hoguin ae984cc364
make: Set CT_LOGS_DIR to top-level logs/ directory
All CT logs will now be under <toplevel>/logs. An improved
test workflow would be to always keep the logs/all_runs.html
page open in the browser and refresh it whenever tests are
run in any of the rabbit applications.
2024-09-30 12:35:43 +02:00
Loïc Hoguin a8df6f32fb
make CI: Set a correct broker version for CLI tests 2024-09-30 12:35:43 +02:00
Loïc Hoguin beaa476aa1
make CI: Always fetch tags
The fetch-tags option of actions/checkout@v4 does not work
as intended so they are fetched manually instead.
2024-09-30 12:35:43 +02:00
Loïc Hoguin 8669b24c07
make CI: Run rabbitmq_cli tests 2024-09-30 12:35:43 +02:00
Loïc Hoguin 55f0559152
make CI: Disable Elixir problem matchers in tests
They are still enabled in the build job.
2024-09-30 12:35:43 +02:00
Loïc Hoguin 7fca6e955c
make CI: Don't warn about missing CT log files
This can happen if the plugin isn't using CT, or if running
Dialyzer, for example.
2024-09-30 12:35:43 +02:00
Loïc Hoguin b8723d6cd2
Fix some OTP-27 Dialyzer errors in rabbitmq_peer_discovery_etcd
And disable those checks in CI for now. We are getting errors
because of the eetcd dependency and we can't upgrade at this
time (see comment in the commit).
2024-09-30 12:35:43 +02:00
Loïc Hoguin 97363dfeae
make CI: Run most rabbitmq_peer_discovery_aws tests
The integration_SUITE has to run in a separate workflow due
to needing secrets.
2024-09-30 12:35:42 +02:00
Loïc Hoguin dad0bfcca9
make CI: Run rabbitmq_aws eunit tests 2024-09-30 12:35:42 +02:00
Loïc Hoguin 9b2d2c1295
make CI: Run Dialyzer 2024-09-30 12:35:42 +02:00
Loïc Hoguin d9770fb18c
make CI: Add xref job 2024-09-30 12:35:42 +02:00
Loïc Hoguin b5011f058f
make CI: Simplify workflows 2024-09-30 12:35:41 +02:00
Loïc Hoguin 5327ce3e2f
make CI: Simplify test-rabbitmq-mqtt 2024-09-30 12:35:41 +02:00
Loïc Hoguin f4a24c7f16
make: Run rabbitmq_mqtt tests via parallel-ct 2024-09-30 12:35:41 +02:00
Loïc Hoguin f4f375c6a9
Use Make in CI
This is a proof of concept that mostly works but is missing
some tests, such as rabbitmq_mqtt or rabbitmq_cli. It also
doesn't apply to mixed version testing yet.
2024-09-30 12:35:41 +02:00
Michael Davis 324288ad6d
Re-enable Khepri mixed versions testing 2024-09-26 13:10:49 -04:00
Michael Klishin 212543a1b1 Discussion templates: wording 2024-09-25 19:00:31 -04:00
Michael Klishin 0cbba950a0 Update discussion templates for 4.0.x GA 2024-09-25 18:59:13 -04:00
Loïc Hoguin f5979f1dd0
CI: Enable test.yaml on v4.0.x 2024-09-25 12:36:19 +02:00
dependabot[bot] b365646657
build(deps): bump peter-evans/create-pull-request from 7.0.3 to 7.0.5
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 7.0.3 to 7.0.5.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v7.0.3...v7.0.5)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-18 18:56:22 +00:00
dependabot[bot] 35a1a0e9d1
build(deps): bump peter-evans/create-pull-request from 7.0.2 to 7.0.3
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 7.0.2 to 7.0.3.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v7.0.2...v7.0.3)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-16 18:34:48 +00:00
Michael Klishin 35fb30f331 One more Discussions template update 2024-09-12 20:52:22 -04:00
Michael Klishin cbdf16448b Cosmetic discussion template update 2024-09-12 20:49:07 -04:00
dependabot[bot] 8de2f17191
build(deps): bump peter-evans/create-pull-request from 7.0.1 to 7.0.2
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 7.0.1 to 7.0.2.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v7.0.1...v7.0.2)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-12 18:21:38 +00:00
Michael Klishin 728c76af34 Discussion template updates 2024-09-11 19:48:12 -04:00
Michael Klishin f5a026c69a Commit a missing discussion category template for Other 2024-09-11 19:48:01 -04:00
Michael Klishin 27545d635c Discussion template updates 2024-09-11 19:46:22 -04:00
Michael Klishin 37d9a9b334 Discussion template for Ideas 2024-09-11 19:40:47 -04:00
Michael Klishin 4bedbbd08e Discussion template WIP 2024-09-11 19:34:22 -04:00
Michael Klishin 5c3c4f36f7 Discussion template WIP 2024-09-11 19:17:55 -04:00
Michael Klishin 5ddf205d69 Discussion template WIP 2024-09-11 19:16:08 -04:00
Michael Klishin a91f40d25b Discussion template WIP 2024-09-11 19:15:26 -04:00
Michael Klishin 396550e86d Discussion template WIP 2024-09-11 19:14:06 -04:00
Michael Klishin 07f0daa567 Discussion template WIP 2024-09-11 19:13:31 -04:00
Michael Klishin 86ee8147d0 Discussion template WIP 2024-09-11 19:12:12 -04:00
Michael Klishin 4254c1bc9b Discussion template WIP 2024-09-11 19:11:40 -04:00
Michael Klishin c45e736a8b Discussion template WIP 2024-09-11 18:58:07 -04:00
Michael Klishin bd475e4c8a Discussion forms WIP 2024-09-11 18:53:55 -04:00
Michael Klishin 5aa62710a3 Discussion form for Questions: WIP 2024-09-11 18:51:33 -04:00
Michael Klishin a02db1eaab Try a very minimalistic discussion form 2024-09-11 18:49:50 -04:00
Michael Klishin 40ce44f83b Try a discussion template [1]
Discussions without any relevant details remain
a fairly common problem. Our small team cannot
afford to justify why we need e.g. RabbitMQ
version information and deployment details
every single time.

If someone is looking for free help from the core
team, they must provide essential details to
save everyone some time, or they can go seek
RabbitMQ help elsewhere.

1. https://docs.github.com/en/discussions/managing-discussions-for-your-community/syntax-for-discussion-category-forms
2024-09-11 18:24:56 -04:00
dependabot[bot] 99310f7478
build(deps): bump peter-evans/create-pull-request from 7.0.0 to 7.0.1
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 7.0.0 to 7.0.1.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v7.0.0...v7.0.1)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-05 18:47:11 +00:00
Michael Klishin 654fb75821
Merge pull request #12198 from rabbitmq/move-selenium-to-root
Move selenium to root
2024-09-04 11:53:55 -04:00
Michael Klishin 7323abc946
Merge pull request #12165 from rabbitmq/run-selenium-on-cluster
Run Selenium suites against a cluster of nodes
2024-09-04 11:14:03 -04:00
Marcial Rosales a1206dc801 Move selenium to the root of the repo 2024-09-04 14:59:58 +01:00
Marcial Rosales 77b63d6799 Run selenium test suite against 3-node cluster 2024-09-04 10:06:54 +01:00
dependabot[bot] f6846e1d01
Bump peter-evans/create-pull-request from 6.1.0 to 7.0.0
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6.1.0 to 7.0.0.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v6.1.0...v7.0.0)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-03 18:24:43 +00:00
Michal Kuratczyk 301424235c
Update .NET to 8.0 2024-08-30 08:42:53 +02:00
Michael Klishin 733ec1ec23 Sync google-github-actions/auth version in workflow files
(cherry picked from commit a627c2c043)
2024-08-21 15:17:56 -04:00
dependabot[bot] 11bad7b7c5
Bump google-github-actions/auth from 2.1.4 to 2.1.5
Bumps [google-github-actions/auth](https://github.com/google-github-actions/auth) from 2.1.4 to 2.1.5.
- [Release notes](https://github.com/google-github-actions/auth/releases)
- [Changelog](https://github.com/google-github-actions/auth/blob/main/CHANGELOG.md)
- [Commits](https://github.com/google-github-actions/auth/compare/v2.1.4...v2.1.5)

---
updated-dependencies:
- dependency-name: google-github-actions/auth
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-08-21 18:34:42 +00:00
Michael Klishin f0f2ed44df gmake .github/workflows/test.yaml, gmake .github/workflows/test-mixed-versions.yaml 2024-08-08 01:42:48 -04:00
Michael Klishin 7634ce9a87 Update workflows 2024-08-08 01:38:09 -04:00
Michael Klishin bec3dcb111 Workflows: update google-github-actions/auth version in the templates 2024-08-06 16:38:08 -04:00
dependabot[bot] 93059acf32
Bump google-github-actions/auth from 2.1.3 to 2.1.4
Bumps [google-github-actions/auth](https://github.com/google-github-actions/auth) from 2.1.3 to 2.1.4.
- [Release notes](https://github.com/google-github-actions/auth/releases)
- [Changelog](https://github.com/google-github-actions/auth/blob/main/CHANGELOG.md)
- [Commits](https://github.com/google-github-actions/auth/compare/v2.1.3...v2.1.4)

---
updated-dependencies:
- dependency-name: google-github-actions/auth
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-08-06 18:22:09 +00:00
Michael Klishin 40903581ef Bump version to 4.0 in a few places 2024-07-19 01:21:30 -04:00
Michael Klishin e6da989e00 Mergify: re-introduce multi-branch backports
to both v4.0.x and v3.13.x at the moment.
2024-07-19 01:04:04 -04:00
Michael Klishin 6eebb20581 gazelle-scheduled.yaml: Drop 3.11.x and 3.10.x, add 3.13.x and 4.0.x 2024-07-11 22:13:38 -04:00
Michael Klishin 4f4e62cf1a dependabot.yaml: drop 3.11.x, add 3.13.x and 4.0.x 2024-07-11 22:11:52 -04:00
Michael Klishin 599727ab22 Revisit Mergify settings
* Add v4.0.x
 * Drop more complex workflows/backport rules we
   do not really use in practice
2024-07-11 10:01:19 -04:00
Michael Davis e08310ab13
CI: Turn off mixed version tests running Khepri
Khepri is experimental and not guaranteed to be upgradable. We should
re-enable mixed version testing as Khepri stabilizes but for now we
should turn it off.

This saves some CI time and avoids some failures that will be introduced
when we upgrade Khepri to v0.14.0.
2024-07-10 13:45:09 -04:00
Michal Kuratczyk e5ca8bd6b7
Rename image to avoid conflicts
Right now when we push the ARM64 image, it overwrites the x86 image.
2024-07-08 12:31:34 +02:00
Michal Kuratczyk d2b49454f6
Separate workflow for building ARM64 OCI on main 2024-07-08 10:36:38 +02:00
Michal Kuratczyk a23196925b
Diable ARM64 OCI (#11624) 2024-07-05 13:47:10 +02:00
Michal Kuratczyk 342702dce0
Use GHA cache for OCI; reenable ARM (#11620) 2024-07-04 18:39:20 +02:00
Michal Kuratczyk 1ebb868810
Don't build ARM64 OCI (#11609)
These are built on x86 runners in CPU emulation mode, which is very slow
(over an hour to build the image, instead of minutes). It'd be nice to have
these back at some point but for now - they are blocking the publication of
amd64 images
2024-07-03 17:19:33 +02:00
Michael Davis f52a7d3cd5
CI: Regenerate test-mixed-versions workflow
These changes are generated by 'make actions-workflows'. The change to
the template that causes the diff is in the parent commit - adding a
suffixed of "-mixed" to the job names.
2024-07-01 14:21:29 -04:00
Michael Davis 1102cdc1ea
CI: Add '-mixed' suffix to mixed-version job names
Without the suffix it's hard to tell apart regular and mixed version
workflows by name. Currently you need to go into the "Set up job" step
or look at the bazelisk cquery. Changing the names should have no
functional change - it's just easier to tell the jobs apart in the
GitHub UI.

This commit only updates the template. The child commit will regenerate
the workflow file via 'make actions-workflows'.
2024-07-01 14:21:25 -04:00
Michael Davis 0f7160e18b
CI: Use 'test-plugin-mixed.yaml' for CLI tests in mixed version template
This appears to be an oversight when creating the mixed version
template: the CLI should use the mixed version workflow template rather
than the regular plugin workflow.
2024-07-01 14:17:22 -04:00
Karl Nilsson ef7de9cbcc Actions: use a secondary cache bucket for mixed versions tests.
To hopefully reduce the amount of write limiting seen in ci.
2024-06-25 17:01:55 +01:00
Michael Klishin 806efd8c65
Merge pull request #11468 from rabbitmq/dependabot/github_actions/main/docker/build-push-action-6
build(deps): bump docker/build-push-action from 5 to 6
2024-06-19 13:21:12 -04:00
Rin Kuryloski 71fa6e3d31
Merge pull request #11478 from rabbitmq/more-turn-off-buildbuddy
Remove remaining buildbuddy usage
2024-06-19 12:47:34 +02:00
Rin Kuryloski b59da95c2d
Merge pull request #11456 from rabbitmq/rin/rules_elixir
Use rules_elixir for rabbitmqctl
2024-06-19 10:57:41 +02:00
dependabot[bot] b50f2ce991
build(deps): bump peter-evans/create-pull-request from 6.0.5 to 6.1.0
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6.0.5 to 6.1.0.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v6.0.5...v6.1.0)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-18 19:00:10 +00:00
Rin Kuryloski a2709dfd05 Remove remaining buildbuddy usage 2024-06-18 15:13:03 +02:00
Rin Kuryloski f2add661f4 Check that the cli can compile with --warnings-as-errors in ci
It's technically a build target, so we didn't need to create a whole
new test rule, but it's marked as "manual" so that it doesn't cause
bazel build //... to fail
2024-06-18 14:50:35 +02:00
Rin Kuryloski b5b7605219 Maybe reduce cache errors
- https://bazel.build/reference/command-line-reference#flag--experimental_guard_against_concurrent_changes
2024-06-18 12:06:29 +02:00
dependabot[bot] c3b0cd884c
build(deps): bump docker/build-push-action from 5 to 6
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 5 to 6.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v5...v6)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-17 18:26:22 +00:00
Marcial Rosales 14bf2bb0c1 Remove branch and add paths on PRs 2024-06-13 11:30:34 +02:00
Marcial Rosales fe304d4447 Add workflow that builds ibm mq docker image 2024-06-13 11:30:34 +02:00
Karl Nilsson 5a94049496 Add alt mirror for setup-beam action.
As the primary mirror may sometimes fail there is now a fall back.
2024-06-12 15:34:52 +01:00
Michael Klishin d109430a42 Bump google-github-actions/auth here 2024-06-06 21:51:43 -04:00
Michael Klishin 2b397246dc
Merge pull request #11402 from rabbitmq/rin/add-back-cli-tests-to-ci
Add back rabbitmq_cli tests to CI
2024-06-06 18:14:35 -04:00
dependabot[bot] 06eb76ec2d
Bump google-github-actions/auth from 2.1.2 to 2.1.3
Bumps [google-github-actions/auth](https://github.com/google-github-actions/auth) from 2.1.2 to 2.1.3.
- [Release notes](https://github.com/google-github-actions/auth/releases)
- [Changelog](https://github.com/google-github-actions/auth/blob/main/CHANGELOG.md)
- [Commits](https://github.com/google-github-actions/auth/compare/v2.1.2...v2.1.3)

---
updated-dependencies:
- dependency-name: google-github-actions/auth
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-06 18:47:44 +00:00
Rin Kuryloski bdd18ed800 Add back rabbitmq_cli tests to CI
Accidentally left out of #11343
2024-06-06 19:48:23 +02:00
Rin Kuryloski a6874e39cc
Turn off BuildBuddy integration (#11343)
Builds now execute on the github actions workers, using Google Cloud Storage (GCS) as a cache
2024-06-06 11:37:10 +02:00
Rin Kuryloski b2bd7a33a2 Setup erlang for the gazelle workflow
It must have been removed from the default actions image
2024-05-17 11:21:42 +02:00
Michael Klishin 6465b4a409
Merge pull request #11129 from rabbitmq/md/ci/bump-runner
CI: Bump actions runner to Ubuntu 22.04
2024-05-01 12:08:56 -04:00
Michael Klishin b7a7b0f429 Update current GitHub issue and PR templates
In the future we should follow [1] to add
a more focussed form.

1. https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository
2024-05-01 11:55:12 -04:00
Michael Davis 6f163592f7
CI: Bump actions runner to Ubuntu 22.04 2024-05-01 11:47:57 -04:00
dependabot[bot] db4e7a6ea7
Bump peter-evans/create-pull-request from 6.0.4 to 6.0.5
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6.0.4 to 6.0.5.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v6.0.4...v6.0.5)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-25 18:40:55 +00:00
Rin Kuryloski 07c97580bd Make the OCI workflows conditional on push access
When dependabot creates PRs, secrets are not injected, and the OCI
workflow can't push the image
2024-04-23 13:23:22 +02:00
David Ansari 6e056e5701 Fix failing test
Fix the failing GitHub action Test Authentication/Authorization backends
via mutiple messaging protocols / selenium originally caused by #11023
2024-04-22 17:51:02 +02:00
Rin Kuryloski 82f001cea6 Set the PROJECT_VERSION to 4.0.0 to match the bazel behavior
in the OCI workflow
2024-04-22 15:45:54 +02:00
dependabot[bot] b2f58075a9
Bump docker/metadata-action from 4 to 5
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 4 to 5.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Upgrade guide](https://github.com/docker/metadata-action/blob/master/UPGRADE.md)
- [Commits](https://github.com/docker/metadata-action/compare/v4...v5)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-19 18:25:56 +00:00
Rin Kuryloski 4e720d7fe1
Use make for OCI workflow (#11042)
* Use make instead of bazel for the OCI workflow
2024-04-19 20:05:24 +02:00
dependabot[bot] ff6678f4b8
Bump actions/upload-artifact from 4.3.1 to 4.3.2
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.3.1 to 4.3.2.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4.3.1...v4.3.2)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-18 18:06:19 +00:00
dependabot[bot] 89673aa33a
Bump peter-evans/create-pull-request from 6.0.3 to 6.0.4
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6.0.3 to 6.0.4.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v6.0.3...v6.0.4)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-17 18:45:23 +00:00
dependabot[bot] 583db619ad
Bump peter-evans/create-pull-request from 6.0.2 to 6.0.3
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6.0.2 to 6.0.3.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v6.0.2...v6.0.3)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-12 18:47:40 +00:00
Rin Kuryloski 2efc3114e7 Use the modern auth for ghcr within actions
https://docs.github.com/en/actions/publishing-packages/publishing-docker-images#publishing-images-to-docker-hub-and-github-packages
2024-04-12 14:31:20 +02:00
Jean-Sébastien Pédron 2ae6e79bb3 GitHub Actions: Add GHCR login step to `test-{authnz,selenium}.yaml` workflows
[Why]
Both workflows build and publish docker images but they currently fail
to publish them because of a permission denied.

[How]
This patch replicates (copy-pastes) what was done to the `oci.yaml`
workflow.
2024-04-12 14:31:20 +02:00
Rin Kuryloski 1581f7e26b
Merge pull request #10941 from rabbitmq/rin/oci-always-push
Always use GHCR when building the docker image
2024-04-09 14:56:57 +02:00
dependabot[bot] f4acb823de
Bump docker/login-action from 1 to 3
Bumps [docker/login-action](https://github.com/docker/login-action) from 1 to 3.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v1...v3)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-08 18:37:47 +00:00
Rin Kuryloski 09f2190606 Always use GHCR when building the docker image
when splitting the image, depending on the platform, docker buildx
cannot always see the imported image. So, we'll just push to GHCR
always. Members of the rabbitmq github org can log into GHCR, and be
able to perform local builds of the image
2024-04-08 18:12:00 +02:00
Rin Kuryloski 9b4486ac0f
Split the docker image to improve caching (#10924)
* Split the docker image to improve caching
2024-04-08 16:01:28 +02:00
dependabot[bot] c695dc593f
Bump lewagon/wait-on-check-action from 1.3.3 to 1.3.4
Bumps [lewagon/wait-on-check-action](https://github.com/lewagon/wait-on-check-action) from 1.3.3 to 1.3.4.
- [Release notes](https://github.com/lewagon/wait-on-check-action/releases)
- [Commits](https://github.com/lewagon/wait-on-check-action/compare/v1.3.3...v1.3.4)

---
updated-dependencies:
- dependency-name: lewagon/wait-on-check-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-04 18:44:49 +00:00
Rin Kuryloski c9ab302e79
Remove rules_docker in favor of rules_oci (#10869)
Local builds of images are extremely slow, but buildbuddy runners
don't have the emulator available
2024-04-02 19:04:38 +02:00
Rin Kuryloski 6531bb45b9
Merge pull request #10849 from rabbitmq/rin/windows-no_focus_no_shard
Use rules_erlang 3.15.1
2024-03-27 15:45:04 +01:00
Rin Kuryloski d59e843540
Update OTP version bump automation (#10857)
* Update OTP version bump automation

Use the newer method of updating WORKSPACE files that does not reorder
all the load statements

* Buildozer exits with status 3 if it makes no change
2024-03-27 11:17:36 +01:00
Rin Kuryloski 86211f240a Cache more artifacts from the windows build
for debugging purposes
2024-03-26 17:46:10 +01:00
Rin Kuryloski d4b86e71cb Add v3.13.x to the bazel/make equivalence check 2024-03-26 13:26:13 +01:00
dependabot[bot] 9463d05eb1
Bump actions/cache from 4.0.1 to 4.0.2
Bumps [actions/cache](https://github.com/actions/cache) from 4.0.1 to 4.0.2.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v4.0.1...v4.0.2)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-19 18:38:47 +00:00
Rin Kuryloski a74ddcc1cd Use the new save-always flag on the cache action
Since bazel does not cache the failures, this is desirable
2024-03-14 10:16:51 +01:00
Rin Kuryloski 0510f450fc Actions, Mergify: 3.13 is in, 3.10, 3.9 are out
(cherry picked from commit f936919cc0)
2024-03-13 11:28:29 +01:00
Rin Kuryloski 8d1ecacb02 Update mergify for v3.13.x branch
The way this was done
1. find replace "v3.12.x" with "v3.13.x"
2. find replace "v3.11.x" with "v3.12.x"
3. find replace "v3.10.x" with "v3.11.x"
4. find replace "v3.9.x" with "v3.10.x"

"v3.10.x" was left in for simplicity, in case we need a 4 branch
cascade again
2024-03-13 11:19:31 +01:00
Rin Kuryloski c51195d4c9 Revert "Actions, Mergify: 3.13 is in, 3.10, 3.9 are out"
This reverts commit f936919cc0.
2024-03-13 11:14:42 +01:00
dependabot[bot] 8a9eaac503
Bump peter-evans/create-pull-request from 6.0.1 to 6.0.2
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6.0.1 to 6.0.2.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v6.0.1...v6.0.2)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-12 18:47:34 +00:00
dependabot[bot] 22d30dbdc8
Bump actions/cache from 4.0.0 to 4.0.1
Bumps [actions/cache](https://github.com/actions/cache) from 4.0.0 to 4.0.1.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v4.0.0...v4.0.1)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-01 18:03:54 +00:00
dependabot[bot] 52d661e096
Bump peter-evans/create-pull-request from 6.0.0 to 6.0.1
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6.0.0 to 6.0.1.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v6.0.0...v6.0.1)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-28 18:29:05 +00:00
Michael Klishin f936919cc0 Actions, Mergify: 3.13 is in, 3.10, 3.9 are out 2024-02-24 20:06:19 -05:00
Marcial Rosales 09ca2f69ea Rename workflow 2024-02-10 20:12:12 +01:00
dependabot[bot] c9aaf42aef
build(deps): bump actions/upload-artifact from 4.3.0 to 4.3.1
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.3.0 to 4.3.1.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4.3.0...v4.3.1)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-06 18:28:18 +00:00
dependabot[bot] 1901811ff8
build(deps): bump peter-evans/create-pull-request from 5.0.2 to 6.0.0
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 5.0.2 to 6.0.0.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v5.0.2...v6.0.0)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-31 18:38:24 +00:00
dependabot[bot] be4b00e8e8
build(deps): bump actions/upload-artifact from 4.2.0 to 4.3.0
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.2.0 to 4.3.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4.2.0...v4.3.0)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-23 18:29:12 +00:00
dependabot[bot] 42143f9e41
build(deps): bump actions/upload-artifact from 4.1.0 to 4.2.0
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.1.0 to 4.2.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4.1.0...v4.2.0)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-19 18:38:29 +00:00
dependabot[bot] 5d1664504f
build(deps): bump actions/cache from 3.3.3 to 4.0.0
Bumps [actions/cache](https://github.com/actions/cache) from 3.3.3 to 4.0.0.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3.3.3...v4.0.0)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-17 18:11:05 +00:00
dependabot[bot] 26e4eb68fa
build(deps): bump actions/upload-artifact from 4.0.0 to 4.1.0
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.0.0 to 4.1.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4.0.0...v4.1.0)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-12 18:02:19 +00:00
Michael Klishin f23e2bdd74
Merge pull request #10323 from rabbitmq/dependabot/github_actions/main/erlef/setup-beam-1.17
build(deps): bump erlef/setup-beam from 1.16 to 1.17
2024-01-11 13:11:44 -05:00
dependabot[bot] 80b0f9c0b6
build(deps): bump erlef/setup-beam from 1.16 to 1.17
Bumps [erlef/setup-beam](https://github.com/erlef/setup-beam) from 1.16 to 1.17.
- [Release notes](https://github.com/erlef/setup-beam/releases)
- [Commits](https://github.com/erlef/setup-beam/compare/v1.16...v1.17)

---
updated-dependencies:
- dependency-name: erlef/setup-beam
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-11 18:09:20 +00:00
dependabot[bot] dd3fb30078
build(deps): bump actions/cache from 3.3.2 to 3.3.3
Bumps [actions/cache](https://github.com/actions/cache) from 3.3.2 to 3.3.3.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3.3.2...v3.3.3)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-11 18:09:16 +00:00
Michal Kuratczyk 8d2657ef2f Adopt OTP 26.2.1 2024-01-09 10:14:08 +01:00
Marcial Rosales 90db360e6b Trigger workflow that run authnz-messaging suite
when either the suite, or the test changes
2023-12-27 14:08:08 +00:00
Michael Klishin 5af5f0cf3d
Selenium: update run-suites.sh argument to match #10200 2023-12-23 20:33:21 -05:00
Michael Klishin f39570cc73
GHA Erlang/OTP updates: keep both 26.2 and 26.1 2023-12-22 01:44:11 -05:00
Michael Klishin a75bee72ee
GHA: use Erlang 26.2.x 2023-12-22 01:41:29 -05:00
Michael Klishin 05ed950806
Do not bump Erlang versions for v3.10.x 2023-12-21 23:23:46 -05:00
dependabot[bot] 89114dd65e
build(deps): bump lewagon/wait-on-check-action from 1.3.1 to 1.3.3
Bumps [lewagon/wait-on-check-action](https://github.com/lewagon/wait-on-check-action) from 1.3.1 to 1.3.3.
- [Release notes](https://github.com/lewagon/wait-on-check-action/releases)
- [Commits](https://github.com/lewagon/wait-on-check-action/compare/v1.3.1...v1.3.3)

---
updated-dependencies:
- dependency-name: lewagon/wait-on-check-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-21 18:53:07 +00:00
Michael Klishin f4bf4fe1d1
Merge pull request #10152 from rabbitmq/dependabot/github_actions/main/actions/download-artifact-4
build(deps): bump actions/download-artifact from 3 to 4
2023-12-14 13:56:07 -05:00
dependabot[bot] 9b3475f5e5
build(deps): bump actions/download-artifact from 3 to 4
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 3 to 4.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-14 18:49:18 +00:00
dependabot[bot] 8ceeaba9d9
build(deps): bump actions/upload-artifact from 3.1.3 to 4.0.0
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 3.1.3 to 4.0.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v3.1.3...v4.0.0)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-14 18:49:13 +00:00
Michael Klishin 9d944390f3
Disable Dependabot for older branches
3.8, 3.9 and 3.10 (which goes EOL in less than three weeks)
2023-12-14 13:48:46 -05:00
Michael Davis a12ea5fdd9
CI: Use 'bazel cquery' to find OCI tests
Bazel 7 can't seem to resolve Osiris 1.7.1 when running this query using
'bazel query':

    ERROR: Error computing the main repository mapping: in module dependency chain <root> -> rabbitmq_osiris@1.7.1: module not found in registries: rabbitmq_osiris@1.7.1

Switching to cquery resolves the tests correctly. It also prints the
configuration hashes per case which we need to cut off with awk.
2023-12-12 11:44:32 -05:00
Jean-Sébastien Pédron b3dac51df1
GitHub Actions: Convert '_' to '-' in $AWS_ECS_CLUSTER_NAME
[Why]
The AWS ECS cluster name must match the regex `[a-zA-Z][-a-zA-Z0-9]*`.
Underscores are forbidden.

[How]
Like '.' was converted, we convert '_' to '.' using Bash parameter
expansion.
2023-12-01 15:32:29 +01:00
Rin Kuryloski 9837a5486e Match the workflow triggering to oci.yaml
But also add a concurrency group to serialize runs
2023-11-30 15:21:02 +01:00
Rin Kuryloski 8905346959 sync otp_version_id across oci.yaml and rabbitmq_peer_discovery_aws.yaml 2023-11-30 15:06:21 +01:00
Michael Klishin e9958e92b6 Workflows: use Erlang 26 as 25.3 is no longer supported 2023-10-25 22:53:57 -04:00
Michael Klishin 680c36d509 Require Erlang/OTP 26.0 (ERTS 14.0) 2023-10-25 22:53:57 -04:00
Diana Parra Corbacho 5f0981c5a3
Allow to use Khepri database to store metadata instead of Mnesia
[Why]

Mnesia is a very powerful and convenient tool for Erlang applications:
it is a persistent disc-based database, it handles replication accross
multiple Erlang nodes and it is available out-of-the-box from the
Erlang/OTP distribution. RabbitMQ relies on Mnesia to manage all its
metadata:

* virtual hosts' properties
* intenal users
* queue, exchange and binding declarations (not queues data)
* runtime parameters and policies
* ...

Unfortunately Mnesia makes it difficult to handle network partition and,
as a consequence, the merge conflicts between Erlang nodes once the
network partition is resolved. RabbitMQ provides several partition
handling strategies but they are not bullet-proof. Users still hit
situations where it is a pain to repair a cluster following a network
partition.

[How]

@kjnilsson created Ra [1], a Raft consensus library that RabbitMQ
already uses successfully to implement quorum queues and streams for
instance. Those queues do not suffer from network partitions.

We created Khepri [2], a new persistent and replicated database engine
based on Ra and we want to use it in place of Mnesia in RabbitMQ to
solve the problems with network partitions.

This patch integrates Khepri as an experimental feature. When enabled,
RabbitMQ will store all its metadata in Khepri instead of Mnesia.

This change comes with behavior changes. While Khepri remains disabled,
you should see no changes to the behavior of RabbitMQ. If there are
changes, it is a bug. After Khepri is enabled, there are significant
changes of behavior that you should be aware of.

Because it is based on the Raft consensus algorithm, when there is a
network partition, only the cluster members that are in the partition
with at least `(Number of nodes in the cluster ÷ 2) + 1` number of nodes
can "make progress". In other words, only those nodes may write to the
Khepri database and read from the database and expect a consistent
result.

For instance in a cluster of 5 RabbitMQ nodes:
* If there are two partitions, one with 3 nodes, one with 2 nodes, only
  the group of 3 nodes will be able to write to the database.
* If there are three partitions, two with 2 nodes, one with 1 node, none
  of the group can write to the database.

Because the Khepri database will be used for all kind of metadata, it
means that RabbitMQ nodes that can't write to the database will be
unable to perform some operations. A list of operations and what to
expect is documented in the associated pull request and the RabbitMQ
website.

This requirement from Raft also affects the startup of RabbitMQ nodes in
a cluster. Indeed, at least a quorum number of nodes must be started at
once to allow nodes to become ready.

To enable Khepri, you need to enable the `khepri_db` feature flag:

    rabbitmqctl enable_feature_flag khepri_db

When the `khepri_db` feature flag is enabled, the migration code
performs the following two tasks:
1. It synchronizes the Khepri cluster membership from the Mnesia
   cluster. It uses `mnesia_to_khepri:sync_cluster_membership/1` from
   the `khepri_mnesia_migration` application [3].
2. It copies data from relevant Mnesia tables to Khepri, doing some
   conversion if necessary on the way. Again, it uses
   `mnesia_to_khepri:copy_tables/4` from `khepri_mnesia_migration` to do
   it.

This can be performed on a running standalone RabbitMQ node or cluster.
Data will be migrated from Mnesia to Khepri without any service
interruption. Note that during the migration, the performance may
decrease and the memory footprint may go up.

Because this feature flag is considered experimental, it is not enabled
by default even on a brand new RabbitMQ deployment.

More about the implementation details below:

In the past months, all accesses to Mnesia were isolated in a collection
of `rabbit_db*` modules. This is where the integration of Khepri mostly
takes place: we use a function called `rabbit_khepri:handle_fallback/1`
which selects the database and perform the query or the transaction.
Here is an example from `rabbit_db_vhost`:

* Up until RabbitMQ 3.12.x:

        get(VHostName) when is_binary(VHostName) ->
            get_in_mnesia(VHostName).

* Starting with RabbitMQ 3.13.0:

        get(VHostName) when is_binary(VHostName) ->
            rabbit_khepri:handle_fallback(
              #{mnesia => fun() -> get_in_mnesia(VHostName) end,
                khepri => fun() -> get_in_khepri(VHostName) end}).

This `rabbit_khepri:handle_fallback/1` function relies on two things:
1. the fact that the `khepri_db` feature flag is enabled, in which case
   it always executes the Khepri-based variant.
4. the ability or not to read and write to Mnesia tables otherwise.

Before the feature flag is enabled, or during the migration, the
function will try to execute the Mnesia-based variant. If it succeeds,
then it returns the result. If it fails because one or more Mnesia
tables can't be used, it restarts from scratch: it means the feature
flag is being enabled and depending on the outcome, either the
Mnesia-based variant will succeed (the feature flag couldn't be enabled)
or the feature flag will be marked as enabled and it will call the
Khepri-based variant. The meat of this function really lives in the
`khepri_mnesia_migration` application [3] and
`rabbit_khepri:handle_fallback/1` is a wrapper on top of it that knows
about the feature flag.

However, some calls to the database do not depend on the existence of
Mnesia tables, such as functions where we need to learn about the
members of a cluster. For those, we can't rely on exceptions from
Mnesia. Therefore, we just look at the state of the feature flag to
determine which database to use. There are two situations though:

* Sometimes, we need the feature flag state query to block because the
  function interested in it can't return a valid answer during the
  migration. Here is an example:

        case rabbit_khepri:is_enabled(RemoteNode) of
            true  -> can_join_using_khepri(RemoteNode);
            false -> can_join_using_mnesia(RemoteNode)
        end

* Sometimes, we need the feature flag state query to NOT block (for
  instance because it would cause a deadlock). Here is an example:

        case rabbit_khepri:get_feature_state() of
            enabled -> members_using_khepri();
            _       -> members_using_mnesia()
        end

Direct accesses to Mnesia still exists. They are limited to code that is
specific to Mnesia such as classic queue mirroring or network partitions
handling strategies.

Now, to discover the Mnesia tables to migrate and how to migrate them,
we use an Erlang module attribute called
`rabbit_mnesia_tables_to_khepri_db` which indicates a list of Mnesia
tables and an associated converter module. Here is an example in the
`rabbitmq_recent_history_exchange` plugin:

    -rabbit_mnesia_tables_to_khepri_db(
       [{?RH_TABLE, rabbit_db_rh_exchange_m2k_converter}]).

The converter module  — `rabbit_db_rh_exchange_m2k_converter` in this
example  — is is fact a "sub" converter module called but
`rabbit_db_m2k_converter`. See the documentation of a `mnesia_to_khepri`
converter module to learn more about these modules.

[1] https://github.com/rabbitmq/ra
[2] https://github.com/rabbitmq/khepri
[3] https://github.com/rabbitmq/khepri_mnesia_migration

See #7206.

Co-authored-by: Jean-Sébastien Pédron <jean-sebastien@rabbitmq.com>
Co-authored-by: Diana Parra Corbacho <dparracorbac@vmware.com>
Co-authored-by: Michael Davis <mcarsondavis@gmail.com>
2023-09-29 16:00:11 +02:00
Rin Kuryloski 35110d8261 Do not backport otp 26 toolchain updates to v3.11.x 2023-09-29 15:39:24 +02:00
Michael Klishin 27d88dafd8 Drop an action that we no longer need
What was running out of disk space were BuildBuddy
worker hosts, not GitHub Actions worker ones.

Plus the Action is currently broken because it tries
to delete a package that was renamed.
2023-09-28 10:08:49 -04:00
Rin Kuryloski 4974adb6cb Fixup a workflow 2023-09-21 09:21:30 +02:00
Michael Klishin db9425e2b5
Merge pull request #9486 from rabbitmq/rin/use-otp-26.1
Use OTP 26.1 as OTP 26 in CI
2023-09-20 15:38:54 -04:00
Rin Kuryloski 75eb0621fc Use OTP 26.1 as OTP 26 in CI 2023-09-20 15:33:34 +02:00
Rin Kuryloski 0ce931a7d7 Add Elixir 1.15 to update-elixir-patches.yaml workflow 2023-09-20 14:58:44 +02:00
Rin Kuryloski 929794c9ee Fix branch checkout in check-build-system-equivalence.yaml 2023-09-14 09:35:58 +02:00
Rin Kuryloski 9ffb3c11fa Fixup nightly build system check
Use different archive names based on the server version, so that they
don't conflict when running the nightly matrix
2023-09-13 11:13:21 +02:00
Rin Kuryloski e6f9d6ef73 More output when comparing build systems in actions 2023-09-13 10:44:37 +02:00
Michael Klishin 6a5b95425c
Merge pull request #9394 from rabbitmq/dependabot/github_actions/main/docker/setup-buildx-action-3
build(deps): bump docker/setup-buildx-action from 1 to 3
2023-09-12 15:25:24 -04:00
Michael Klishin f25bbb863c
Merge pull request #9392 from rabbitmq/dependabot/github_actions/main/docker/build-push-action-5
build(deps): bump docker/build-push-action from 4 to 5
2023-09-12 15:25:16 -04:00
Michael Klishin 3d50d2c5b1
Merge pull request #9391 from rabbitmq/dependabot/github_actions/main/docker/login-action-3
build(deps): bump docker/login-action from 1 to 3
2023-09-12 15:25:11 -04:00
dependabot[bot] aeb0911f02
build(deps): bump docker/setup-buildx-action from 1 to 3
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 1 to 3.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v1...v3)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-12 18:34:29 +00:00
dependabot[bot] 31460fa8bf
build(deps): bump docker/setup-qemu-action from 2 to 3
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 2 to 3.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-12 18:34:27 +00:00