My workflow is developing RabbitMQ on an Ubuntu server.
Since it doesn't have a desktop or browser, let's fallback
for `bazel run test-logs` command to start an HTTP server instead.
From now on, I can run conveniently:
```
david@nuc:~/workspace/rabbitmq-server$ bazel run test-logs //deps/rabbitmq_auth_backend_oauth2:unit_SUITE --config=local
INFO: Analyzed target //:test-logs (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:test-logs up-to-date:
bazel-bin/open-test-logs.sh
INFO: Elapsed time: 0.047s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
Archive: outputs.zip
Couldn't find a suitable web browser!
Set the BROWSER environment variable to your desired browser.
Warning: program returned non-zero exit code #256
/usr/bin/open: 882: www-browser: not found
/usr/bin/open: 882: links2: not found
/usr/bin/open: 882: elinks: not found
/usr/bin/open: 882: links: not found
/usr/bin/open: 882: lynx: not found
/usr/bin/open: 882: w3m: not found
xdg-open: no method available for opening 'index.html'
Open your browser at http://nuc:8000/index.html
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
```
and then open http://nuc:8000/index.html in my browser on MacOS.
to have the equivalent of `make start-cluster` and `make stop-cluster`.
To create a 3-node RabbitMQ cluster:
```
bazel run --config=local start-cluster
```
To define number of nodes or a custom directory:
```
bazel run --config=local start-cluster NODES=5 TEST_TMPDIR="$HOME/scratch/myrabbit"
```
To stop the cluster:
```
bazel run --config=local stop-cluster
```
or, if started by the 2nd command:
```
bazel run --config=local stop-cluster NODES=5 TEST_TMPDIR="$HOME/scratch/myrabbit"
```
'us' is used when Unicode is not available.
Prior to this commit:
```
$ kubectl logs r1-server-0 -c rabbitmq | ag time
2022-06-30 13:37:35.253927+00:00 [debug] <0.336.0> wal: recovered 00000003.wal time taken 0ms
2022-06-30 13:37:35.262592+00:00 [debug] <0.349.0> wal: recovered 00000003.wal time taken 0ms
2022-06-30 13:37:35.489016+00:00 [debug] <0.352.0> Feature flags: time to find supported feature flags: 76468 �s
2022-06-30 13:37:35.495193+00:00 [debug] <0.352.0> Feature flags: time to regen registry: 6032 �s
2022-06-30 13:37:35.500574+00:00 [debug] <0.361.0> Feature flags: time to find supported feature flags: 937 �s
2022-06-30 13:37:35.500603+00:00 [debug] <26705.398.0> Feature flags: time to find supported feature flags: 891 �s
2022-06-30 13:37:35.507998+00:00 [debug] <26705.398.0> Feature flags: time to regen registry: 7199 �s
2022-06-30 13:37:35.509092+00:00 [debug] <0.361.0> Feature flags: time to regen registry: 8396 �s
```
Before this commit:
```
Consistent hashing exchange: removing binding from exchange '"exchange 'e1' in vhost '/'"' to destination '"queue 'qq2' in vhost '/'"' with routing key '2'
```
The single quotes do not make sense.
First binding wins.
Duplicate bindings, i.e. bindings with the same source exchange and
same destination queue / exchange but possibly different routing key
(weight) are ignored from now on by the consistent hash exchange.
This applies only to bindings being added.
For bindings being deleted, any duplicate binding (independent of its
routing key) will delete all buckets for the given source and
destination. (This is to ensure that buckets for a given source and
destination can be deleted for when upgrading from a version prior
to this commit. This was also the behaviour prior to this commit,
so nothing changes in that regard.)
Note that duplicate bindings continue to be created in RabbitMQ.
(They are only ignored by the consistent hash exchange.)
Adding a binding will perform linear search in the bucket map.
This is already stated in the README:
"These two operations use linear algorithms to update the ring."
The linear search when adding a binding could be optimised by
adding another Mnesia table field which will require a new migration and
feature flag. Hence, such an optimization is left out in this commit.
Fixes#3386.
common_test installs its own logger handler, which is great.
Unfortunately, this logger handler drops all messages having a domain,
except when the domain is among the domains used by Erlang itself.
In RabbitMQ, we use logger domains to categorize messages. Therefore
those messages are dropped by the common_test's logger handler.
This commit introduces another logger handler which sits on top of the
common_test one and makes sure messages with a domain are logged as
well.
This gen_statem-based process is responsible for handling concurrency
when feature flags are enabled and synchronized when a cluster is
expanded.
This clarifies and stabilizes the behavior of the feature flag subsystem
w.r.t. situations where e.g. a feature flag migration function takes
time to update data and a new node joins a cluster and synchronizes its
feature flag states with the cluster. There was a chance that the
feature flag was marked as enabled on the joining node, even though the
migration function didn't take care of that node.
With this new feature flags controller, enabling or synchronizing
feature flags blocks and delays any concurrent operations which try to
modify feature flags states too.
This change also clarifies where and when the migration function is
called: it is called at least once on each node who knows the feature
flag and when the state goes from "disabled" to "enabled" on that node.
Note that even if the feature flag is being enabled on a subset of the
nodes (because other nodes already have it enabled), it is marked as
"state_changing" everywhere during the migration. This is to prevent
that a node where it is enabled assumes it is enabled on all nodes who
know the feature flag.
There is a new feature as well: just after a feature flag is enabled,
the migration function is called a second time for any post-enable
actions. The feature flag is marked as enabled between these "enable"
and "post-enable" steps. The success or failure of this "post-enable"
run does not affect the state of the feature flag (i.e. it is ignored).
A new migration function API is introduced to allow more advanced
things. The new API is:
my_migration_function(
#ffcommand{name = ...,
props = ...,
command = enable | post_enable,
extra = #{...}})
The record is defined in `include/feature_flags.hrl`. Here is the
meaning of each field:
* `name` and `props` are the equivalent of the `FeatureName` and
`FeatureProps` arguments of the previous migration function API.
* `command` is basically the same as the previous `Arg` arguments.
* `extra` is map containing context-specific information. For instance, it
contains the list of nodes where the feature flag state changes.
This whole new behavior is behind a new feature flag called
`feature_flags_v2`. If a feature flag uses the new migration function
API, `feature_flags_v2` will be automatically enabled.
If many feature flags are enabled at once (like when a fresh RabbitMQ
node is started for the first time), `feature_flags_v2` will be enabled
first if it is in the list.
Michael Klishin
Michael Klishin
David Ansari
Michael Klishin
Arnaud Cogoluègnes
dependabot[bot]
Karl Nilsson
Gabriele Santomaggio
Jean-Sébastien Pédron
Ayanda Dube