Use the `sys_dist` ets table to get distribution port information.
Fixes#4981
Get cluster links stats for TLS dist
Use code from prometheus.erl to get dist links info
Also rework elixir dependency handling, so we no longer rely on mix to
fetch the rabbitmq_cli deps
Also:
- Specify ra version with a commit rather than a branch
- Fixup compilation options for erlang 23
- Add missing ra reference in MODULE.bazel
- Add missing flag in oci.yaml
- Reduce bazel rbe jobs to try to save memory
- Use bazel built erlang for erlang git master tests
- Use the same cache for all the workflows but windows
- Avoid using `mix local.hex --force` in elixir rules
- Fetching seems blocked in CI, and this should reduce hex api usage in
all builds, which is always nice
- Remove xref and dialyze tags since rules_erlang 3 includes them in
the defaults
This will be used to fixrabbitmq/osiris#78
If a RabbitMQ `advanced.config` file contains the following:
```
{customize_hostname_check, [
{match_fun, public_key:pkix_verify_hostname_match_fun(https)}
]}
```
...`file:consult/1` will fail because it does not evaluate terms in the
file.
The code in `rabbit_consult` was copied from this OTP module:
https://github.com/erlang/otp/blob/master/lib/ssl/src/ssl_dist_sup.erl
...and then modified for our use.
Add Bazel suite
Use the same license as Erlang/OTP, add link to source cc @dumbbell
Add test and ensure value returned matches file:consult/1
Add test data file
Ensure that Funs are converted to binaries before jsx:encode is called
Add a check that customize_hostname_check can be JSON encoded
Ensure that customize_hostname_check and match_fun are filtered out from listener data
When applications accidentally set an unreasonable high value for
the message TTL expiration field, e.g. 6779303336614035452,
before this commit quorum queue and classic queue processes crashed:
```
2022-05-17 13:35:26.488670+00:00 [notice] <0.1000.0> queue 'test' in vhost '/': candidate -> leader in term: 2 machine version: 2
2022-05-17 13:35:26.489492+00:00 [error] <0.1000.0> crasher:
2022-05-17 13:35:26.489492+00:00 [error] <0.1000.0> initial call: ra_server_proc:init/1
2022-05-17 13:35:26.489492+00:00 [error] <0.1000.0> pid: <0.1000.0>
2022-05-17 13:35:26.489492+00:00 [error] <0.1000.0> registered_name: '%2F_test'
2022-05-17 13:35:26.489492+00:00 [error] <0.1000.0> exception error: bad argument
2022-05-17 13:35:26.489492+00:00 [error] <0.1000.0> in function erlang:start_timer/4
2022-05-17 13:35:26.489492+00:00 [error] <0.1000.0> called as erlang:start_timer(6779303336614035351,<0.1000.0>,
2022-05-17 13:35:26.489492+00:00 [error] <0.1000.0> {timeout,expire_msgs},
2022-05-17 13:35:26.489492+00:00 [error] <0.1000.0> [])
2022-05-17 13:35:26.489492+00:00 [error] <0.1000.0> *** argument 1: exceeds the maximum supported time value
2022-05-17 13:35:26.489492+00:00 [error] <0.1000.0> in call from gen_statem:loop_timeouts_start/16 (gen_statem.erl, line 2108)
2022-05-17 13:35:26.489492+00:00 [error] <0.1000.0> ancestors: [<0.999.0>,ra_server_sup_sup,<0.250.0>,ra_systems_sup,ra_sup,
2022-05-17 13:35:26.489492+00:00 [error] <0.1000.0> <0.186.0>]
```
In this commit, we disallow expiry fields higher than 100 years.
This causes the channel to be closed which is better than crashing the
queue process.
This new validation applies to message TTLs and queue expiry.
From the docs of erlang:start_timer:
"The absolute point in time, the timer is set to expire on, must be in the interval
[erlang:convert_time_unit(erlang:system_info(start_time), native, millisecond),
erlang:convert_time_unit(erlang:system_info(end_time), native, millisecond)].
If a relative time is specified, the Time value is not allowed to be negative.
end_time:
The last Erlang monotonic time in native time unit that can be represented
internally in the current Erlang runtime system instance.
The time between the start time and the end time is at least a quarter of a millennium."
rabbitmq_cli uses some private rules_erlang apis that have changed in
the upcoming release
Additionally:
- Avoid including both standard and test versions of amqp_client in
integration test suites
- Eliminate most of the compilation order hints (explicit first_srcs)
in the bazel build
- Fix an include statement - in bazel, an app is not available to
itself as a library at compilation time
In particular:
- io_file_handle_open_attempt
- queue_index_journal_write
Neither have proven to be very useful in recent years
and with the move to FHC-less and journal-less v2 index
they will slowly become irrelevant. This should be a
good compromise until we can switch to v2 permanently
or rework the stats module to use counters.
During most of the time the file_handle_cache_stats ets table is
used for writing only.
By enabeling `write_concurrency` on the table we allow different values
to be written concurrently without taking a global lock.
There the only codepath reading from the ets table is run on the
`collect_statistics_interval` interval and reads the whole table.
So we can assume we are not blocking any large amount of concurrent reads.
as an opt-in feature. The goal is to avoid re-importing the definition
from the definition file/directory/source if we know the content
has not changed. Since this feature won't be appropriate for
every environment (sometimes unconditional reimporting is expected),
the feature is opt-in.
This is still a WIP.
bazel-erlang has been renamed rules_erlang. v2 is a substantial
refactor that brings Windows support. While this alone isn't enough to
run all rabbitmq-server suites on windows, one can at least now start
the broker (bazel run broker) and run the tests that do not start a
background broker process
This is to address another memory leak on win32 reported here:
https://groups.google.com/g/rabbitmq-users/c/UE-wxXerJl8
"RabbitMQ constant memory increase (binary_alloc) in idle state"
The root cause is the Prometheus plugin making repeated calls to `rabbit_misc:otp_version/0` which then calls `file:read_file/1` and leaks memory on win32.
See https://github.com/erlang/otp/issues/5527 for the report to the Erlang team.
Turn `badmatch` into actual error
This is copied from https://github.com/rabbitmq/rabbitmq-common/pull/349
If a message is sent to only one queue(in most application scenarios), passing through the 'delegate' is meaningless. Otherwise, it increases the delay of the message and the possibility of 'delegate' congestion.
Here are some test data：
node1: Pentium(R) Dual-Core CPU E5300 @ 2.60GHz
node2: Pentium(R) Dual-Core CPU E5300 @ 2.60GHz
Join node1 and node2 to a cluster. Create 100 queues on node2, and start 100 consumers to receive messages from these queues.
Start 100 publishers on node1 to send messages to the queues of node2. Each publisher will send 10k messages at the rate of 100/s(10k/s theoretically in total), and all the messages for all publishers is 1 million.
Before optimisation:
{1,[{msg_time,812312(=<1ms),177922(=<5ms),9507(=<50ms),221(=<500ms),38(=<1000ms),0,0,0,0,1061,1069,0,0}]}
After optimisation:
{1,[{msg_time,902854(=< 1ms),93993(=<5ms),3038(=<50ms),96(=<500ms),19(=<1000ms),0,0,0,0,1049,1060,0,0}]}
Additional information:
Time counted here is the stay time of a message in the cluster, that is, Time(leaving from node2 at) - Time(reaching node1 at).
"812312(=<1ms)" is the number of messages with time consumption less than or equal to 1ms.
Overall, the optimisation is effective.
It was automatically happening for e.g. `make start-cluster`.
But some plugins were not covered by default generated config, and
running rabbit from 2 different worktrees was a bit complicated.
A value that is too low will prevent the index from shutting
down in time when there are many queues. This leads to the
process being killed and on the next RabbitMQ restart a
(potentially very long) dirty recovery is needed.
The value of 10 minutes was chosen to mirror the shutdown
timeout of the message store. Since both queues and message
store need to have shut down gracefully in order to have
a clean restart it makes sense to use the same value.
Related: c40c2628a9
When we fail to parse name of cipher suite from PROXY protocol
just say that no ssl is used, instead of trying to fill that
with data from connection between proxy and our server.
A user could already enable single-line logging (the `single_line`
option of `logger_formatter` or RabbitMQ internal formatters) from the
configuration file. For example:
log.console.formatter.single_line = on
With this patch, the option can be enabled from the `$RABBITMQ_LOG`
environment variable as well:
make run-broker RABBITMQ_LOG=+single_line
Those environment variables are unset by default. The default values are
set in the `rabbit` application environment and can be configured in the
configuration file. However, the environment variables will take
precedence over them respectively if they are set.
They were trying to run `hostname` and `which`, which produced a bunch
of error messages in a hermetic build environment.
And performance of those `shell` calls is not very important, as they
are caled just a few times during script runtime anyway (there is a
hack to make these lazy, but evaluating only once - but it's hardly
worth it).
Unlike pg2, pg in Erlang 24 is eventually consistent. So this
reintroduces some of the same kind of locking mirrored_supervisor
used to rely on implicitly via pg2.
Per discussion with @lhoguin.
Closes#3260.
References #3132, #3154.
This has the unfortunate side effect of causing a rebuild of all
applications every time. I need to figure out another place to build and
install the CLI during build time (instead of as part of the dist
target).
This reverts commit 4322cca66e.
and assume it is a string-like value ("directory string")
because other values would not make much sense in the
username extraction context.
References #2983.
instead of specific ones since they will vary with the payload
(one of them likely indicates UTF string length).
This is still not perfect because we limit the maximum
allowed length but it works fine with identifiers up to 100
characters long, which should be good enough for this
best effort handling of an abscure SAN type.
References ##2983.
The parser didn't handle literals of the form:
'single-quoted'unquoted'single-quoted-again'"or-even-double-quoted"
In particular, the unquoted parsing assumed that nothing else could
follow it. The testsuite is extended with the issue reporter's case.
While here, improve escaped characters handling. They used to be not
parsed specifically at all.
Fixes#2969.
Note that the type by definition contains arbitrary values. According
to the OTP types, they are triplets that represent effectively
a key/value pair. So we assume the pair is a string that needs a bit
massaging, namely stripping the UTF encoding prefix OTP AnotherName
decoder leaves in.
Kudos to @Thibi2000 for providing an example value.
Closes#2983.
for usability. It is not any different from when a float value
is used and only exists as a counterpart to '{absolute, N}'.
Also nothing changes for rabbitmq.conf users as that format performs
validation and correct value translation.
See #2694, #2965 for background.
Adds WORKSPACE.bazel, BUILD.bazel & *.bzl files for partial build & test with Bazel. Introduces a build-time dependency on https://github.com/rabbitmq/bazel-erlang
The consolidation of `rabbitmq-components.mk` broke the previous
method by which rabbit components were detected. Now we check
$(RABBITMQ_COMPONENTS) directly.
In kind version 0.10.0, when creating a 5-node RabbitMQ cluster
with the new parallel PodManagementPolicy, we observed that some
pods were restarted. Their logs included:
```
10:10:03.794 [error]
10:10:03.804 [error] BOOT FAILED
10:10:03.805 [error] ===========
BOOT FAILED
10:10:03.805 [error] ERROR: epmd error for host r1-server-0.r1-nodes.rabbitmq-system: nxdomain (non-existing domain)
10:10:03.805 [error]
===========
ERROR: epmd error for host r1-server-0.r1-nodes.rabbitmq-system: nxdomain (non-existing domain)
10:10:04.806 [error] Supervisor rabbit_prelaunch_sup had child prelaunch started with rabbit_prelaunch:run_prelaunch_first_phase() at undefined exit with reason {epmd_error,"r1-server-0.r1-nodes.rabbitmq-system",nxdomain} in context start_error
10:10:04.806 [error] CRASH REPORT Process <0.152.0> with 0 neighbours exited with reason: {{shutdown,{failed_to_start_child,prelaunch,{epmd_error,"r1-server-0.r1-nodes.rabbitmq-system",nxdomain}}},{rabbit_prelaunch_app,start,[normal,[]]}} in application_master:init/4 line 138
```
Eventually, after some pods restarted up to 2 times, all pods were running and ready.
In kind, we observed that during the first couple of seconds, nslookup was failing as well for that domain
with nxdomain.
It took up to 30 seconds until nslookup succeeded.
With this commit, pods don't need to be restarted when creating a fresh
RabbitMQ cluster.
This allows including additional applications or third party
plugins when creating a release, running the broker locally,
or just building from the top-level Makefile.
To include Looking Glass in a release, for example:
$ make package-generic-unix ADDITIONAL_PLUGINS="looking_glass"
A Docker image can then be built using this release and will
contain Looking Glass:
$ make docker-image
Beware macOS users! Applications such as Looking Glass include
NIFs. NIFs must be compiled in the right environment. If you
are building a Docker image then make sure to build the NIF
on Linux! In the two steps above, this corresponds to Step 1.
To run the broker with Looking Glass available:
$ make run-broker ADDITIONAL_PLUGINS="looking_glass"
This commit also moves Looking Glass dependency information
into rabbitmq-components.mk so it is available at all times.
Lager strips trailing newline characters but OTP logger with the default
formatter adds a newline at the end. To avoid unintentional multi-line log
messages we have to revisit most messages logged.
Some log entries are intentionally multiline, others
are printed to stdout directly: newlines are required there
for sensible formatting.
The configuration remains the same for the end-user. The only exception
is the log root directory: it is now set through the `log_root`
application env. variable in `rabbit`. People using the Cuttlefish-based
configuration file are not affected by this exception.
The main change is how the logging facility is configured. It now
happens in `rabbit_prelaunch_logging`. The `rabbit_lager` module is
removed.
The supported outputs remain the same: the console, text files, the
`amq.rabbitmq.log` exchange and syslog.
The message text format slightly changed: the timestamp is more precise
(now to the microsecond) and the level can be abbreviated to always be
4-character long to align all messages and improve readability. Here is
an example:
2021-03-03 10:22:30.377392+01:00 [dbug] <0.229.0> == Prelaunch DONE ==
2021-03-03 10:22:30.377860+01:00 [info] <0.229.0>
2021-03-03 10:22:30.377860+01:00 [info] <0.229.0> Starting RabbitMQ 3.8.10+115.g071f3fb on Erlang 23.2.5
2021-03-03 10:22:30.377860+01:00 [info] <0.229.0> Licensed under the MPL 2.0. Website: https://rabbitmq.com
The example above also shows that multiline messages are supported and
each line is prepended with the same prefix (the timestamp, the level
and the Erlang process PID).
JSON is also supported as a message format and now for any outputs.
Indeed, it is possible to use it with e.g. syslog or the exchange. Here
is an example of a JSON-formatted message sent to syslog:
Mar 3 11:23:06 localhost rabbitmq-server[27908] <0.229.0> - {"time":"2021-03-03T11:23:06.998466+01:00","level":"notice","msg":"Logging: configured log handlers are now ACTIVE","meta":{"domain":"rabbitmq.prelaunch","file":"src/rabbit_prelaunch_logging.erl","gl":"<0.228.0>","line":311,"mfa":["rabbit_prelaunch_logging","configure_logger",1],"pid":"<0.229.0>"}}
For quick testing, the values accepted by the `$RABBITMQ_LOGS`
environment variables were extended:
* `-` still means stdout
* `-stderr` means stderr
* `syslog:` means syslog on localhost
* `exchange:` means logging to `amq.rabbitmq.log`
`$RABBITMQ_LOG` was also extended. It now accepts a `+json` modifier (in
addition to the existing `+color` one). With that modifier, messages are
formatted as JSON intead of plain text.
The `rabbitmqctl rotate_logs` command is deprecated. The reason is
Logger does not expose a function to force log rotation. However, it
will detect when a file was rotated by an external tool.
From a developer point of view, the old `rabbit_log*` API remains
supported, though it is now deprecated. It is implemented as regular
modules: there is no `parse_transform` involved anymore.
In the code, it is recommended to use the new Logger macros. For
instance, `?LOG_INFO(Format, Args)`. If possible, messages should be
augmented with some metadata. For instance (note the map after the
message):
?LOG_NOTICE("Logging: switching to configured handler(s); following "
"messages may not be visible in this log output",
#{domain => ?RMQLOG_DOMAIN_PRELAUNCH}),
Domains in Erlang Logger parlance are the way to categorize messages.
Some predefined domains, matching previous categories, are currently
defined in `rabbit_common/include/logging.hrl` or headers in the
relevant plugins for plugin-specific categories.
At this point, very few messages have been converted from the old
`rabbit_log*` API to the new macros. It can be done gradually when
working on a particular module or logging.
The Erlang builtin console/file handler, `logger_std_h`, has been forked
because it lacks date-based file rotation. The configuration of
date-based rotation is identical to Lager. Once the dust has settled for
this feature, the goal is to submit it upstream for inclusion in Erlang.
The forked module is calld `rabbit_logger_std_h` and is based
`logger_std_h` in Erlang 23.0.
Subsequent nodes fail to start since ports are already in use. This
makes it possible to start multiple nodes locally with all plugins
enabled.
Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
as node names grow.
Prior to this change, direct reply-to consumer channels
were encoded using term_to_binary/1, which means the result
would grow together with node name (since node name
is one of the components of an Erlang pid type).
This means that with long enough hostnames, reply-to
identifiers could overflow the 255 character limit of
message property field type, longstr.
With this change, the encoded value uses a hash of the node name
and then locates the actual node name from a map of
hashes to current cluster members.
In addition, instead of generating non-predictable "secure"
GUIDs the feature now generates "regular" predictable GUIDs
which compensates some of the additional PID pre- and post-processing
outlined above.
Now that dependencies are packaged as directories and not .ez
files, the fact that both LG and LZ4 are NIFs is no longer
an issue. And having it as regular dependencies simplifies
REPL-driven profiling.
Per discussion with @dumbbell.
The `set` command in the implementation of `/bin/sh` included in the
official RabbitMQ Docker image returns multi-line variable values
differently than the tested Bourne shell implementation (GNU Bash, dash
and FreeBSD sh).
I don't know what implementation is used by that Docker image, but here
is the output of `set`, for a variable set to "\n'test'":
TEST_VAR='
'"'"'test'"'"
The problem was reported in the following discussion:
https://github.com/rabbitmq/rabbitmq-server/discussions/2458
While here, add a small testcase to check a couple outputs.
... instead of 23.0.
Erlang 23.1 is the version the Concourse pipelines use. We expect the
Concourse pipelines and the GitHub Actions workflow to be on the same
page.
GC collection can then be done by deleting all entries on the ETS table
and total counters per protocol can be kept without individually scanning
all entries
net_adm:name/1 returns a new value, 'noport', in Erlang 24. This value
being absent in the function spec in previous versions of Erlang, we get
a warning from Dialyzer until we start to the yet-to-be-release Erlang
24 in CI. Therefore we disable this specific warning.
... instead of .ez archives.
The benefits for doing this:
* We can use native code, as is the case for lz4 and zstd bindings in
the Tanzu RabbitMQ version for instance. Indeed, Erlang does not
support loading native code (NIF or port drivers) from .ez archives.
* We can remove custom code to handle .ez archives. We have special
cases in Erlang.mk plugins as well as the `rabbit_plugins` module, in
particular the code to extract .ez archives (even though Erlang knows
how to use them directly).
* Prevent hard to debug situations when the .ez archive name does not
match the top-level directory inside the archive. In this case, Erlang
says it can't load the application but doesn't tell much more.
* Debugging and "hot-patching" plugins become easier: we just have to
copy the recompiled .beam file in place of the existing one. There
is no need to unpack the plugin, replace the file and recreate the
archive.
* Release packages can be smaller. gzip, bzip2 and xz, common
compression algorithm for Unix packages, give much better result if
they compress the .beam files directly instead of "compressing" zip
files (the .ez archives are plain zip archives). For instance, the
generic-unix package goes from 15 MiB when using .ez archives to just
12 MiB when using directory.
I would also like to experiment with Erlang releases in the future.
Using directories for Erlang applications instead of .ez archives is
mandatory for this to work according to my latest tests.
Of course, this change doesn't break support for .ez archives (and we
will keep support for this). End users can still download third-party
plugins as .ez archives and drop them in the plugins directory.
On Windows, the current working directory is also searched, which can
lead to problems. Instead, use `init:get_argument(root)` to get the root
of the Erlang release, then we know `bin/erl` will always be present.
In addition to the `rabbitmq-components.mk` existence check, we now
verfy that the directory is named `deps`.
This is to increase the chance that, if we find a
`rabbitmq-componentS.mk` file in the upper directories, this project is
indeed inside a DEPS_DIR.
For instance, in our GitHub Actions workflows, when we prepared the
secondary umbrellas for mixed-version testing, it happened that the
secondary umbrellas were under a clone of rabbitmq-server. Therefore
the first (and only) condition was met and the Makefile erroneously
considered it was inside a DEPS_DIR. As a consequence, dependencies of
the umbrellas were fetched in the wrong place.
net_adm:name/1 returns a new value, 'noport', in Erlang 24. This value
being absent in the function spec in previous versions of Erlang, we get
a warning from Dialyzer until we start to the yet-to-be-release Erlang
24 in CI. Therefore we disable this specific warning.
... instead of .ez archives.
The benefits for doing this:
* We can use native code, as is the case for lz4 and zstd bindings in
the Tanzu RabbitMQ version for instance. Indeed, Erlang does not
support loading native code (NIF or port drivers) from .ez archives.
* We can remove custom code to handle .ez archives. We have special
cases in Erlang.mk plugins as well as the `rabbit_plugins` module, in
particular the code to extract .ez archives (even though Erlang knows
how to use them directly).
* Prevent hard to debug situations when the .ez archive name does not
match the top-level directory inside the archive. In this case, Erlang
says it can't load the application but doesn't tell much more.
* Debugging and "hot-patching" plugins become easier: we just have to
copy the recompiled .beam file in place of the existing one. There
is no need to unpack the plugin, replace the file and recreate the
archive.
* Release packages can be smaller. gzip, bzip2 and xz, common
compression algorithm for Unix packages, give much better result if
they compress the .beam files directly instead of "compressing" zip
files (the .ez archives are plain zip archives). For instance, the
generic-unix package goes from 15 MiB when using .ez archives to just
12 MiB when using directory.
I would also like to experiment with Erlang releases in the future.
Using directories for Erlang applications instead of .ez archives is
mandatory for this to work according to my latest tests.
Of course, this change doesn't break support for .ez archives (and we
will keep support for this). End users can still download third-party
plugins as .ez archives and drop them in the plugins directory.
On Windows, the current working directory is also searched, which can
lead to problems. Instead, use `init:get_argument(root)` to get the root
of the Erlang release, then we know `bin/erl` will always be present.
In addition to the `rabbitmq-components.mk` existence check, we now
verfy that the directory is named `deps`.
This is to increase the chance that, if we find a
`rabbitmq-componentS.mk` file in the upper directories, this project is
indeed inside a DEPS_DIR.
For instance, in our GitHub Actions workflows, when we prepared the
secondary umbrellas for mixed-version testing, it happened that the
secondary umbrellas were under a clone of rabbitmq-server. Therefore
the first (and only) condition was met and the Makefile erroneously
considered it was inside a DEPS_DIR. As a consequence, dependencies of
the umbrellas were fetched in the wrong place.
and add a VMware copyright notice.
We did not mean to make this code Incompatible with Secondary Licenses
as defined in [1].
1. https://www.mozilla.org/en-US/MPL/2.0/FAQ/
When we source the $CONF_ENV_FILE script, we set a few variables which
this script expects. Those variables are given without their prefix. For
instance, $MNESIA_BASE.
The $CONF_ENV_FILE script can set $RABBITMQ_MNESIA_BASE. Unfortunately
before this patch, the variable would be ignored, in favor of the
default value which was passed to the script ($MNESIA_BASE).
The reason is that variables set by the script are handled in the
alphabetical order. Thus $MNESIA_BASE is handled first, then
$RABBITMQ_MNESIA_BASE.
Because the code didn't give any precedence, the first variable set
would "win". This explains why users who set $RABBITMQ_MNESIA_BASE in
$CONF_ENV_FILE, but using RabbitMQ 3.8.4+ (which introduced
`rabbit_env`), unexpectedly had their node use the default Mnesia base
directory.
The patch is rather simple: when we check if a variable is already set,
we give precedence to the $RABBITMQ_* prefixed variables. Therefore, if
the $CONF_ENV_FILE script sets $RABBITMQ_MNESIA_BASE, this value will be
used, regardless of the value of $MNESIA_BASE.
This didn't happen with variables set in the environment (i.e. the
environment of rabbitmq-server(8)) because the prefixed variables
already had precedence.
Fixesrabbitmq/rabbitmq-common#401.
This allows RabbitMQ to configure `rabbit_log` as a Logger handler.
See a related commit in rabbit_prelaunch_early_logging in
rabbitmq-server, where `rabbit_log` is being configured as a Logger
handler. The commit message explains the reason behind this.
The default timeout of 30 seconds was not sufficient to allow graceful shutdown of a message store with millions of persistent messages. Rather than increase the timeout in general, introduce a new macro with a default of 600 seconds
... instead of the cache action.
The cache action is quite unstable (failing to download the cached
files). In this commit, we try to use the artefacts instead. At this
point, we don't know if it is more reliable, but we'll see with time.
As an added bonus, we can download the archives passed between jobs for
inspection if we need.
Otherwise, for instance, running Dialyzer in the Erlang client fails with the
following error if it was cloned directly (i.e. outside of the Umbrella):
dialyzer: Bad directory for -pa: .../amqp_client/deps/rabbitmq_cli/_build/dev/lib/rabbitmqctl/ebin
... and their value.
Both prefixed and non-prefixed variables are returned by this function.
While here, fix a conflict between $RABBITMQ_HOME and $HOME in
var_is_used/1: the latter shouldn't be considered as used.
When we generate the workflows, we pick the latest tag of each release
branch. That list of tags is used to clone secondary umbrellas in the
workflows and run the testsuites against each of them.
When generating workflows for `master`, we take the latest tag of each
release branch.
When generating workflows for a release branch, we take the latest tag
of each older release branch, plus the first tag of the same release
branch.
Some examples:
* `master` is tested with 3.8.3 and 3.7.25
* `v3.8.x` is tested with 3.8.0 and 3.7.25
We need a monotonically increasing number for the version used by the
Concourse S3 resource. A Git commit hash does not work because they do
not have this property.
The main entry point is `make github-actions` which generates the
workflows.
Currently, it handles workflows to test the project with different
versions of Erlang.
It generates a file called `$(PROJECT)-rabbitmq-deps.mk` which has a
dependency definition line of the form expected by Erlang.mk, for each
RabbitMQ component the project depends on.
Therefore the line indicates:
* `git` as the fetch method
* the repository URL
* the Git commit hash the dependency is on
Here is an example for rabbitmq-server:
dep_rabbit_common := git https://github.com/rabbitmq/rabbitmq-common.git d9ccd8d9cdd58310901f318fed676aff59be5afb
dep_rabbitmq_cli := git https://github.com/rabbitmq/rabbitmq-cli.git f6eaae292d27da4ded92b7c1b51a8ddcfefa69c2
dep_rabbitmq_codegen := git https://github.com/rabbitmq/rabbitmq-codegen.git 65da2e86bd65c6b6ecd48478ab092721696bc709
The double-quoting was requited in the flock(1)/lockf(1) blocks because
of the use of `sh -c`. However it's incorrect in the `else` block.
Follow-up to commit 3f32a36e50.
The CLI has a high startup time. To speed up the
`start-background-broker` and `stop-node` recipes, two CLI calls are
replaced by two more basic commands which achieve the same goal.
The problem with the previous approach was that the `$(wildcard ...)`
directives might be evaluated too early: `deps/rabbit` might not be
available yet.
Moving the computation to the body of the recipe fixes the problem
because dependencies are available at this point.
In other words, if instead of cloning the Umbrella, one cloned
rabbitmq-server directly, the `install-cli-scripts` recipe would fail to
copy the scripts because it assumed `rabbit` was under `$(DEPS_DIR)`.
Now expected places are checked and an error is emitted if the recipe
can't find the right one.
Michael Klishin
Arnaud Cogoluègnes
Philip Kuryloski
Luke Bakken
Michael Klishin
Loïc Hoguin
David Ansari
Michael Klishin
Felix Huettner
Karl Nilsson
tomyouyou
Sagi Sarussi
Alexey Lebedeff
Michal Kuratczyk
Paweł Chmielowski
Jean-Sébastien Pédron
Ilya Khaprov
Jean-Sébastien Pédron
Pavel Heimlich
kjnilsson
Carl Hörberg
Gerhard Lazu
dcorbacho
Gabriele Santomaggio
Luke Bakken
Loïc Hoguin
Nicolas Caille
Michael Klishin
Anupama Singh
Ayanda-D
Philip Kuryloski