We don't need this anymore, now that the high watermark is bumped
automatically when the log level is set to `debug` in rabbit_lager.
This reverts commit 49956c6423.
The stop-node command is the only make target still using erl_call who
is prone to breakage (broken in OTP 21.3) and can readily be replaced
with rabbitmqctl stop.
The seq command should include a -1 increment to stop nodes in reverse
order. Previously, as an example with NODES=2, will run `seq 2 1`
which produces no items to iterate, so the entire stop-node loop does
not execute and the brokers are left running.
* Lager: 3.6.4 -> 3.6.5
* Ranch: 1.5.0 -> 1.6.1
* ranch_proxy_protocol: 1.5.0 -> 2.1.0-rc.1
Note that ranch_proxy_protocol 2.1.0-rc.1 is not an official release
from upstream: it was published by us (the RabbitMQ team) because we
don't have feedback from upstream about a pull request to update Ranch
to 1.6.x (heroku/ranch_proxy_protocol#49). Hopefully upstream will merge
the pull request and cut a new official release.
Fixes rabbitmq/rabbitmq-common#269.
[#160270896]
erl_call(1) is broken in Erlang 21.0-rc.2 when we use it to evaluate
some code on a remote node. The issue was reported upstream but we can
replace erl_call(1) by `rabbitmqctl eval` to achieve the same result.
The output is still filtered using sed(1) to ensure it remains
unchanged, in case testsuites expect a particular format.
We still continue to use erl_call(1) to stop a node because we can't
achieve the same behavior using rabbitmqctl(1). This use of erl_call(1)
is working with Erlang 21.0-rc.2.
[#157964874]
That way, it doesn't interfere with testcases working with multiple
RabbitMQ nodes. Furthermore, independent sets of nodes won't try to
autoconnect to the common_test node, and thus unrelated nodes after.
This is useful for the upcoming VM-based test helpers.
[#153749132]
(cherry picked from commit 47a5bdfff548a5c278af2d947feeeb7836aae0c3)
That way, it doesn't interfere with testcases working with multiple
RabbitMQ nodes. Furthermore, independent sets of nodes won't try to
autoconnect to the common_test node, and thus unrelated nodes after.
This is useful for the upcoming VM-based test helpers.
[#153749132]
rabbitmq-common shouldn't see this, otherwise it's a reverse dependency
and thus a dependency circle.
In the future, we should fix this by adding rabbitmq-cli as an explicit
dependency to whatever component needs it.
[#153850881]
This way, we are sure that the possibly newer recipe in
`rabbitmq-tools.mk` handles the operation instead of a possibly obsolete
`rabbitmq-components.mk` copy.
[#154761483]
To avoid a copy of `rabbitmq-components.mk` from rabbitmq-common's
`v3.6.x` branch to another component's `master` branch, we want to
compare branch names of both repositories.
This situation may occur when checking out the `v3.6.x` in the Umbrella
using `gmake up BRANCH=v3.6.x` but for instance some plugins don't have
such a branch. In this case, they remain on the previous branch. We
don't want to overwrite their `rabbitmq-components.mk` if this happens.
[#154761483]
If the test broker is started using:
gmake run-broker PLUGINS_FROM_DEPS_DIR=1
Then plugins are taken from `$(DEPS_DIR)` even though, the `.ez`
archives were created.
This can be handy when e.g. working in the management plugin's static
files: it makes it possible to modify a CSS and just reload the page to
see the change. There is no need to re-run `gmake run-broker`.
The value of `PLUGINS_FROM_DEPS_DIR` is not verified.
Caveats:
* Irrelevant plugins which happend to be already compiled will
be enabled because rabbitmq-plugins(8) will find them inside
`$(DEPS_DIR).
* This feature isn't tested when a plugin is cloned directly and
it probably doesn't work. The developer is expected to use the
Umbrella for now.
[#154435736]
This target is useful to get the list of commits since the last
(pre)release or a specified tag in all RabbitMQ repositories involved
(i.e. the current project and its RabbitMQ dependencies). In particular,
from rabbitmq-server-release, it can help during the review of all
commits, issues and pull requests while preparing a release.
To see an interactive list of commits for each repository:
gmake commits-since-release
To get the same list formatted as Markdown (to publish on GitHub):
gmake commits-since-release MARKDOWN=yes
To get the commits since the last prerelease:
gmake commits-since-release SINCE_TAG=last-prerelease
To get the commits since a specific release:
gmake commits-since-release SINCE_TAG=v3.7.1
[#153087397]
We are about to rename `stable` to `v3.6.x`. That's why we need to
update the script which determines if a development branch was forked
from `v3.6.x` (instead of `stable`) and falls back to `master` if it's
not the case.
While here, handle the case where the `master` branch doesn't exist
locally. This happens when using `git clone -b $branch` or in CI.
(cherry picked from commit 037bab7f799bf9cc180b0dfa2b7899b7a464cf9d)
This file is copied to Elixir-based components (e.g. rabbitmq_cli) when
we create the source archive of RabbitMQ. It allows RabbitMQ to build
offline (which is the case in many package build farms).
References rabbitmq/rabbitmq-server-release#61.
[#153358632]
Namely:
* Depend on ranch_proxy_protocol 1.4.4 (which depends on ranch 1.4.0,
the same as us).
* Fix `check-rabbitmq-components.mk` for rabbitmq-common.
This change was committed to the management-related plugins directly
during the switch to Cowboy 2.0. Unfortunately, it was lost when
rabbitmq-components.mk was updated globally.
Therefore this commit restores the new expected pinning.
Unfortunately, the "skip CI" feature isn't what I would expect: I
expected that such a commit would not trigger a job, but that the commit
would still be picked by a job triggered for another reason. In fact,
it's entirely ignored: the resource will be stuck at the previous
commit, until another regular commit (i.e. without the "skip CI"
message) is pushed.
This reverts commit c9e636c0b4.
Therefore, after we branch `v3.7.x`, rabbitmq-components.mk will try to
determine if a development branch was forked from `v3.7.x` (instead of
`stable`) and fallback to `master` if it's not the case.
While here, handle the case where the `master` branch doesn't exist
locally. This happens when using `git clone -b $branch` or in CI.
In other words, instead of copying rabbitmq-common's credentials which
won't be valid in another project, keep the existing credentials (if
any).
Tha said, if CREDS is set, they will be overwritten again.
[#152509619]
It assumes that all projects use the same `.travis.yml` usually (we try
to achieve the same in Concourse). For special cases, like e.g.
rabbitmq-amqp1.0 which needs .NET Core, there is a `.travis.yml.patch`
in the project which is applied to the stock `.travis.yml`.
Like `rabbitmq-components-mk`, it accepts a `DO_COMMIT=yes` variable to
automatically create a commit if there is a change to `.travis.yml`.
There is also a `update-travis-yml` to update it recursively.
[#152509619]
Escape sequences used to move cursor around are not properly translated
in the HTML output, thus defeating the purpose of the Common Test output
module.
[#152509619]
... for `rabbit_common` and `amqp_client`.
This should only be used for testing purpose (e.g. dry-run in CI),
otherwise dependency tracking will break: `amqp_client` depends on a
specific version of `rabbit_common`.
However in CI, we want to be able to do a publish dry-run of
`amqp_client`. As it requires `rabbit_common` to be published, we need
to override the version pinning to point it to an already published
version of `rabbit_common` (the corresponding version of `rabbit_common`
was not published either). This is ok because nothing is published in
the end.
[#150482173, #150482202]
When we publish our packages to Hex.pm, we use the simplified
rabbitmq-components.hexpm.mk to replace the regular
rabbitmq-components.mk.
Before commit ba59f969b7,
rabbitmq-components.mk took 3rd-party dependencies from GitHub. Now that
it takes them from Hex.pm, we don't need to override and hard-code them
again in rabbitmq-components.hexpm.mk.
Thus now, we extract them from rabbitmq-components.mk and put them at
the end of rabbitmq-components.hexpm.mk when we publish to Hex.pm.
The other benefit is that we don't have to remember to change version
pinning in both rabbitmq-components.mk and rabbitmq-components.hexpm.mk.
[#150482173]
This way, when we publish our own packages to Hex.pm (rabbitmq-common
and rabbitmq-erlang-client as of now), we are sure that no dependency
will miss from Hex.pm.
Before that, 3rd-party dependencies were taken from GitHub, and only
when we published to Hex.pm, the source was overriden and set to Hex.pm.
This led to bad surprises such as `ranch_proxy_protocol` which was
unavailable on Hex.pm, preventing us from publishing our packages.
While here, remove pinning for mochiweb and webmachine which we are not
using anymore.
[#150482173]
We were using the `master` branch before.
While here, we extract the version of `ranch_proxy_protocol` and store
it in a dedicated variable. This will be used in `rabbitmq-hexpm.mk` to
prepare `rebar.config`.
... as well as their matching `stop-*` targets.
The goal is to ease the start of many nodes with a single command.
To start 6 unclustered RabbitMQ nodes:
gmake start-brokers NODES=6
To start 6 clustered RabbitMQ nodes:
gmake start-cluster NODES=6
To stop them:
gmake stop-brokers NODES=6
gmake stop-cluster NODES=6
Nodes are called `rabbit-$N@$HOSTNAME`. $N starts at 1. TCP ports are
taken from 5672/15672/25672 and increased by one for each subsequent
node.
This also works from plugins, so you can start a cluster with eg. the
management plugin enabled on all nodes.
For Erlang-based applications, the path is expected to be
`$application/ebin` and Erlang.mk sets up everything for us.
For Elixir-based applications, the path is something like
`$application/_build/$env/lib/*/ebin`. To add those, first we make sure
that $(ERLANG_MK_RECURSIVE_DEPS_LIST) is generated because it will
contains all build and runtime dependencies paths, and we pass that
filename to xrefr(1) using an environment variable.
Then xrefr(1) parses that file and looks at all of them if they contain
`_build/dev/lib/*/ebin` directories. If that's the case, those are added
to the code path.
Later in the script, the code path is added to xref library path.
[#140125673]
Erlang.mk removes the copy of xrefr(1) during `make distclean-xref` (and
thus `make distclean`) because it assumes the script was fetched and can
be refetched.
In our case, the script is committed to this repository and all projects
are using it. Therefore, `make distclean` shouldn't remove our script.
To work around this, we do not set `$(XREFR)` if the target is either
`distclean` or `distclean-xref`.
In particular:
- If an MFA is listed in -ignore_xref(), ignore both the error in
the calling module having this -ignore_xref() and the undefined
MFA if any.
- Ignore all errors from functions listed in
-erlang_version_support() that will be "compiled-out" at runtime.
[#140125673]
It is forked from xref_runner. Compared to upstream, it allows to
exclude entire modules: we need that for the *_compat modules.
Furthermore it's compatible with Erlang R16B03 because:
- It's a plain text escript, not a precompiled one.
- It doesn't use maps.
[#140125673]
Set `$(XREFR_ARGS)` for all projects using the `rabbitmq-early-test.mk`
early-stage plugin so that they also use this configuration file.
[#118490793]
(cherry picked from commit 44dd06343d)
... the target is *-on-concourse. This fixes the make recipe because the
dependency might be missing on the host, which caused fly(1) to complain
because the specified input was not there.
This makes the loop on `$(RMQ_CI_CT_HOOKS)` useless in rabbitmq-test.mk,
because those inputs are handled with the previous loop on TEST_DEPS.
[#146078591]
It allows to run any target on Concourse. All test dependencies are
taken locally and uploaded to Concourse before the make target is
executed. The test-output is written to logs/on-concourse.
By default, the oldest version of Erlang supported by the branch is
used. One can override that with the $(ERLANG_VERSION) make variable.
Here are some examples:
gmake tests-on-concourse
gmake dialyzer-on-concourse
gmake ct-on-concourse V=1
gmake ct-unit-on-concourse t=parallel_tests:pid_decompose_compose
[#146078591]
... to `rabbit_upgrade_functions.erl`. We need it in this module
specifically, but nowhere else for now. So restore the default behavior
of the compiler (Erlang 20.x warns about `export_all` by default).
We need to exclude looking_glass and lz4 from plugins and use
a different loading method because it is not possible to load
NIFs from within archives.
An example usage follows:
$ make run-broker RABBITMQ_TRACER=rabbit_tracer:connections
In another shell:
$ bin/runjava com.rabbitmq.perf.PerfTest
Then some time later stop the test and the node, then run:
$ make profile
$ qcachegrind
With the latest update of Erlang.mk, regular plugins are loaded near the
end of `erlang.mk`. This means our plugins can't add any DEPS.
Fortunately we can do that from early-stage plugins which are loaded
near the beginning of `erlang.mk`.
Therefore, we move part of `rabbitmq-build.mk` to
`rabbitmq-early-test.mk` because it appends to `$(TEST_DEPS)`. We also
create `rabbitmq-early-plugin.mk` wrapper for plugins, like the
`rabbitmq-plugin.mk`.
A project can now load local plugins, so rabbitmq-common can load them
like other projets.
We can get rid of the compilation flags duplication because plugins are
loaded late enough that they can append to `ERLC_OPTS`.
[#144697185]
... instead of hardcoding the value.
This flag takes a list of hooks separated by `and`. This formatting
should really go into Erlang.mk which would accept a `$(CT_HOOKS)`
variable.
For instance, `$(CT_OPTS)` was broken with Jenkins because multiple
`ct_hooks` were passed instead of using the `and` syntax.
Some dependencies will be archived by erlang.mk itself,
so there is no point in rebuilding them using archive.build.deps
Using the new --skip switch we can skip archives already built and
also rabbit application itself.
[Fixes#191]
The goal of this hook is to change the default output of common_test on
stdout to something colored and concise. The HTML reports still contain
everything.
This hook only works with Erlang 19.2+. With previous version, it will
be a no-op and the outout will remain unchanged.
With this change, we restore the default verbosity of common_test on
Concourse. It means way too many messages with Erlang up-to 19.1 but
that's ok. On Travis, we keep the lowered verbosity because we could
reach the limit set by Travis. Note that the verbosity affects both
stdout and the HTML reports so on Concourse, we should have more data to
debug failures.
While here, reorganize a bit the common_flags to separate each "topic".
[#138917395]
After changing any .erl-file and running `make dist` .ez files are
regenerated. But this is not the case for Elixir plugins - so it's
necessary to clean `plugins/` to refresh.
The difference is that for erlang.mk projects there is an explicit
dependencies on `ebin`, `include` and `priv` folder content -
b2d635f5a2/mk/rabbitmq-dist.mk (L52)
But there were no similar rule around
b2d635f5a2/mk/rabbitmq-dist.mk (L91)
So I've tried to add some reasonable dependencies there, and it helps.
One of the thing RabbitMQ expects from a plugin is a dependency on the
`rabbit` app. The problem is that `mix run` tries to start plugin app,
which leads to an unsuccessful attempt to start `rabbit` app.
We pass `-erl_args` followed by several flags to ct_run(1) in order to
turn off messages from Erlang's kernel specifically on Concourse.
`-erl_flags` explicitely tells ct_run(1) to pass all following flags to
the emulators and thus common_test doesn't see or interpret them.
Before this change, we added the `-ct_hook` at the end of the command
line. This was fine when trying it on our workstations because we don't
try to silence extra logging there (ie. `-erl_args` is not passed).
However on Concourse, this made ct_run(1) pass `-ct_hooks` to the
emulator (which had no effect) and common_test never enabled the hook.
Therefore this patch moves `-ct_hooks cth_fail_fast` before messing with
`-erl_args`.
Because of this, our use of `$(CT_OPTS)` is fragile because all changes
later in other Makefiles will be ignored. Erlang.mk would need to expose
two variables for common_test and emulator flags. I added a comment in
rabbitmq-build.mk so we pay attention to flags order in this file at
least.
While here, only enable `cth_fail_fast` on Concourse. Everywhere else,
the full testsuites are executed no matter the failures.
Signed-off: Gerhard Lazu <gerhard@rabbitmq.com>
[#141155967]
Loading a native library (eg. a NIF) from an .ez archive in unsupported
by Erlang and the team decided to revert this change again.
This reverts commit a855222dfa.
[#136622317]
This caused a circular dependency in Makefile targets:
test-dist -> test-build -> test-dist
This was reported by GNU Make as:
gmake: Circular test-dist <- test-build dependency dropped.
This problem didn't cause any issue. However, adding `clean` as a
dependency of `test-dist` broke several assumptions, as well as many
testsuites.
This reverts commit 0599f0ac7a.
Support for reading static files from Erlang .ez archives was added to
Cowboy and various RabbitMQ plugins were updated to take advantage of
the new API.
So this commit restores commit ef00860597.
It was reverted in 9037b78a3c because
Cowboy didn't have this support at the time.
Signed-off-by: Gerhard Lazu <gerhard@rabbitmq.com>
[#118562759, #136622317]
The new feature of Cowboy 1.1.0 of interest to RabbitMQ is the
possibility to serve static files from Erlang application .ez archives.
This will allow us to get rid of the plugins expansion during RabbitMQ
startup.
Cowboy 1.1.0 also comes with a stricter URL parser. In particular, `+`
characters are not converted to spaces anymore! They are left alone.
Users of the HTTP API must make sure that their requests are encoded
using path segment encoding, not query string encoding (where `+` are
converted to spaces).
References ninenines/cowboy#1070.
References ninenines/cowboy#1080.
[#136779967]
This is the default behavior so it shouldn't be needed, but Syntastic
sets this flag by default. So disabling it here overrides Syntastic
internal default.
We rely on the `mix_task_archive_deps` Mix task to create the project
.ez archive, as well as all its dependencies' archive. It also creates
an archive for Elixir runtime itself.
This task is also used to list the created archives, in order to filter
the one we want from the staled archives.
Write 'y' to mix(1) stdin, because it may ask confirmation to install
Hex.
References #rabbitmq/rabbitmq-auth-backend-uaa#3.
References #rabbitmq/rabbitmq-auth-backend-uaa#5.
[#136341089]
... before assigning the result to $(ELIXIR_LIB_DIR).
This removes the mixing of `:` used as a PATH separator and a drive
letter marker at the same time. This was causing future Erlang VMs to
crash with "Bad address".
[#137695585]
Those targets are used by both Erlang checkers to query compiler
options. Those options are required to properly report errors and
warnings.
This is a temporary commit to this project: once the patches to the
Syntastic Erlang checkers have stabilized, the change should be
submitted to Erlang.mk instead, and removed from this project.
[#136364285]
To do it, run:
make hex-publish PROJECT_VERSION=3.6.6
You can also publish documentation to hexdocs.pm. The documentation is
generated first with the `docs` Erlang.mk target and the content of the
`doc` subdirectory is published. To do it, run:
make hex-publish-docs PROJECT_VERSION=3.6.6
Generated files are published to Hex.pm so that users of this package
don't have to depend on rabbitmq-codegen and Python.
`git-revisions.txt` is generated in the process to carry the version of
the project. Not really standard, but it's the way other RabbitMQ
projects do it for now, so take advantage of the existing
infrastructure. Once a package is published to Hex.pm, it's good to
`make clean` so this file is removed.
Other RabbitMQ components can also be published to Hex.pm with a few
modifications. For them, rabbit_common/mk/rabbitmq-components.hexpm.mk
is sent in place of their copy of rabbitmq-components.mk. This modified
file pulls dependencies from Hex.pm using version $(PROJECT_VERSION),
instead of using Git.
All projects published to Hex.pm can use $(RABBITMQ_HEXPM_DEFAULT_FILES)
as the list of files to put in the `files` key in the .app file.
[#134268483]
If `git-revisions.txt` exists, take the version from this file.
Otherwise, use git-describe(1) to compute a version from the working
copy. If it's not a Git clone, fall back to `0.0.0`.
[#130992027]
The pinned dependencies are:
o cowboy
o mochiweb
o ranch
o webmachine
In particular, pinning webmachine fixes the testsuites of
rabbitmq-shovel-management and rabbitmq-tracing.
This has two advantages:
o We don't have to rely on a package ability to put `erl_call` in
the $PATH.
o We are sure that `erl_call` comes from the same Erlang as `erl`.
[#130099459]
The rule is still there and we can enable it in CI, but it's not
executed out-of-the-box anymore. It was more painful than anything else:
the file doesn't change often and doing the check in CI will be more
than enough.
Now that everything was moved to common_test, we don't need the old
framework anymore.
The only target still in use was `cluster-other-node`. As it was only
used by rabbitmq-java-client, the target was moved there.
The `test` target, which was an alias to `tests` (the regular Erlang.mk
target), was removed at the same time.
[#127356157]
Jean-Sébastien Pédron
Michael Klishin
kjnilsson
Yury.Alioshinov
Luke Bakken
Loïc Hoguin
John Eckersberg
Diana Corbacho
Jean-Sébastien Pédron
Daniil Fedotov
Gerhard Lazu
Michael Klishin
Alexey Lebedeff
Michael Klishin
Daniil Fedotov
Daniil Fedotov
Alexey Lebedeff
Arnaud Cogoluègnes