Commit Graph

1148 Commits

Author SHA1 Message Date
Michael Klishin a5098b28a7
Bump Lager to 3.8.2 for OTP 24 compatibility 2021-02-24 12:53:30 +03:00
dcorbacho 8592291afa Fix notify_decorators and policy notifications 2021-02-21 22:56:06 +01:00
Michael Klishin d0506cc225
Add a single channel mode-specific test 2021-02-20 10:32:24 +03:00
Michael Klishin a9dd03bdf5
Make it possible to use a single channel for message transfers and commands
For the rare case where federated exchange experience rapid
binding changes. As of rabbitmq/rabbitmq-federation#97,
such environments have a race condition between binding
and message propagation.
2021-02-20 00:42:39 +03:00
dcorbacho 699cd1ab29 Add federation support for quorum queues 2021-02-18 17:15:47 +01:00
Michael Klishin b11a79cccf
Bump (c) year in header files 2021-02-04 07:04:58 +03:00
Arnaud Cogoluègnes b921ac11a8
Merge pull request #2712 from rabbitmq/rabbitmq-stream-prometheus
Add stream prometheus plugin
2021-01-27 16:46:37 +01:00
Michael Klishin 52479099ec
Bump (c) year 2021-01-22 09:00:14 +03:00
Arnaud Cogoluègnes bf72683eb2
Add stream prometheus plugin 2021-01-11 16:49:56 +01:00
Arnaud Cogoluègnes 08891a734e
Merge branch 'master' into rabbitmq-stream-management 2020-11-30 09:42:54 +01:00
Arnaud Cogoluègnes 23d7e8114c
Introduce stream management plugin 2020-11-19 14:48:25 +01:00
dcorbacho f7c127a9ef Ensure pg2 group is created only when links are created and removed with them
The queue decorator is called even when no links are present, which caused the
pg2 group to be created. At the same time, deleting a link wasn't deleting
the queue group. Closes #2497
2020-11-18 12:11:51 +00:00
Jean-Sébastien Pédron 47686ee1f0
Remove unused .github directories
They were valid until the switch to the "monorepository" when everything
was merged into a single Git repository.
2020-11-17 13:33:16 +01:00
Arnaud Cogoluègnes a2f249e30e Update rabbitmq-components.mk 2020-11-03 14:27:39 +01:00
Philip Kuryloski ad401bb9dd Increase rabbitmq_federation test timeouts
To address test flakiness
2020-10-23 10:32:59 +02:00
Michael Klishin f66476b555 Update rabbitmq-components.mk 2020-10-21 12:55:35 +03:00
Ayanda-D be713c4a7a If amqp_connection:start/2 succeeds, it would be
very rare for amqp_connection:open_channel/2 to
timeout and result in the same fate of direct
connection and channel leaks. However, it can't
be ruled out in an unstable TCP network. This
commit prevents possibility of process leaks
on such failed channel establishment cases.
2020-10-20 08:27:15 +01:00
Ayanda-D 986f5830e4 Fix some bad direct connection and channel process leaks
on failed upstream connection attempts due to AMQP client
timeouts. On long running nodes, these have potential to
eventually take down a node on reaching or exceeding the
the process limit.
2020-10-19 19:06:09 +01:00
Philip Kuryloski 4323c643e7 rabbitmq_federation: use dynamic waits where possible
to address spurious failures in CI

Building upon recent changes and discussion with @michaelklishin
2020-10-13 11:19:31 +02:00
Michael Klishin 1aa2e059bb Rework two more tests
Per discussion with @pjk25.
2020-10-13 06:26:23 +03:00
Michael Klishin 5bedcf2fdc Correct status suite expectations
Per discussion with @pjk25.
2020-10-12 18:50:24 +03:00
Michael Klishin 63f0068d79 Correct upstream URIs in this test 2020-10-10 00:49:14 +03:00
Michael Klishin d9252f2342 Merge pull request #118 from rabbitmq/queue-suite-flake
Replace a fixed test suite delay with a dynamic one
2020-09-29 17:46:46 +03:00
Philip Kuryloski 287d0c67e6 Replace a fixed test suite delay with a dynamic
queue_SUITE > without_disambiguate > cluster_size_1 >
dynamic_plugin_stop_start fails often in CI
2020-09-29 16:07:37 +02:00
Jean-Sébastien Pédron 7621a74b6e exchange_SUITE: Fix format string
`~s` was used to print an integer. This was causing a badarg exception.
2020-09-23 12:04:39 +02:00
Michael Klishin 9403fe931b Merge pull request #115 from rabbitmq/rabbitmq-federation-114
Use mirrored_supervisor for queue federation
2020-09-17 03:25:01 +03:00
Luke Bakken 7040f1de25 Revert "Remove timer"
This reverts commit a621950561.
2020-09-16 13:13:20 -07:00
Luke Bakken a621950561 Remove timer 2020-09-16 11:10:31 -07:00
Michael Klishin 9ffb39c3ca Give links more time to initialize [for CI] 2020-09-16 20:12:40 +03:00
Michael Klishin a87fc37791 await_bindings for a limited amount of time 2020-09-16 19:41:45 +03:00
Luke Bakken 50df6d4d4f Use await_binding, add vhost support 2020-09-16 08:44:27 -07:00
Michael Klishin e77bea6c7d Log payloads received in exchange federation tests 2020-09-16 04:50:15 +03:00
Michael Klishin 353aebbabd More exchange federation test massaging 2020-09-16 04:43:08 +03:00
Michael Klishin df01ae64f9 More selective assertions on link status
Asserting on links across all virtual hosts makes no sense
for some tests, and creates additional shared state and
makes these assertions significantly more timing-sensitive.

Per discussion with @lukebakken.
2020-09-16 04:28:23 +03:00
Michael Klishin b599f40e8b Begin splitting test groups to reduce the amount of shared state 2020-09-16 04:06:58 +03:00
Michael Klishin bfc26c1642 Compile 2020-09-16 00:32:21 +03:00
Michael Klishin a5f32394b5 Make this test clean up its resources at the end 2020-09-16 00:31:57 +03:00
Michael Klishin cb61d620ff Take virtual host into account when detecting binding propagation cycles
The same way we do it for forwarded messages.
2020-09-16 00:13:02 +03:00
Michael Klishin c52af7aad2 A WIP integration test for multi-hop intra-cluster exchange federation
The test currently fails. Added more debug logging to ease
the investigation.

In order to inspect a running node, modify the end_per_group/2
to not shut the node down or do it after an hour long sleep,
then run [1] to inspect the topology.

[2] can be used to run a Bunny-based version of effectively
the same test, which works as expected.

Pair: @lukebakken.

1. https://gist.github.com/michaelklishin/a7bcdc9a2be3540ac6d47297976864f6
2. https://gist.github.com/michaelklishin/9b7fb572f3d22749b982a49e3b3049d7
2020-09-15 18:49:48 +03:00
Luke Bakken 9b2cee8346 Use mirrored_supervisor for queue federation
Fixes #114
2020-09-14 15:20:05 -07:00
Michael Klishin cfcbee18fa Use a combination of cluster name and virtual host when detecting cycles
This makes it possible to federate messages between more than two
virtual hosts in a single cluster. Previously cycle detection would
drop messages that have traversed a "hop" (federation link)
in the same cluster.

Pair: @lukebakken.
2020-09-08 21:13:43 +03:00
Michael Klishin 0163068eed Extract cycle detection node identifier function 2020-09-08 15:34:12 +03:00
Michael Klishin 5eacab3b7b Coerce password to a binary
when obfuscating an upstream.

credentials_obfuscation:encrypt/1 requires a binary and
in some rare cases we've seen that the password value can
be passed in as a string.
2020-08-14 08:36:50 +03:00
Luke Bakken 5f436dae28 Update rabbitmq-components.mk 2020-08-04 08:41:47 -07:00
Jean-Sébastien Pédron 6bb45a5992 Update rabbitmq-components.mk 2020-07-30 12:06:53 +02:00
Luke Bakken 3ec36d2e9f Update rabbitmq-components.mk 2020-07-29 10:02:03 -07:00
dcorbacho a2b5d4433b Update erlang.mk 2020-07-21 14:32:24 +01:00
Michael Klishin be67c3d203 Update rabbitmq-components.mk 2020-07-21 13:12:47 +03:00
Michael Klishin 0992a2d434 Update rabbitmq-components.mk 2020-07-21 03:42:54 +03:00
dcorbacho 9355b957ed Revert drop of Exhibit B on MPL 2.0 2020-07-20 16:59:20 +01:00
dcorbacho 1d720569c5 Update LICENSE 2020-07-20 10:53:52 +01:00
Michael Klishin ef0b262d02 Update MPL2 license file, drop Exhibit B
and add a VMware copyright notice.

We did not mean to make this code Incompatible with Secondary Licenses
as defined in [1].

1. https://www.mozilla.org/en-US/MPL/2.0/FAQ/
2020-07-17 14:53:07 +03:00
Michael Klishin eb33152de4 Handle more pgroup_name_cluster_id values 2020-07-15 18:15:43 +03:00
Michael Klishin 81809c741a Handle cases where rabbitmq_federation.pgroup_name_cluster_id is undefined
it's highly unusual to see but being more defensive is
very easy in this case.
2020-07-15 17:35:13 +03:00
dcorbacho 26a4365188 Switch to Mozilla Public License 2.0 (MPL 2.0) 2020-07-12 22:52:53 +01:00
Jean-Sébastien Pédron 12c186d0b0 Update erlang.mk 2020-06-23 17:13:56 +02:00
Michael Klishin fcec5c61bd Log effective internal exchange check interval 2020-06-17 20:29:31 +03:00
Michael Klishin 25b712527d Handle more failures around one-off connection calls
Timeouts were not handled gracefully which could disrupt
the link for no good reason. Such one-off checks are
meant to be best effort and not performed particularly
frequently. Their failures should result in visible
log messages but not link restarts.

While at it, improve logging in related code paths.

Per discussion with @kjnilsson.
2020-06-17 18:12:15 +03:00
Michael Klishin 468c4b7cb7 Decrease internal exchange check frequency
it should not run more often than a connection timeout.
90s is a reasonable default interval for this check.
2020-06-17 14:15:23 +03:00
Michael Klishin 51d1939c12 Bump Recon to 2.5.1
for Erlang 23 compatibility of 'rabbitmq-diagnostics observer'

References zhongwencool/observer_cli#68.
2020-06-09 08:22:14 +03:00
Michael Klishin 9b9ca65690 Ditto 2020-04-07 04:01:21 +03:00
Michael Klishin 8d20112e1b Wording 2020-04-07 03:58:47 +03:00
Michael Klishin 3e1cd1d566 Introduce a new upstream parameter, resource-cleanup-mode
Set it to 'never' to make sure the link never removes its internal queue.
It's highly advisable to then specify queue expiration (TTL) to avoid
resource leaks.
2020-04-04 02:18:05 +03:00
Michael Klishin 834f0044bf Remove a log message we ended up not adding 2020-04-01 20:08:35 +03:00
Michael Klishin 2dffac287c Handle one more possible termination reason 2020-04-01 19:59:08 +03:00
Michael Klishin 758234687f Exchange federation: do not clean up internal queue on abnormal link termination
The queue is durable and might contain valuable data.
On the other hand, when federation link is shutting down
because policies have changed or the plugin is being disabled,
it makes sense to delete the resources it was using.

References #105.
2020-04-01 19:27:50 +03:00
Jean-Sébastien Pédron 80651fe605 README.md: Add Travis CI badge 2020-03-31 10:46:06 +02:00
Jean-Sébastien Pédron 57a4683bbd exchange_SUITE: Remove the `binding_recovery` testcase
It still fails way too often and I can't find the appropriate wait
condition.
2020-03-30 17:38:31 +02:00
Jean-Sébastien Pédron fab830b862 queue_SUITE: Wait for federation in `dynamic_plugin_stop_start` 2020-03-30 17:07:07 +02:00
Jean-Sébastien Pédron 75a80d7eb4 exchange_SUITE: Bump wait in several testcases 2020-03-30 14:52:57 +02:00
Jean-Sébastien Pédron f902ef5377 exchange_SUITE: Add a wait condition in `dynamic_plugin_stop_start` 2020-03-30 13:16:23 +02:00
Jean-Sébastien Pédron a845aad507 exchange_SUITE: Review all wait conditions
They should be closer to the expected states now. Hopefully this will
further reduce the number of transient test failures.
2020-03-27 15:25:07 +01:00
Jean-Sébastien Pédron 4fee32b989 exchange_SUITE: Improve wait condition in `binding_recovery`
We wait for the new parameter to be applied, not just any federation to
be ready.
2020-03-27 14:32:47 +01:00
Jean-Sébastien Pédron a34fe8d106 exchange_SUITE: Wait for federation on the correct node 2020-03-27 11:48:03 +01:00
Jean-Sébastien Pédron 7987940c9d exchange_SUITE: Bump a few more timeouts
Follow-up to commit 2e8cdd5ba6.
2020-03-26 17:55:05 +01:00
Jean-Sébastien Pédron 2e8cdd5ba6 exchange_SUITE: Throw an exception if wait_for_federation() times out
While here, increase two timers in the `binding_recovery` testcase.
Hopefully this will help in Concourse.
2020-03-26 17:32:42 +01:00
Jean-Sébastien Pédron c278947717 Update copyright (year 2020) 2020-03-10 16:10:02 +01:00
Gerhard Lazu d2a72bba64 Update rabbitmq-components.mk 2020-03-06 09:19:15 +00:00
Gerhard Lazu bfe5c1108c Update erlang.mk 2020-03-06 09:17:40 +00:00
Jean-Sébastien Pédron 0858bdff4a Travis CI: Update config from rabbitmq-common 2020-03-04 14:24:28 +01:00
Jean-Sébastien Pédron 3ef26b5b30 Travis CI: Update config from rabbitmq-common 2020-03-04 11:17:15 +01:00
Michael Klishin 1c7bf496c7 Transitionary step for OTP 23 [master] compatibility 2020-02-11 21:16:24 +03:00
Michael Klishin a07c6f49ad Transitionary step for OTP 23 [master] compatibility 2020-02-11 21:11:26 +03:00
Gerhard Lazu 346c7435da Update rabbitmq-components.mk 2020-02-11 15:31:44 +00:00
Michael Klishin 8b5bd10c43 (c) bump 2019-12-29 05:50:28 +03:00
Michael Klishin 407f095955 Error handling around command channel 2019-12-22 08:31:31 +03:00
Michael Klishin c0e73759e9 Merge branch 'cmd-channel' of https://github.com/velimir/rabbitmq-federation into velimir-cmd-channel 2019-12-22 04:23:30 +03:00
Luke Bakken aba6f430ab Merge branch 'lrb-delete-federation-exchange' 2019-12-12 11:58:03 -08:00
Michael Klishin d196789164 Merge pull request #99 from rabbitmq/delete-federation-exchange
Use vhost to delete federated exchange
2019-12-12 20:17:16 +03:00
Jean-Sébastien Pédron 5e566144ce Git: Ignore copied CLI 2019-12-12 13:30:17 +01:00
Jean-Sébastien Pédron ce7c0a5ac6 Update rabbitmq-components.mk 2019-12-12 13:14:54 +01:00
Luke Bakken d9183e1372 Filter by vhost for federated queues 2019-12-11 17:05:20 -08:00
Luke Bakken 1f123eed05 add test that fails for queue 2019-12-11 16:18:42 -08:00
Michael Klishin 1629d2ae1e Clarify the intent 2019-12-10 23:23:06 +03:00
dcorbacho db2ab26930 Use vhost to delete federated exchange
Avoids deleting several links with the same upstream, but on different vhosts
[#169936237]
2019-12-10 17:08:58 +01:00
Grigory Starinkin 7eacec1497 use separate channel for command operations
using `bind-nowait` and separate channel for bindings operations allows
to avoid message forwarding delays on the federation link
2019-11-07 07:21:07 +00:00
kjnilsson e59df101df Update rabbitmq-components.mk 2019-09-13 10:24:20 +01:00
kjnilsson 7326faf205 Update rabbitmq-components.mk 2019-09-04 10:30:57 +01:00
Jean-Sébastien Pédron 0af17a2333 Update erlang.mk 2019-08-29 20:45:08 +02:00
Diana Corbacho 69af5dfb02 Obfuscate credentials for federated queues and exchanges.
Credentials are also obfuscated in the ETS table of rabbit_federation_status
so they are not shown in plain text in a crash dump

[#167070941]
2019-08-16 13:35:00 +01:00