Commit Graph

24667 Commits

Author SHA1 Message Date
Michael Klishin cf2c6099b3
Merge pull request #3263 from rabbitmq/rabbitmq-server-3260
Partially reintroduce locking to mirrored_supervisor
2021-08-05 15:33:06 +03:00
Michael Klishin c84115f102
Warn when lock could not be acquired successfully 2021-08-05 11:31:35 +03:00
Michael Klishin 22add3c8ca
Acquire locks when starting mirrored supervisor children
Unlike pg2, pg in Erlang 24 is eventually consistent. So this
reintroduces some of the same kind of locking mirrored_supervisor
used to rely on implicitly via pg2.

Per discussion with @lhoguin.

Closes #3260.

References #3132, #3154.
2021-08-05 00:39:16 +03:00
Michael Klishin 9a0f4b17cd
More mirrored supervisor logging 2021-08-04 16:58:44 +03:00
Michael Klishin 4f3d5e5e13
Mirrored supervisor: more logging at debug level 2021-08-04 15:56:45 +03:00
Philip Kuryloski 04371b0340 Bump a test timeout 2021-08-04 14:55:44 +02:00
Philip Kuryloski a3410904d6 Bump a test timeout 2021-08-04 13:06:29 +02:00
Philip Kuryloski aefb8ad753 bump a test timeout 2021-08-04 09:42:03 +02:00
Philip Kuryloski 35ab19c4a6 Bazel test timeouts 2021-08-03 17:06:12 +02:00
Philip Kuryloski 40954c1ca2 Update some bazel test timeouts 2021-08-03 15:51:02 +02:00
Philip Kuryloski c9e016d545 Remove duplicate compilation options in rabbit_common 2021-08-03 14:31:05 +02:00
Michael Klishin c12544f545
Update rabbitmq.conf.example header
mostly to test Mergify integration with @pjk25.
2021-08-03 13:19:52 +03:00
David Ansari 4b774db5c1 Use same threshold color for "Errors since boot" 2021-08-02 17:05:17 +02:00
David Ansari c99ee6961e Use same colorMode in all RabbitMQ-Stream panels
Co-authored-by: Gerhard Lazu <gerhard@lazu.co.uk>
2021-08-02 13:33:00 +02:00
David Ansari ea18c31288 Make RabbitMQ-Stream dashboard work via ConfigMap
Before this commit, importing the dashboard via ConfigMap as seen in
1eb1dc618e
didn't work because DS_PROMETHEUS variable was undefined in Grafana.

Related to https://github.com/rabbitmq/rabbitmq-server/pull/3250

Co-authored-by: Gerhard Lazu <gerhard@lazu.co.uk>
2021-08-02 13:12:48 +02:00
Gerhard Lazu 65afbb931b
Ensure RabbitMQ-Stream dashboard works correctly after import
This breaks the docker-compose integration, but we need to move away
from it anyways, the whole dev flow needs revisiting after our focus on
K8s.

$__rate_interval does not work with irate, dropping it in favour of 60s,
same as all other dashboards.

This is a follow-up to https://github.com/rabbitmq/rabbitmq-server/pull/3250

Thanks @ansd for mentioning about the post-import issues.

It was uploaded as https://grafana.com/api/dashboards/14798/revisions/3/download

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2021-07-30 13:53:02 +01:00
Gerhard Lazu 35a6369327
Restart stream-perf-test on-failure
This handles the scenario where rmq2 is not available, and
stream-perf-test exits with a non-zero exit code. Good spot @ansd!

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2021-07-30 11:25:36 +01:00
David Ansari 47d572908d Convert string to integer for ulimits.nofile
Before this commit:

> make overview metrics
services.rmq1.ulimits.nofile.hard must be a integer
make: *** [Makefile:68: overview] Error 15

Accoring to the docs
https://docs.docker.com/compose/compose-file/compose-file-v3/#ulimits
this must be an integer.
2021-07-30 09:46:38 +02:00
Gerhard Lazu 6f5c4118ea
Publish RabbitMQ-Stream dashboard to grafana.com
Removed the Dockerfile and slimmed down the Makefile, all of this is now
handled by https://github.com/rabbitmq/rabbitmq-server/blob/master/.github/workflows/oci.yaml
cc @Zerpet @pjk25

More details here (including the steps used to publish to grafana.com):
https://github.com/rabbitmq/release-engineering/issues/11#issuecomment-887627938

I don't want to hold up this PR, will invest in automating the
steps described in the previous link another time. Time to 🚀

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2021-07-29 19:34:05 +01:00
Philip Kuryloski cb3c93cc9e Increase a test timeout 2021-07-29 14:50:43 +02:00
Michael Klishin 1596f031ee
Merge pull request #3239 from rabbitmq/some-sort-of-unicode-support-in-env-files
Do not crash on badarg when env file has certain Unicode characters
2021-07-28 23:18:41 +03:00
Michael Klishin 6ef0c513e8
Merge pull request #3241 from rabbitmq/stream-absent
Handle 'absent' return value
2021-07-28 23:13:51 +03:00
dcorbacho 606429e90c Handle 'absent' return value 2021-07-28 21:48:37 +02:00
Michael Klishin 0617419e75
This *is* a Unicode character, U+060E 2021-07-28 19:04:16 +03:00
Ilya Khaprov 854f26ac18
Do not crash on badarg when env file has an 'unusual' unicode character
closes #3212
2021-07-28 16:39:17 +02:00
Philip Kuryloski c8cfd023a9
Merge pull request #3189 from rabbitmq/docker-library-rabbitmq-500
Bring the docker image up to date with the docker-library version
2021-07-28 11:15:27 +02:00
Philip Kuryloski 967f314f6f bazel timeouts 2021-07-28 10:38:04 +02:00
Philip Kuryloski a0a5bf3c01 Update peer discovery aws tests for docker image changes 2021-07-28 10:19:11 +02:00
Philip Kuryloski b8cabfe3dd bump test timeouts 2021-07-28 08:37:40 +02:00
Michael Klishin 2efc3d22fa
Merge pull request #3176 from rabbitmq/stream-error-handling
Better error handling for streams
2021-07-27 22:25:06 +03:00
Michael Klishin 9c2eeee595
Wording 2021-07-27 21:44:33 +03:00
Michael Klishin 2d3f31eb21
Merge pull request #3204 from rabbitmq/keep-state-and-data
Use keep_state_and_data
2021-07-27 21:39:06 +03:00
Gerhard Lazu 1b5c7e328e
Merge pull request #3230 from rabbitmq/fix-notify-limiter
Fix channel rate limiter for ack messages
2021-07-27 16:49:22 +01:00
Philip Kuryloski 68361d5d79 Extend the bazel timeout for unit_disk_monitor_mocks_SUITE 2021-07-27 16:02:10 +02:00
Philip Kuryloski 6967019495 Bazel test timeouts 2021-07-27 13:32:06 +02:00
Philip Kuryloski 2a6a9c786b Bazel test timeouts 2021-07-27 10:57:50 +02:00
Philip Kuryloski ee23cf1872 Bump test timeouts in bazel 2021-07-27 09:54:44 +02:00
Michael Klishin 5bfbe5b8b1
Merge pull request #3233 from britto/fix-ensure-safe-call-timeout-clause
Fix typo in ensure_safe_call_timeout/2
2021-07-26 23:48:39 +03:00
João Britto a27a9fd214 Adjust initial gen_server_call_timeout
to prevent `amqp_connection:start/2` from logging a warning when we use default
values.

The default for `connection_timeout` is already 60000. When we don't explicitly
set it to a lower value, `amqp_connection:maybe_update_call_timeout/2` adjusts
it to 70000 and logs a warning message, which may appear unexpected, especially
for users upgrading to 3.8.10+ with no config changes.

This change addresses that problem by pre-adjusting it to 70000, making it safe
by default and ensuring our default values don't conflict with each other.
2021-07-26 16:59:32 -03:00
João Britto 25b6ce1d9f Fix typo in ensure_safe_call_timeout
Matching on `ignored` instead of `ignore`, as described in the documentation of
net_kernel:get_net_ticktime/0.

https://erlang.org/doc/man/net_kernel.html#get_net_ticktime-0
2021-07-26 16:54:36 -03:00
Arnaud Cogoluègnes 4486584166
Fix channel rate limiter for ack messages 2021-07-26 13:57:44 +02:00
David Ansari 38a6ce9601 Fix link of stream docs 2021-07-26 12:03:06 +02:00
Philip Kuryloski abc8703fd8 Bump test timeouts in bazel 2021-07-26 11:09:09 +02:00
Philip Kuryloski 563a8f876a
Update bazel timeout for rabbitmq_amqp1_0 dialyze 2021-07-26 11:06:07 +02:00
Michael Klishin ca4866b964
Introduce rabbit_policy:list_op/2
and export rabbit_policy:sort_by_priority/1
2021-07-26 11:25:35 +03:00
Michael Klishin 23432f3130
Merge pull request #3229 from rabbitmq/stream-status-tracking
Fix function_clause error in tracking_status/2
2021-07-23 21:08:57 +03:00
David Ansari 6d968718c8 Fix function_clause error in tracking_status/2
Before this commit:

> ./sbin/rabbitmq-streams stream_status --tracking s1
Status of stream s1 on node rabbit@localhost ...
Error:
{:function_clause,
[{:rabbit_stream_queue, :"-tracking_status/2-fun-0-",
[:offsets, %{"s1-1" => 5}, []],
[file: 'src/rabbit_stream_queue.erl', line: 608]},
{:maps, :fold_1, 3, [file: 'maps.erl', line: 410]},
{:rabbit_stream_queue, :tracking_status, 2, []}]}

After this commit:

> ./sbin/rabbitmq-streams stream_status --tracking s1
Status of stream s1 on node rabbit@localhost ...
┌────────┬───────────┬───────┐
│ type   │ reference │ value │
├────────┼───────────┼───────┤
│ offset │ s1-1      │ 51    │
└────────┴───────────┴───────┘
2021-07-23 19:34:31 +02:00
Philip Kuryloski a9688624a8 Bump bazel test timeouts for some suites 2021-07-23 13:59:17 +02:00
Philip Kuryloski 97946f3666 bump a test timeout in bazel 2021-07-23 13:38:53 +02:00
Philip Kuryloski 7d3d601d88 Bump test timeouts
A recent release of buildbuddy eliminated a timeout extension that
they had applied in the past. Now that they honor timeouts exactly,
we have had to adjust the timeout for many tests.
2021-07-23 13:36:09 +02:00
Philip Kuryloski 6628df33ef Increase some bazel test timeouts 2021-07-23 12:57:16 +02:00
Jean-Sébastien Pédron ac7df333b6
Revert "rabbitmq-dist.mk: Install CLI scripts as part of the build"
This has the unfortunate side effect of causing a rebuild of all
applications every time. I need to figure out another place to build and
install the CLI during build time (instead of as part of the dist
target).

This reverts commit 4322cca66e.
2021-07-23 10:46:56 +02:00
Philip Kuryloski 04e6121b08 Increase the test timeout in bazel for unit_file_handle_cache_SUITE 2021-07-23 10:44:51 +02:00
Philip Kuryloski ae955d44c6 Avoid building the cli scripts in rabbitmq_stream_common
This isn't an issue in the monorepo, but causes problems externally
projects using rabbitmq-components.mk
2021-07-23 10:40:20 +02:00
Philip Kuryloski f95fc8aa0c Increase some suite timeouts in bazel 2021-07-23 09:43:05 +02:00
David Ansari e3ed9c21b0 Fix list_stream_publishers additional usage output 2021-07-22 18:13:07 +02:00
Philip Kuryloski 0443c4ab8a Increase the timeout for the queue_master_location_SUITE in bazel 2021-07-22 16:07:06 +02:00
Jean-Sébastien Pédron 9258ff1197
Merge pull request #3214 from rabbitmq/fix-bugs-reported-by-release-handler
Fix bugs reported by release handler
2021-07-22 11:20:10 +02:00
Philip Kuryloski b26289cb47 Adjust rabbitmq_prometheus test suite timeouts in bazel 2021-07-22 11:00:14 +02:00
Jean-Sébastien Pédron 8cbdbb87a5
rabbit_web_dispatch_sup: Fix modules list in supervisor spec
This fixes a bug causing a crash of the release handler.
2021-07-22 10:21:37 +02:00
Jean-Sébastien Pédron 23a04c2692
rabbit_mgmt_agent_sup_sup: Fix worker type for the pg group
It must be a worker (the default type), not a supervisor. This fixes a
bug causing a crash of the release handler.
2021-07-22 10:21:37 +02:00
Jean-Sébastien Pédron 80b428d5d3
rabbit_runtime: Find erl(1) in the ERTS bin directory
erl(1) might not be in the directory pointed by init's root argument.
This is the case with Erlang releases built with relx for instance.
2021-07-22 10:21:37 +02:00
Jean-Sébastien Pédron 4322cca66e
rabbitmq-dist.mk: Install CLI scripts as part of the build
... instead of the `dist` target. This way, the Erlang release can use
them.
2021-07-22 10:21:33 +02:00
Michael Klishin 2d8aad78bf
Merge pull request #3216 from rabbitmq/mk-quorum-queue-oldest-known-message-command
New aux quorum queue command: oldest_entry_timestamp
2021-07-21 19:04:58 +03:00
Michael Klishin a0ad332727
New aux quorum queue command: oldest_entry_timestamp
Per discussion with @kjnilsson
2021-07-21 18:28:39 +03:00
Gerhard Lazu 66ef8adfc8
Fix accept dependency in rabbitmq_prometheus
It's a runtime dependency, not a build dependency.

This is a fix and should be backported to v3.9.x, after rc.2 and just
before the final release. Would you disagree @dumbbell?

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2021-07-21 13:38:54 +01:00
Iliia Khaprov 53d67fda1f
Merge pull request #3205 from rabbitmq/send-www-authenticate-when-basic-auth-present
Send www-authenticate header when basic auth present but it's wrong
2021-07-21 11:19:22 +02:00
Karl Nilsson f38c023aa8
Correct Publish frame docs 2021-07-21 09:13:55 +01:00
Philip Kuryloski 6fec1faea5 Additional rpc -> erpc updates 2021-07-21 09:21:41 +02:00
Philip Kuryloski 5bc25fb2ff Fix accidental test case skip when not using mixed versions 2021-07-21 08:35:00 +02:00
Michael Klishin 0d06f34c66
rabbit_stream_reader: convert most log messages to debug ones 2021-07-21 01:38:13 +03:00
Johan Rhodin 6d0a432888
Fix link for stream.html 2021-07-20 16:35:24 -05:00
Ilya Khaprov 39693cfb07
Send www-authenticate header when basic auth present but it's wrong
close #3181
2021-07-20 21:44:36 +02:00
Gabriele Santomaggio 71c355c963
Fix links 2021-07-20 17:42:56 +02:00
Michael Klishin c973a93445
Merge pull request #3203 from rabbitmq/remove_project_maturity
Remove project maturity warning
2021-07-20 18:36:34 +03:00
David Ansari 644335de86 Use keep_state_and_data 2021-07-20 16:11:22 +02:00
Philip Kuryloski 0593e8307e Increase timeouts in aws peer discovery integration suite 2021-07-20 11:41:40 +02:00
Gabriele Santomaggio de0f7952e9 Add resources link
to client guide line and streams internals
2021-07-20 09:55:25 +02:00
Gabriele Santomaggio 84f51d7926 Remove project maturity warning 2021-07-20 09:00:11 +02:00
Michael Klishin 5c3f456131
Improve a Cuttlefish validator message 2021-07-20 08:34:40 +03:00
Michael Klishin e20bca44cc
rabbit_stream_reader: these should not be logged at info level 2021-07-20 00:55:40 +03:00
Michael Klishin c1e3710140
Squash a compiler warning 2021-07-20 00:55:40 +03:00
Michael Klishin 4ba8d05535
Merge pull request #3202 from rabbitmq/validate-amqp0-9-1-queue-name-length-in-management-ui
Validate amqp0.9.1 queue name length in management UI
2021-07-19 21:57:41 +03:00
Michael Klishin 70ebefc0f2
rabbit_stream: ignore escript and sbin 2021-07-19 20:26:16 +03:00
Michael Klishin 532d076907
Merge pull request #3194 from rabbitmq/stream-reader-state-timeouts
Add stream reader state timeouts
2021-07-19 20:16:04 +03:00
David Ansari 863b899079 Remove TEST macro
since it fails with Bazel.

As discussed with @pjk25, let's set this value via application env,
make it configurable to the test, but not configurable to the user.
2021-07-19 16:42:54 +02:00
Ilya Khaprov c5665d6da9
Validate AMQP0.9.1 queue name length when creating queue directly or via shovel in Management UI. close #3074 2021-07-19 14:57:50 +02:00
David Ansari 13b03b8530 Remove unused variable warning 2021-07-19 14:36:22 +02:00
Philip Kuryloski d6399bbb5b
Mixed version testing in bazel (#3200)
Unlike with gnu make, mixed version testing with bazel uses a package-generic-unix for the secondary umbrella rather than the source. This brings the benefit of being able to mixed version test releases built with older erlang versions (even though all nodes will run under the single version given to bazel)

This introduces new test labels, adding a `-mixed` suffix for every existing test. They can be skipped if necessary with `--test_tag_filters` (see the github actions workflow for an example)

As part of the change, it is now possible to run an old release of rabbit with rabbitmq_run rule, such as:

`bazel run @rabbitmq-server-generic-unix-3.8.17//:rabbitmq-run run-broker`
2021-07-19 14:33:25 +02:00
Philip Kuryloski 0f4cf2755d Increase a timeout for flakiness sake 2021-07-19 14:24:46 +02:00
Michael Klishin a4cc096728
rabbit_vhost: introduce all_tagged_with/1 2021-07-18 08:33:29 +03:00
Michael Klishin ef14bf067e
ctl add_vhost: propagate tags if no description is given 2021-07-18 08:28:53 +03:00
Philip Kuryloski 0a78484999 Make things a little more consistent between per_*_limit suites 2021-07-16 14:40:51 +02:00
Philip Kuryloski 4f514f435b Try to reduce flakes in per_user_connection_channel_limit_partitions_SUITE 2021-07-16 14:32:35 +02:00
Michael Klishin 9a023b2547
Event exchange: explicitly translate two more known event types
new in 3.9
2021-07-16 08:09:19 +03:00
João Britto 5c69b91e15 Remove amqp_ssl:add_verify_fun_to_opts/2,3
These functions have become no-op after d76234f.

The maybe_add_verify_fun/3 function has become unreachable and was removed too.
2021-07-15 17:28:49 -03:00
David Ansari 4053f729dd Rename STATE_TIMEOUT to CONNECTION_NEGOTIATION_STEP_TIMEOUT 2021-07-15 20:56:56 +02:00
David Ansari 694804d0d2 Add timeout reason to log message 2021-07-15 20:53:46 +02:00
David Ansari 3964da37b4 Close TCP connection when stream reader times out
Add state timeouts.
If the client takes more than 10s for a single step in the authentication
protocol, make the server close the TCP connection.

Also close the TCP connection if the server times out in state
close_sent. That's the case when the client sends an invalid command
(after successful authentication), the server requests the client to
close the connection, but the client doesn't respond anymore.
2021-07-15 19:29:24 +02:00
Philip Kuryloski 97e8037b80 Replace some static sleeps in tests with dynamic waits
This should help with flakiness
2021-07-15 16:42:14 +02:00
Philip Kuryloski 88b091ae2e Fix shard_count for rabbit_stream_queue_SUTIE 2021-07-15 10:46:26 +02:00
dcorbacho e65ba8347c Fix delete_replica bug
It caused a lot of flakiness on the rabbit_stream_queue_SUITE, both on `delete_replica`
and `delete_last_replica` test cases.
2021-07-14 17:18:20 +02:00
dcorbacho 6052ecdc9c Split cluster_size_3_parallel in two groups
Faster to test locally the flaky tests and isolate them
2021-07-14 17:18:20 +02:00
dcorbacho 6610c3443d Include queue name on log message 2021-07-14 17:17:28 +02:00
dcorbacho e58ee79b4a Avoid double spawn on coordinator and link spawned processes
If phases are not linked, they can continue running *after* the coordinator has
been stopped. This could (or not) lead to unexpected behaviours,
but for sure it makes debugging harder.
2021-07-14 17:17:28 +02:00
dcorbacho 0103ab9723 Capture delete_stream errors and return them as protocol errors
Clients get the reason of the failure, instead of a generic internal error
2021-07-14 17:17:28 +02:00
dcorbacho 9e128b72b4 Set info/2 timeout to infinity to list connections
Default gen_server timeout is not enough to list busy connections.
Setting it to infinity allows the caller to decide the timeout,
as classic queues do. The `emit_info` function's family sets its
own timeout for all the cli commands.
2021-07-14 17:16:22 +02:00
Philip Kuryloski 8a339aae6c Fixup rabbitmq_federation:queue_SUITE for mixed version testing 2021-07-13 18:05:04 +02:00
Michael Klishin 29bb9c5b0c
Merge pull request #3175 from processone/proxy_protocol_tls_info
Extract TLS informations that are delivered in PROXY protocol frame
2021-07-13 15:08:40 +03:00
Philip Kuryloski 860333a088
Merge pull request #3177 from rabbitmq/stream-commit-offset-becomes-store-offset
Use "store" instead of "commit" for offset tracking
2021-07-13 12:11:37 +02:00
Philip Kuryloski ff619eea6a Mark per_vhost_connection_limit_SUITE as flaky 2021-07-12 22:21:02 +02:00
Philip Kuryloski 6edb1ea016 Use bazel-erlang 1.1.0 2021-07-12 19:06:20 +02:00
Philip Kuryloski 3102946d45 Use bazel-erlang 1.1.0 2021-07-12 19:05:14 +02:00
Michael Klishin 7de491fd82
Merge pull request #3187 from rabbitmq/less-chatty-mqtt
Change a log line from INFO to DEBUG
2021-07-12 19:07:29 +03:00
Philip Kuryloski 8f9de08de7 Also assert no missing suites for all other deps 2021-07-12 18:05:55 +02:00
Michal Kuratczyk 41922b96cf
Change a log line from INFO to DEBUG
This line is printed on every new MQTT connection which leads to very chatty logs when there is a lot of connections. Given that the way MQTT uses vhosts is generally static (once set up, always the same for all connections), I think this can be a debug message instead.
2021-07-12 16:50:25 +02:00
Philip Kuryloski 05111b3653 Add an assertion that no deps/rabbit common test suites are missing
in bazel
2021-07-12 15:55:56 +02:00
Philip Kuryloski 71ae7e7d14
Merge pull request #3186 from rabbitmq/use-bazel-erlang-native-sharding
Use bazel erlang native sharding
2021-07-12 12:36:37 +02:00
Philip Kuryloski 3eac3cf8a8 Remove unused load statements from bazel files 2021-07-12 12:10:26 +02:00
Philip Kuryloski 923d87f847 Avoid using a duplicate group name in rabbitm_stream_queue_SUITE
Since bazel-erlang doesn't support this with sharding
2021-07-09 16:26:56 +02:00
Philip Kuryloski 8421100008 Use bazel-erlang semi-automatic suite sharding 2021-07-09 10:05:16 +02:00
tomyouyou f55a3e1b08
'start_ssl_listener/4' has wrong parameter orfer
The 'ConcurrentConnsSupsCount' should be the last paramter
2021-07-08 18:12:31 +08:00
Arnaud Cogoluègnes 8ddff0faf8
Use "store" instead of "commit" for offset tracking 2021-07-08 11:28:33 +02:00
Philip Kuryloski 19b8fa1409 Mark //deps/rabbitmq_stomp:python_SUITE as flaky 2021-07-08 11:22:15 +02:00
Arnaud Cogoluègnes 7cb2645283
Replace commit with store for offset persistence
In stream protocol. Commit have a strong consistency connotation,
which is not actually enforced by the offset persistence
mechanism.
2021-07-08 10:32:04 +02:00
Philip Kuryloski 2fc112e29c Correct some test cleanup in rabbit_mgmt_http_SUITE 2021-07-07 18:05:55 +02:00
Michael Klishin 00f647cf04
Channel tracking: reduce log verbosity
When a connection is closed, we should not log that all of its
channels are closed every single time.

Per discussion with @mkuratczyk
2021-07-06 18:54:49 +03:00
Arnaud Cogoluègnes 76c4c2792b
Merge pull request #3171 from rabbitmq/stream-add-uncompressed-length-to-sub-batch-format
Add uncompressed size in stream sub-entry
2021-07-06 14:45:45 +02:00
Michael Klishin 2ea732b7ef
Drop a couple of non-essential debug log messages
Note that we still have epmd recovery-related log messages.

Per discussion with @mkuratczyk.
2021-07-06 14:18:32 +03:00
Michael Klishin 1bdecdb2c4
Introduce a few new rabbit_policy listing functions 2021-07-06 05:38:51 +03:00
Arnaud Cogoluègnes f9867f1f82
Add uncompressed size field for pub ids generation 2021-07-05 16:22:21 +02:00
Karl Nilsson 284809e750
Merge pull request #3170 from rabbitmq/stream-flaky
Fix restart of stream coordinator when there are no stream queues
2021-07-05 15:03:45 +01:00
Philip Kuryloski da6da8d6c7 Clear memory alarms on all nodes in the memory_alarm_rolls_wal test
If the alarm is triggered directly with `rabbit_alarm` it has to be
cleared on all nodes
2021-07-05 15:57:56 +02:00
dcorbacho deaa42ecac Fix restart of stream coordinator when there are no stream queues
Recovering from an existing queue is fine but if a node is restarted when
there are no longer stream queues on the system, the recovery process won't
restart the pre-existing coordinator as that's only performed on queue recovery.
The first attempt to declare a new stream queue on this cluster will crash with
`coordinator unavailable` error, as it only restarts the local coordinator
and not the whole ra cluster, thus lacking quorum.

Recovering the coordinator during the boot process ensures that a pre-existing
coordinator cluster is restarted in any case, and does nothing if there was
never a coordinator on the node.
2021-07-05 15:34:05 +02:00
Philip Kuryloski dbb09f1bf5 Add ?awaitMatch/4 to rabbit_assert.hrl
?awaitMatch/4 is the same as ?awaitMatch/3, but where the polling
interval is configurable
2021-07-05 15:21:27 +02:00
Paweł Chmielowski d5daf7598b Extract TLS informations that are delivered in PROXY protocol frame 2021-07-05 13:29:59 +02:00
Michael Klishin 65ccf7ce42
Mirrored supervisor: make it easier to keep track of group membership changes
in the logs.

Referenes #3148.
2021-07-05 14:02:37 +03:00
Philip Kuryloski f8c09121c1 Revert "Add ?awaitMatch/4 to rabbit_assert.hrl"
This reverts commit 3bb9d0084d.
2021-07-05 11:46:51 +02:00
Philip Kuryloski 3bb9d0084d Add ?awaitMatch/4 to rabbit_assert.hrl
?awaitMatch/4 is the same as ?awaitMatch/3, but where the polling
interval is configurable
2021-07-05 11:40:42 +02:00
Philip Kuryloski 973a8c5189 Adjustments to hopefully make rabbitmq_federation suite less flaky
and print more informative errors in a failure case
2021-07-05 11:26:56 +02:00
Philip Kuryloski 390a00b828 Handle feature flag enablement failure more gracefully in test setup 2021-07-05 11:22:38 +02:00
Arnaud Cogoluègnes be9cc22dc1
Add uncompressed size in stream sub-entry 2021-07-02 16:19:47 +02:00
Michael Klishin 5f8f57dac6
Drop a few exclusive terms still present in man pages
References #3159
2021-07-02 14:12:43 +03:00
Michael Klishin 59942f89d6
Do not update virtual host tracing state when it's not necessary
Primarily to avoid refreshing state of all channels.

Per discussion with @ansd
2021-07-01 22:44:15 +03:00
Gerhard Lazu ef4303a486
Merge pull request #3157 from rabbitmq/stream-protocol-counters
Add specific stream protocol counters to track protocol errors
2021-07-01 17:56:14 +01:00
Arnaud Cogoluègnes f1f733445e
Check publisher still exists on osiris_written event 2021-07-01 10:47:58 +02:00
Michael Klishin c20cb568a1
Acquire locks before starting or restarting dynamic Shovels
When N cluster nodes are configured to import the same definition
file that includes Shovel declarations, there is a natural race
condition that can lead to Shovels being started on multiple
nodes under the same name.

See #3154 for the background.
2021-06-30 21:17:38 +03:00
Philip Kuryloski 07220e4e13 Fix handling of `broker_with_plugins` config "flag" under bazel
If unset, the flag actually has no effect, rather than assumed to be false
2021-06-30 16:20:11 +02:00
Michael Klishin a52a424578
Merge pull request #3163 from rabbitmq/mk-epmd-starter-loop-timeout
Make sure not to wait for epmd-starter subprocess forever
2021-06-30 16:06:38 +03:00
Michael Klishin 4f01d450e0
Make sure not to wait for epmd-starter subprocess forever
Per discussion in #3162
2021-06-30 15:20:09 +03:00
dcorbacho b636ad2565 Rename protocol error counters to _total 2021-06-30 12:46:41 +02:00
Philip Kuryloski 1b92fadd80 Skip additional quorum_queue_SUITE cases under mixed versions 2021-06-30 12:41:59 +02:00
Philip Kuryloski d086af8070 Reduce test case flakyness in quorum_queue_SUITE
for the clustered/cluster_size_3/confirm_availability_on_leader_change case
2021-06-30 10:05:23 +02:00
Philip Kuryloski ef9647671f Introduce dynamic wait in parts of the quorum_queue_SUITE
to help with test flakes
2021-06-29 18:32:53 +02:00
Philip Kuryloski e67575441a Use bazel-erlang 1.0.0 2021-06-29 17:16:25 +02:00
Philip Kuryloski 77073529a6 Use bazel-erlang 1.0.0 2021-06-29 16:49:53 +02:00
Philip Kuryloski a8ae32e2f7 Skip an additional quorum_queue_SUITE case in mixed versions 2021-06-29 16:43:19 +02:00
Philip Kuryloski 5c7f579ccd rabbitmq_stream_management bazel fix 2021-06-29 13:26:39 +02:00
Philip Kuryloski b807db3fd0 Update rabbitmq_stream deps in bazel
for changes occurring in 58e36b6417
2021-06-29 13:07:06 +02:00
dcorbacho 58e36b6417 Add specific stream protocol counters to track protocol errors 2021-06-29 12:50:00 +02:00
Michael Klishin cf147ebfe5
Merge branch 'master' into mk-stricter-stop-start-assertions-in-quorum-queue-suite 2021-06-29 12:49:31 +03:00
Michael Klishin a1ab7452ef
Improve assertions in a QQ suite test 2021-06-29 12:16:23 +03:00
Michael Klishin a3e98c2a7f
Merge pull request #3137 from rabbitmq/faster-binding-recovery
Faster implementation of rabbit_binding:recover/0
2021-06-29 12:09:27 +03:00
Philip Kuryloski 3cb8ff1ab9 Mixed version testing skip updates 2021-06-29 10:49:06 +02:00
Gerhard Lazu 3078d05ddc
Merge pull request #3136 from rabbitmq/global-gauges
Gauges for global publishers & consumers metrics
2021-06-29 09:20:59 +01:00
Philip Kuryloski 9c81acf524 Mark //deps/rabbit:vhost_SUITE as flaky 2021-06-29 09:59:37 +02:00
dcorbacho c9305d948a
Use number of publishing channels as global publishers in amqp091 2021-06-29 08:10:42 +01:00
dcorbacho 228ea40e34
Gauges for global publishers & consumers metrics 2021-06-29 08:10:42 +01:00
Michael Klishin 2867b0747b
Merge pull request #3156 from rabbitmq/rabbitmq-server-3155
Deduplicate parsed list of plugin directories
2021-06-28 22:04:22 +03:00
Michael Klishin bed64f2cc9
Reduce priority_queue_SUITE to single node tests
Other tests (that produce flakes) arguably test classic mirrored
queues, a deprecated feature reasonably well
covered in other suites.

Per discussion with @gerhard.
2021-06-28 21:59:16 +03:00
Jean-Sébastien Pédron 24c29733a9
rabbit_prelaunch_early_logging: Don't call dbg:tracer() if unused
Indeed, if there is no module to trace, don't initialize the debugger.
2021-06-28 18:32:57 +02:00
Michael Klishin 1d3d8623de
CLI tests: make mock plugin declare compatibility with 3.8 through 3.10 2021-06-28 18:39:40 +03:00
Michael Klishin 17a1f4fa3e
Deduplicate parsed plugin directory list in one more place
References #3155
2021-06-28 18:38:22 +03:00
Michael Klishin d0ad77f10d
Deduplicate parsed list of plugin directories
See #3155 for context. It is also possible that someone might
override the path with duplicate values.

Note that the list of directories will lose its original
ordering. That should be fine as we expect unique plugin
paths: it is not a "preference list" (ordering should
not matter).

Closes #3155
2021-06-28 17:45:13 +03:00
Michael Klishin 9a15d787ac
Merge pull request #3152 from rabbitmq/rabbitmq-server-3149
Canonicalize a few more paths in the startup banner
2021-06-28 16:30:02 +03:00
Michael Klishin 4e787a7f6b
Log locations are already canonicalized 2021-06-28 14:59:15 +03:00
Arnaud Cogoluègnes 8bc39d6e50
Merge pull request #3151 from rabbitmq/stream-reader-statem
Convert rabbit_stream_reader into state machine
2021-06-28 12:52:37 +02:00
Michael Klishin 62e7785a48
Canonicalize a few more paths in the startup banner
References #3149
2021-06-28 13:48:13 +03:00
Michael Klishin a19a0f924a
quorum_queue_SUITE: don't unconditionally skip node_removal_is_not_quorum_critical
Unintentionally introduced in a3c97d491f
2021-06-28 13:02:55 +03:00
David Ansari b145684b1b Remove useless ensure_stats_timer calls
Calling ensure_stats_timer after init_stats_timer and reset_stats_timer
is enough.

The idea is to call stop_stats_timer before hibernation and
ensure_stats_timer on wakeup. However, since we never call
stop_stats_timer in rabbit_stream_reader, we don't need to call
ensure_stats_timer on every network activity.
2021-06-28 11:27:45 +02:00
Philip Kuryloski 818818cdd6 Mark rabbitmq_stream_management suite not mixed version compatible
Since streams are 3.9 only we would not expect them to be used until
the cluster has been fully upgraded
2021-06-28 09:35:35 +02:00
Philip Kuryloski 8e88c6e967 Update flaky setting for rabbit_stream_queue_SUITE 2021-06-28 09:35:27 +02:00
David Ansari 896d879f8d Fix heartbeater exception exit
Before this commit test AlarmsTest.diskAlarmShouldNotPreventConsumption
of the Java client was failing.
When executing that test, the server failed with:

2021-06-25 16:11:02.886935+02:00 [error] <0.1301.0>     exception exit: {unexpected_message,resume}
2021-06-25 16:11:02.886935+02:00 [error] <0.1301.0>       in function  rabbit_heartbeat:heartbeater/3 (src/rabbit_heartbeat.erl, line 138

because the heartbeater was tried to be resumed without being paused
before.

Above exception exit also happens on master branch when executing this
test. However, the test falsely succeeds on master because the following FIXME was
never implemented:
8e569ad8bf/deps/rabbitmq_stream/src/rabbit_stream_reader.erl (L778)
2021-06-26 14:04:05 +02:00
David Ansari 8c4e2e009d Log at debug level when state machine terminates 2021-06-26 14:02:00 +02:00
Philip Kuryloski 8e569ad8bf Mark dynamic_ha_SUITE as flaky 2021-06-25 17:33:19 +02:00
David Ansari 81ee05f9ce Convert rabbit_stream_reader into state machine
This is pure refactoring - no functional change.

Benefits:
* code is more maintainable
* smaller methods (instead of previous 350 lines listen_loop_post_auth function)
* well defined state transitions (e.g. useful to enforce authentication protocol)
* we get some gen_statem helper functions for free (e.g. debug utilities)

Useful doc: https://ninenines.eu/docs/en/ranch/2.0/guide/protocols/
2021-06-25 15:07:34 +02:00
Philip Kuryloski 6a42a92839 Mark additonal suites flaky 2021-06-25 12:09:06 +02:00
David Ansari ff174eaa5f Add behaviour declaration for rabbit_stream_metrics_gc
since it implements a gen_server.
2021-06-25 11:57:14 +02:00
Philip Kuryloski a3c97d491f Update additional test skipping for 3.8/3.9 mixed versions 2021-06-25 11:17:46 +02:00
Philip Kuryloski f0941c806d Add is_mixed_versions/0
So that the check can be performed before run_setup_steps on a Config
2021-06-25 11:06:17 +02:00
Michael Klishin 9aa1389713
Startup banner wording: SSL => TLS 2021-06-25 01:58:29 +10:00
Gerhard Lazu 2853ec9286
Use emu_flavor in commercial startup log too
cc @dumbbell @mkuratczyk

Part of https://github.com/rabbitmq/rabbitmq-server/pull/2777

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2021-06-24 16:43:11 +01:00
Gerhard Lazu fa0364d37f
Merge pull request #2777 from rabbitmq/banner-with-otp-version
Add OTP version to the startup banner
2021-06-24 16:11:16 +01:00
Michael Klishin 07740be9d3
Merge branch 'SzumiecM-master' 2021-06-24 23:31:43 +10:00
Michael Klishin 471294af67
AuthN/Z backend HTTP: rename two new settings, add tests 2021-06-24 23:26:13 +10:00
Philip Kuryloski 1c5cd716e5 Mark //deps/rabbit:rabbitmqctl_integration_SUITE as flaky 2021-06-24 15:18:27 +02:00
Philip Kuryloski cbdd80830c Mark //deps/rabbit:dynamic_qq_SUITE as flaky 2021-06-24 14:55:43 +02:00
Philip Kuryloski a145c35aaa
Marking of flaky tests 2021-06-24 21:59:10 +10:00
Philip Kuryloski 7112a1e7ab
Mark deps/rabbitmq_stream:commands_SUITE as flaky 2021-06-24 21:59:10 +10:00
Philip Kuryloski a798b51739
Fixup deps/rabbitmq_management/BUILD.bazel
"True" -> True
2021-06-24 21:59:10 +10:00