Commit Graph

3395 Commits

Author SHA1 Message Date
kjnilsson 9835a43b99 type name fix 2021-01-13 17:12:14 +00:00
kjnilsson ca53234ce3 Remove debug log message 2021-01-13 12:54:59 +00:00
kjnilsson 03ed11c055 bugfix 2021-01-13 12:09:47 +00:00
kjnilsson 2f0dba45d8 Stream: Channel resend on leader change
Detect when a new stream leader is elected and make stream_queues
re-send any unconfirmed, pending messages to ensure they did not get
lost during the leader change. This is done using the osiris
deduplication feature to ensure the resend does not create duplicates of
messages in the stream.
2021-01-13 12:09:44 +00:00
kjnilsson 61de203fc5 Remove use of non-deterministic function
Inside stream coordinator.
2021-01-13 10:40:44 +00:00
kjnilsson 9b8d38d2b9 stream coord: fixes
Make use of rabbit_stream_queue:update_stream_conf deterministic in the
state machine.
2021-01-12 15:09:02 +00:00
dcorbacho 9ef9dde6ce Apply retention policy in all osiris members 2021-01-12 12:18:13 +00:00
dcorbacho e5a2eaaa0d Update retention when only stream retention policy has changed
In any other case, the worker needs to be restarted
2021-01-12 12:18:13 +00:00
Karl Nilsson 015ec8b47f
Merge pull request #2681 from rabbitmq/stream-coordinator-restart-failure
Ensure leader is deleted from supervisor in case of re-election
2021-01-12 12:17:04 +00:00
kjnilsson dfa0775914 Remove unused value 2021-01-12 12:14:42 +00:00
Arnaud Cogoluègnes bf72683eb2
Add stream prometheus plugin 2021-01-11 16:49:56 +01:00
Jean-Sébastien Pédron a0cd2e5fd0
rabbit: Run plugins' boot steps during rabbit start/2
This restores the behavior prior the commit making `rabbit` closer to a
standard Erlang application.

Plugins are still actually started after rabbit is started (because they
depend on the `rabbit` application). Only the execution of their boot
steps was moved earlier.

With the behavior restored, it also means that a plugin's dependencies
are not started yet when its boot steps are executed.

V2: Move the maintenance mode reset before the plugin boot steps run.

V3: Add a `core_started` boot state. That state is reached at the end of
    the `rabbit` app start function. It indicates when the RabbitMQ core
    is started but the full service is not yet ready.

    We now use this state in direct connection code to determine if
    clients can open a direct connection. We have to do that because
    some plugins open a direct connection as part of their own startup
    (i.e. they can't wait for the `ready` boot state which comes later).
2021-01-08 12:31:25 +01:00
Arnaud Cogoluègnes cbd3c8dfdd
Merge branch 'master' into rabbitmq-stream-management 2021-01-04 09:50:47 +01:00
Michal Kuratczyk 6a81589c11 Expose `bypass_pem_cache` through rabbitmq.conf
Bypassing PEM cache may speed up TLS handshakes in some cases as described
here:
https://blog.heroku.com/how-we-sped-up-sni-tls-handshakes-by-5x
2020-12-17 16:53:14 +01:00
Arnaud Cogoluègnes e3bbdfe6df
Merge pull request #2676 from rabbitmq/rabbitmq-server-2667
Definition export: change user tags to a JSON array
2020-12-15 14:14:23 +01:00
dcorbacho 43ee7d45b5 Ensure leader is deleted from supervisor in case of re-election
If the supervisor returns {error, already_present} we can't assume
is the same pid stored as the process is dead
2020-12-14 15:37:53 +01:00
Michael Klishin 8abe0c4328
rabbitmq-upgrade(8): add missing commands
References rabbitmq/rabbitmq-website#1109
2020-12-13 20:58:15 +03:00
Arnaud Cogoluègnes c4d07467da
Merge branch 'master' into rabbitmq-stream-management 2020-12-09 12:00:56 +01:00
Michael Klishin 4ea9ce1c0b
Clarify what version will be the first to use this format 2020-12-09 12:48:56 +03:00
Michael Klishin ab0ade0e4c
Remove a stray debug logging line 2020-12-09 12:47:42 +03:00
Michael Klishin bcf6ac0515
Export user tags as a list
instead of a comma-separated list in a string.

When importing, both formats are now supported (as of
the previous commit).

Closes #2667.
2020-12-09 11:17:55 +03:00
Michael Klishin e4c37db689
Support importing users with arrays of tags
as opposed to a comma-separated binary.

Part of #2667.
2020-12-08 18:22:56 +03:00
Arnaud Cogoluègnes e5ed53c5e2
Merge branch 'master' into rabbitmq-stream-management 2020-12-08 11:41:22 +01:00
Luke Bakken 7a3bd539d3
Pass RABBITMQ_NODENAME via Windows service environment
Without this change using anything other than `rabbit` or the `rabbitmq-env-conf.bat` file will result in `erlang_dist_running_with_unexpected_nodename`

Follow-up to #2673

cc @dumbbell @michaelklishin
2020-12-07 12:03:06 -08:00
Luke Bakken 4306902309
Merge pull request #2673 from luos/allow_configuring_rabbitmq_base_multiple_times2
Windows service: allow overriding base service location
2020-12-07 10:04:00 -08:00
Lajos Gerecs 8fdbc222e3 allow configuring multiple rmq bases for multiple services
Currently RABBITMQ_BASE is always dynamically picked up from the
environment. This change would fix it at the time of configuration
of the service allowing multiple RabbitMQ services to be configured.
2020-12-05 22:13:00 +01:00
Arnaud Cogoluègnes 224e9914b2
Merge branch 'master' into rabbitmq-stream-management 2020-12-04 10:26:42 +01:00
Arnaud Cogoluègnes db5a5f57e8
Send shutdown message to non network/direct connection
Connections to the stream plugin does not have a type, so they can
trigger some function_clause errors. This was the case when trying to
close a connection from rabbit_connection_tracking module. The function
now falls back to a simple gen_server call to the connection process for
connections without a type.
2020-12-04 09:54:21 +01:00
kjnilsson 6fdb7d29ec Handle errors in crashing_queues_SUITE
As the connection may crash during the previous declaration and a caught
error would be returned in amqp_connection:open_channel/1 that wasn't
handled previously. Exactly how things fail in this test is most likely
very timing dependent and may vary.

Also fixes mqtt test where the process that set up a mock auth ETS table
was transient when an rpc timeout was introduced
2020-12-03 13:56:09 +00:00
Luke Bakken 8525a65970
Handle undefined args case
Fixes #2668
2020-12-02 12:33:02 -08:00
Luke Bakken ccf624211a
Add test that fails prior to the change for #2668 2020-12-02 12:33:02 -08:00
Arnaud Cogoluègnes 08891a734e
Merge branch 'master' into rabbitmq-stream-management 2020-11-30 09:42:54 +01:00
Arnaud Cogoluègnes ffd66027af
Merge pull request #2506 from rabbitmq/stream-timestamp-offset
Support timestamp offsets for stream consumers
2020-11-27 14:49:38 +01:00
Arnaud Cogoluègnes 43cfb45a74
Convert AMQP 091 timestamp to millisecond
For start offset in stream queue.
2020-11-27 14:47:36 +01:00
Arnaud Cogoluègnes 8f97ea400a
Start adding publishing dedup support for streams 2020-11-24 17:48:41 +01:00
Arnaud Cogoluègnes c8249a304f
Filter stream connections where metrics are disabled
This implied defining the protocol field in tracked connection to be
able to filter out non-stream connections.
2020-11-20 09:29:55 +01:00
kjnilsson ea7c9e9b61 QQ: Emit release cursor for empty basic gets
Else an application that polled an empty quorum queue frequntly using basic.get
would never result in a snapshot being taken and results in unlimited
log growth.
2020-11-19 15:59:51 +00:00
Arnaud Cogoluègnes 23d7e8114c
Introduce stream management plugin 2020-11-19 14:48:25 +01:00
dcorbacho f23a51261d Merge remote-tracking branch 'origin/master' into stream-timestamp-offset 2020-11-18 14:27:41 +00:00
Jean-Sébastien Pédron 778e8dad5c
rabbit_common: Remove the rabbitmq-github-actions Erlang.mk plugin
This is unused after the switch to the "monorepository".
2020-11-17 15:29:05 +01:00
Jean-Sébastien Pédron 47686ee1f0
Remove unused .github directories
They were valid until the switch to the "monorepository" when everything
was merged into a single Git repository.
2020-11-17 13:33:16 +01:00
Michael Klishin 5b8dba5e2f
Move issue and PR templates to monorepo root 2020-11-17 14:23:58 +03:00
Philip Kuryloski 8ff5273827 Remove unused function 2020-11-16 10:45:10 +01:00
kjnilsson d88b623c18 Use correct credit mode x-credit
When the x-credit consumer arg is defined Quorum Queues should use use
credit mode `credited` and not `simple_prefetch`.
2020-11-16 10:45:10 +01:00
Philip Kuryloski a1fe3ab061 Change repo "root" to deps/rabbit
rabbit must not be the monorepo root application, as other applications depend on it
2020-11-13 14:34:42 +01:00