Commit Graph

629 Commits

Author SHA1 Message Date
Luke Bakken 6de0656fce
Add redbug library
`redbug` compliments `recon` well and has a better tracing interface IMHO.
2022-01-11 16:22:05 -08:00
Luke Bakken 95a60fc3be
Replace one use of filelib:is_regular/1
This specific case is called multiple times by the Prometheus plugin. It eventually calls `file:read_file_info/1` which leaks on Windows

See #3936
2022-01-11 09:08:34 -08:00
Luke Bakken fd781441f3
Fix issue with fsutil
Fsutil has language-specific messages. Fix by using powershell.exe instead.

Follow-up to #3895

Reported here:
https://groups.google.com/g/rabbitmq-users/c/ypk51AtmrSM
2022-01-08 05:37:41 -08:00
Karl Nilsson 82470e9d1c Stream coordinator: handle machine_version command 2022-01-07 12:30:03 +00:00
Karl Nilsson 9a5d0f9d85 Make stream coodinator machine versioned
In order to retain deterministic results of state machine applications
during upgrades we need to make the stream coordinator versioned such
that we only use the new logic once the stream coordinator switches to
machine version 1.
2022-01-07 12:11:11 +00:00
tomyouyou e5ccf267ff 'rabbit_stream_coordinator:select_leader' runs with wrong comparison
The list consists of candidates which is a tuple {node, tail}, and the tail is made of {epoch, offset}.
While the 'select_leader' think the tail is made of {offset, epoch}. 

Suppose there are two candidates:
[{node1,{1,100}},{node2,{2,99}}] 

It selects node1 as the leader instead of node2 with larger epoch.
2022-01-07 10:06:16 +00:00
Michael Klishin 5a07f728c3
Merge pull request #3956 from tomyouyou/record_dist_ip
Distribution listener IP address is hardcoded in listener metadata
2022-01-05 17:38:35 +04:00
tomyouyou fac249b755
The wrong distribution listener IP address is recorded when it is configured.
Add an item to the configuration file(/etc/rabbitmq/rabbitmq.config):
{kernel, [{inet_dist_use_interface, {8193,291,0,0,0,0,0,1}}]}

Use the netstat command to check the IP address of the distribution port(25672):
netstat -anp | grep 25672
tcp6       0      0 2001:123::1:25672       :::*                    LISTEN      2075/beam.smp

However, 'rabbitmqctl status' shows:
...
Interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication
...
2022-01-05 16:03:24 +08:00
Michael Klishin c75ac14efa
Merge pull request #3936 from rabbitmq/lukebakken/fix-all-read-file
Fix all uses of file:read_file/1
2022-01-04 12:46:34 +04:00
Philip Kuryloski 12f58beb04
Merge pull request #3941 from rabbitmq/bazel-run-dev-broker
Disable +deterministic in compilation_mode dbg under bazel
2022-01-04 09:30:15 +01:00
Luke Bakken 7f0285834e
Fix all uses of file:read_file/1
This is to address another memory leak on win32 reported here:

https://groups.google.com/g/rabbitmq-users/c/UE-wxXerJl8

"RabbitMQ constant memory increase (binary_alloc) in idle state"

The root cause is the Prometheus plugin making repeated calls to `rabbit_misc:otp_version/0` which then calls `file:read_file/1` and leaks memory on win32.

See https://github.com/erlang/otp/issues/5527 for the report to the Erlang team.

Turn `badmatch` into actual error
2022-01-03 11:33:36 -08:00
dcorbacho 0bd8d41b72 Skip new import testcase on mixed environments 2022-01-03 17:37:06 +01:00
Philip Kuryloski 70ef6b6984 Disable +deterministic in compilation_mode dbg under bazel
This allows compiling and reloading code from the erlang shell when
running the broker with `bazel run -c dbg broker`
2022-01-03 12:38:06 +01:00
Michael Klishin 4993fefa08
#3925 follow-up: add a rabbit_common Bazel dep 2021-12-28 01:30:07 +03:00
Michael Klishin 19ae35aa14
#3925 follow-up: don't include Erlang client headers 2021-12-28 01:24:32 +03:00
Michael Klishin 7ded41f26a
#3925 follow-up: update Bazel files to match new suite names 2021-12-28 01:19:00 +03:00
Michael Klishin b569ab5d74
Rename two newly introduced test modules 2021-12-28 00:35:55 +03:00
Michael Klishin 202f881601
Make xref happy 2021-12-26 04:32:00 +03:00
dcorbacho c88605aab4
Import definitions: support user limits 2021-12-26 04:32:00 +03:00
Michael Klishin 5d2a735ae7
Cosmetics 2021-12-26 04:32:00 +03:00
Lajos Gerecs c972f07816
wrap authentication calls in try catch to avoid leaking error 2021-12-26 04:32:00 +03:00
Luke Bakken d1496a2c7c
Fix tests 2021-12-26 04:32:00 +03:00
Luke Bakken 043641c99f
Use protected ets so that data can be read quickly 2021-12-26 04:31:59 +03:00
Luke Bakken eecfa0a1e9
Clarify warning message 2021-12-26 04:31:59 +03:00
Luke Bakken fe8ae3c713
Restore old win32 free disk query using `dir` as a last resort 2021-12-26 04:31:59 +03:00
Luke Bakken c6271a90a0
Be smarter about extracting the drive letter from a directory on win32 2021-12-26 04:31:59 +03:00
Luke Bakken 6200887f84
Disk monitor improvements
Related to VESC-1015

* Remove `infinity` timeouts
* Improve free disk space retrieval on win32

Run commands with a timeout

This PR fixes an issue I observed while reproducing VESC-1015 on Windows
10. Within an hour or so of running a 3-node cluster that has health
checks being run against it, one or more nodes' memory use would spike.
I would see that the rabbit_disk_monitor process is stuck executing
os:cmd to retrieve free disk space information. Thus, all
gen_server:call calls to the process would never return, especially
since they used an infinity timeout.

Do something with timeout

Fix unit_disk_monitor_mocks_SUITE
2021-12-26 04:31:59 +03:00
Thuan Duong Ba 542d2cf7a5 add bazel rule definition for rabbit_mirror_queue_misc_SUITE and rabbit_mirror_queue_sync_SUITE 2021-12-20 17:39:06 -08:00
Thuan Duong Ba fe8bd1508a fix the sync pause time calculation 2021-12-20 17:39:06 -08:00
Thuan Duong Ba 83b94ca6a9 reset counter after each sync throughput check interval 2021-12-20 17:39:06 -08:00
Thuan Duong Ba dc6fb24761 minor fix on condition to stop batching when total batch size is large 2021-12-20 17:39:06 -08:00
Thuan Duong Ba 1ab485b44c minor update for batching messages when syncthroughput is 0 2021-12-20 17:39:06 -08:00
Thuan Duong Ba 157bffa332 Support configure max sync throughput in CMQs 2021-12-20 17:39:06 -08:00
Michael Klishin cee6c25bc0
A slightly improved log message wording 2021-12-20 12:56:20 +05:00
Luke Bakken 555c7efa27
read term file using with_handle/1 2021-12-20 12:36:08 +05:00
Luke Bakken bc1a912af8
Use `raw` option for `file:read_file_info/2` to work around win32 mem leak erlang/otp#5527 2021-12-20 12:36:08 +05:00
Luke Bakken 6a01d80ebe
Fix win32 memory leak by using `raw` file operations. Not ideal but this
skips all file servers.
2021-12-20 12:36:08 +05:00
Loïc Hoguin c3d3f89ca8
Fix log level when halting the VM
When switching from lager to logger a mistake was made and
the list of applications remaining when the node stops was
made to log at error level instead of info.

https://github.com/rabbitmq/rabbitmq-server/commit/cdcf602749a#diff-5d9f448b943b2c77af7822440fc8df8af02bf2001d1c51a484413f69cad37a62L473-R486
2021-12-20 12:36:08 +05:00
Luke Bakken 8260f4cc04
Make health checks less resource intensive
Alternative to #3893

Also see VESC-1015
2021-12-20 12:36:06 +05:00
polaris-alioth fa2c5ab0e9
Code style rectification 2021-12-20 14:49:00 +08:00
polaris-alioth 6431584a10 Prevent creating unnamed policy when loading definition 2021-12-19 12:52:26 +08:00
Karl Nilsson d72719a1ce Stream coordinator: avoid mnesia update process crashing after delete
If a delete happens shortly after a declare or other stream change
there is a chance the mnesia update process that is spawned will crash
when the amqqueue record cannot be recovered from durable storage.

This isn't harmful but does pollute the logs.
2021-12-17 14:41:24 +00:00
Philip Kuryloski 249e8c853c Adjust the way rabbit_fifo.hrl is referenced in rabbit_fifo_SUITE
For erlang_ls convenience
2021-12-16 16:41:15 +01:00
dcorbacho 08b78aa2d7 Optimise user declare/update/import by reducing the number of operations
User, tags and hashed password can be inserted/updated at once,
increasing the performance of the storage backend - mnesia.
2021-12-14 15:35:47 +01:00
Michael Klishin ebd79836c1 Revisit operator policy merging rules for boolean fields
For booleans, we can prefer the operator policy value
unconditionally, without any safety implications.

Per discussion with @binarin @pjk25

(cherry picked from commit 6edb7396fd)
2021-12-10 19:48:16 +00:00
Loïc Hoguin 1b0eb9a4a3
Fix case where confirms may not be sent
A channel that first sends a mandatory publish before enabling
confirms mode may not receive confirms for messages published
after that. This is because the publish_seqno was increased
also for mandatory publishes even if confirms were disabled.
But the mandatory feature has nothing to do with publish_seqno.

The issue exists since at least
38e5b687de

The test case introduced focuses for multiple=false. The issue
also exists for multiple=true but it has a different impact:
sending multiple=true,delivery_tag=2 results in both messages
1 and 2 being acked, even if message 2 doesn't exist as far
as the client is concerned. If the message does exist
it might get confirmed earlier than it should have been. The
issue is a bigger problem the more mandatory messages were
sent before enabling confirms mode.
2021-12-08 15:53:47 +01:00
Michael Klishin 29763a9318
Merge pull request #3838 from rabbitmq/lukebakken/allow-compress-on-rotate
Add compress option for file rotation
2021-12-03 13:44:02 +03:00
Michael Klishin 65de676a01
Log rotation date spec: parse $H the same way Lager does 2021-12-03 13:22:37 +03:00
Luke Bakken bf70e41c09
Add compress option for file rotation
Related to #3835

https://www.erlang.org/doc/man/logger_std_h.html
2021-12-02 15:10:06 -08:00
Michael Klishin 4e9bee8bab
rabbit_logger_std_h: make Dialyzer run 2021-12-02 19:30:05 +03:00