Commit Graph

49842 Commits

Author SHA1 Message Date
tomyouyou 2d58ce7c5f
Optimisation for 'delegate'
This is copied from https://github.com/rabbitmq/rabbitmq-common/pull/349


If a message is sent to only one queue(in most application scenarios), passing through the 'delegate' is meaningless. Otherwise, it increases the delay of the message and the possibility of 'delegate' congestion.

Here are some test data:
node1: Pentium(R) Dual-Core CPU E5300 @ 2.60GHz
node2: Pentium(R) Dual-Core CPU E5300 @ 2.60GHz

Join node1 and node2 to a cluster. Create 100 queues on node2, and start 100 consumers to receive messages from these queues.
Start 100 publishers on node1 to send messages to the queues of node2. Each publisher will send 10k messages at the rate of 100/s(10k/s theoretically in total), and all the messages for all publishers is 1 million.

Before optimisation:
{1,[{msg_time,812312(=<1ms),177922(=<5ms),9507(=<50ms),221(=<500ms),38(=<1000ms),0,0,0,0,1061,1069,0,0}]}

After optimisation:
{1,[{msg_time,902854(=< 1ms),93993(=<5ms),3038(=<50ms),96(=<500ms),19(=<1000ms),0,0,0,0,1049,1060,0,0}]}

Additional information:

Time counted here is the stay time of a message in the cluster, that is, Time(leaving from node2 at) - Time(reaching node1 at).
"812312(=<1ms)" is the number of messages with time consumption less than or equal to 1ms.
Overall, the optimisation is effective.
2021-12-26 04:31:58 +03:00
Michael Klishin 49809d1516
Merge branch 'delegate_opt' 2021-12-26 03:48:08 +03:00
Michael Klishin 394ee4880a
delegate: documentation edits 2021-12-26 03:19:33 +03:00
Michael Klishin 8a0ad56182
Merge branch 'master' into delegate_opt 2021-12-26 01:46:29 +03:00
Michael Klishin 6835a5b9a7
Merge pull request #3930 from rabbitmq/import-user-limits
Import definitions: support user limits
2021-12-23 18:06:14 +03:00
Michael Klishin fac7199023
Make xref happy 2021-12-23 17:28:01 +03:00
dcorbacho a22e9abb89 Import definitions: support user limits 2021-12-23 11:33:44 +01:00
Michael Klishin ba238053c2
Merge branch 'luos-wrap_auth_calls_in_try_catch' 2021-12-22 18:42:46 +03:00
Michael Klishin 91aab6533d
Cosmetics 2021-12-22 18:42:33 +03:00
Lajos Gerecs 3fe9112533
wrap authentication calls in try catch to avoid leaking error 2021-12-21 06:43:02 -08:00
Luke Bakken dcb8e0f877
Merge pull request #3895 from rabbitmq/lukebakken/rabbit_disk_monitor-improvements
Disk monitor improvements
2021-12-21 06:28:37 -08:00
Thuan Duong Ba 542d2cf7a5 add bazel rule definition for rabbit_mirror_queue_misc_SUITE and rabbit_mirror_queue_sync_SUITE 2021-12-20 17:39:06 -08:00
Thuan Duong Ba fe8bd1508a fix the sync pause time calculation 2021-12-20 17:39:06 -08:00
Thuan Duong Ba 83b94ca6a9 reset counter after each sync throughput check interval 2021-12-20 17:39:06 -08:00
Thuan Duong Ba dc6fb24761 minor fix on condition to stop batching when total batch size is large 2021-12-20 17:39:06 -08:00
Thuan Duong Ba 1ab485b44c minor update for batching messages when syncthroughput is 0 2021-12-20 17:39:06 -08:00
Thuan Duong Ba 157bffa332 Support configure max sync throughput in CMQs 2021-12-20 17:39:06 -08:00
Luke Bakken 7a3ab5e737
Fix tests 2021-12-20 09:10:52 -08:00
Luke Bakken 2b63bae50d
Use protected ets so that data can be read quickly 2021-12-20 08:42:51 -08:00
Luke Bakken 6ab3304c6c
Clarify warning message 2021-12-20 08:41:03 -08:00
Luke Bakken a023908642
Restore old win32 free disk query using `dir` as a last resort 2021-12-20 08:41:03 -08:00
Luke Bakken fad573cd93
Be smarter about extracting the drive letter from a directory on win32 2021-12-20 08:41:03 -08:00
Luke Bakken e0c4681eba
Disk monitor improvements
Related to VESC-1015

* Remove `infinity` timeouts
* Improve free disk space retrieval on win32

Run commands with a timeout

This PR fixes an issue I observed while reproducing VESC-1015 on Windows
10. Within an hour or so of running a 3-node cluster that has health
checks being run against it, one or more nodes' memory use would spike.
I would see that the rabbit_disk_monitor process is stuck executing
os:cmd to retrieve free disk space information. Thus, all
gen_server:call calls to the process would never return, especially
since they used an infinity timeout.

Do something with timeout

Fix unit_disk_monitor_mocks_SUITE
2021-12-20 08:41:02 -08:00
Michael Klishin 11c892dd94
Merge branch 'polaris-alioth-prevent_unnamed_policy_when_loading_definition' 2021-12-20 12:56:38 +05:00
Michael Klishin cee6c25bc0
A slightly improved log message wording 2021-12-20 12:56:20 +05:00
Anh Thi Lan Nguyen 89fd4aba46
Increase token expiration time 2021-12-20 12:36:10 +05:00
Anh Thi Lan Nguyen 8bcfbd594f
Start SSL app for testing server 2021-12-20 12:36:10 +05:00
Anh Thi Lan Nguyen 77608eb624
Standardise README.md 2021-12-20 12:36:10 +05:00
Anh Thi Lan Nguyen 20af75bcdd
Correct configuration example in README.md 2021-12-20 12:36:09 +05:00
Anh Thi Lan Nguyen 19ea17c652
Add timeout for httpc request 2021-12-20 12:36:09 +05:00
Anh Thi Lan Nguyen 0e46c873de
Add configurable crl_check and fail_if_no_peer_cert
- Add configuration: crl_check, fail_if_no_peer_cert
- Correct configuration: hostname_verification
2021-12-20 12:36:09 +05:00
Anh Thi Lan Nguyen b803f9ea75
Add wildcard configuration
A "wildcard" configuration is added to enable key server verification with wildcard certificate
2021-12-20 12:36:09 +05:00
Anh Thi Lan Nguyen 9565e7d975
Update README.md
- Update new configuration document
- Add configurable "depth" for key server verification
2021-12-20 12:36:09 +05:00
Anh Thi Lan Nguyen 0ff2e0c4e4
Set peer_verification default as verify_none 2021-12-20 12:36:09 +05:00
Anh Thi Lan Nguyen f658a51cbc
Update better configuration names
- "strict" changes to "https.peer_verification"
- "cacertfile" changes to "https.cacertfile"
2021-12-20 12:36:09 +05:00
Anh Thi Lan Nguyen 5abfc2b547
Oauth2 plugin improvements
- Validate JWKS server when getting keys
- Restrict usable algorithms
2021-12-20 12:36:08 +05:00
Luke Bakken 555c7efa27
read term file using with_handle/1 2021-12-20 12:36:08 +05:00
Luke Bakken bc1a912af8
Use `raw` option for `file:read_file_info/2` to work around win32 mem leak erlang/otp#5527 2021-12-20 12:36:08 +05:00
Luke Bakken 6a01d80ebe
Fix win32 memory leak by using `raw` file operations. Not ideal but this
skips all file servers.
2021-12-20 12:36:08 +05:00
Loïc Hoguin c3d3f89ca8
Fix log level when halting the VM
When switching from lager to logger a mistake was made and
the list of applications remaining when the node stops was
made to log at error level instead of info.

https://github.com/rabbitmq/rabbitmq-server/commit/cdcf602749a#diff-5d9f448b943b2c77af7822440fc8df8af02bf2001d1c51a484413f69cad37a62L473-R486
2021-12-20 12:36:08 +05:00
Luke Bakken 8260f4cc04
Make health checks less resource intensive
Alternative to #3893

Also see VESC-1015
2021-12-20 12:36:06 +05:00
polaris-alioth fa2c5ab0e9
Code style rectification 2021-12-20 14:49:00 +08:00
Michael Klishin a9c123812e
Merge pull request #3887 from anhanhnguyen/master
OAuth 2 plugin improvements
2021-12-19 16:50:24 +05:00
polaris-alioth 6431584a10 Prevent creating unnamed policy when loading definition 2021-12-19 12:52:26 +08:00
Michael Klishin 2ec8596597
Merge pull request #3906 from rabbitmq/lukebakken/fix-mem-leak
Fix win32 memory leak by using `raw` file operations
2021-12-19 09:15:34 +05:00
Luke Bakken 365de60c24
read term file using with_handle/1 2021-12-18 08:32:22 -08:00
Luke Bakken 20977f68fd
Use `raw` option for `file:read_file_info/2` to work around win32 mem leak erlang/otp#5527 2021-12-18 07:39:26 -08:00
Luke Bakken 458306dc8b
Fix win32 memory leak by using `raw` file operations. Not ideal but this
skips all file servers.
2021-12-18 07:33:37 -08:00
Michael Klishin e316d989d6
Merge pull request #3907 from rabbitmq/lh-fix-log-level-on-stop
Fix log level when halting the VM
2021-12-18 16:38:22 +05:00
Michael Klishin 40491b6ac9
Merge pull request #3905 from rabbitmq/lukebakken/node-health-check-fix
Make health checks less resource intensive
2021-12-18 16:37:47 +05:00