Commit Graph

366 Commits

Author SHA1 Message Date
Michael Klishin 4cb9b001d7 Manually recover ring state from durable exchanges and bindings
Only relevant for some use cases post-3.8.4 which has moved
plugin activation to a later point in the boot cycle, after
topology recovery.

Previously this state was naturally recovered during the topology
recovery boot step.

Closes #45.
2020-06-12 10:39:53 +03:00
Michael Klishin 6df9370c6c Mention routing locality in the README
References #32 and specifically [1].

1. https://github.com/rabbitmq/rabbitmq-consistent-hash-exchange/issues/32#issuecomment-643074328
2020-06-12 08:39:12 +03:00
Michael Klishin 1b54c9a2f5 Ditto for the Python example 2020-06-11 19:38:31 +03:00
Michael Klishin 9eb9a4660c Explain what the [Ruby] example does 2020-06-11 19:36:55 +03:00
Michael Klishin 98be220b50 Update Bunny dependency 2020-06-11 19:36:42 +03:00
Michael Klishin 2d5e015791 Update rabbitmq-components.mk 2020-06-10 19:51:15 +03:00
Jean-Sébastien Pédron f73587775a Update copyright (year 2020) 2020-03-10 16:08:09 +01:00
Gerhard Lazu d9e06337e0 Update rabbitmq-components.mk 2020-03-06 09:19:14 +00:00
Gerhard Lazu 00b26fc9bf Update erlang.mk 2020-03-06 09:17:29 +00:00
Jean-Sébastien Pédron 1f711c8fe0 Travis CI: Update config from rabbitmq-common 2020-03-04 14:24:27 +01:00
Jean-Sébastien Pédron f1df2f9e88 Travis CI: Update config from rabbitmq-common 2020-03-04 11:17:14 +01:00
Jean-Sébastien Pédron 6331101208 Travis CI: Update config from rabbitmq-common 2020-03-03 14:53:38 +01:00
Gerhard Lazu 518e124810 Update rabbitmq-components.mk 2020-02-11 15:31:43 +00:00
Michael Klishin 8c28dff573 (c) bump 2019-12-29 05:50:26 +03:00
Jean-Sébastien Pédron b0df8b3893 Git: Ignore copied CLI 2019-12-12 13:27:39 +01:00
Jean-Sébastien Pédron 6a55ef18e6 Update rabbitmq-components.mk 2019-12-12 13:14:53 +01:00
kjnilsson d84a531d1d Update rabbitmq-components.mk 2019-09-13 10:24:17 +01:00
kjnilsson e80156dbfd Update rabbitmq-components.mk 2019-09-04 10:30:54 +01:00
Jean-Sébastien Pédron fa5db839a0 Update erlang.mk 2019-08-29 20:44:58 +02:00
Michael Klishin a180dc5ab5 Update rabbitmq-components.mk 2019-08-11 01:47:51 +10:00
Jean-Sébastien Pédron 27533feef3 Update erlang.mk 2019-08-02 09:53:41 +02:00
Michael Klishin b45765ead7 Update rabbitmq-components.mk 2019-08-01 17:13:12 +03:00
Arnaud Cogoluègnes bea4abae24 Update rabbitmq-components.mk 2019-07-09 16:05:53 +02:00
Jean-Sébastien Pédron d6080a0af0 Update rabbitmq-components.mk 2019-06-28 16:05:27 +02:00
Jean-Sébastien Pédron 7e74ba9785 Update erlang.mk 2019-06-28 16:01:52 +02:00
Gerhard Lazu 468358361d Update rabbitmq-components.mk 2019-06-03 02:23:34 +01:00
Jean-Sébastien Pédron f90ce74486 Update rabbitmq-components.mk 2019-05-17 15:17:15 +02:00
Michael Klishin c91c3a6488 Update rabbitmq-components.mk 2019-05-17 08:36:30 +03:00
Michael Klishin 1797522427 Update rabbitmq-components.mk 2019-05-16 23:22:46 +03:00
Michael Klishin 0797fa32c5 Update rabbitmq-components.mk 2019-05-14 11:50:43 +03:00
Michael Klishin 8bd9ef97a6 Update rabbitmq-components.mk 2019-05-13 16:40:28 +03:00
Gerhard Lazu 3f3fe2c2c9 Update rabbitmq-components.mk 2019-05-01 13:18:10 +01:00
Luke Bakken 252bd79b60 Update rabbitmq-components.mk 2019-04-30 16:20:54 -07:00
Jean-Sébastien Pédron 2d3c774f97 Update rabbitmq-components.mk 2019-04-30 14:47:27 +02:00
Gerhard Lazu 6887674c53 Update rabbitmq-components.mk 2019-04-22 21:55:01 +01:00
Spring Operator f1ac305a24 URL Cleanup
This commit updates URLs to prefer the https protocol. Redirects are not followed to avoid accidentally expanding intentionally shortened URLs (i.e. if using a URL shortener).

# HTTP URLs that Could Not Be Fixed
These URLs were unable to be fixed. Please review them to see if they can be manually resolved.

* http://blog.listincomprehension.com/search/label/procket (200) with 1 occurrences could not be migrated:
   ([https](https://blog.listincomprehension.com/search/label/procket) result ClosedChannelException).
* http://dozzie.jarowit.net/trac/wiki/TOML (200) with 1 occurrences could not be migrated:
   ([https](https://dozzie.jarowit.net/trac/wiki/TOML) result SSLHandshakeException).
* http://dozzie.jarowit.net/trac/wiki/subproc (200) with 1 occurrences could not be migrated:
   ([https](https://dozzie.jarowit.net/trac/wiki/subproc) result SSLHandshakeException).
* http://e2project.org (200) with 1 occurrences could not be migrated:
   ([https](https://e2project.org) result AnnotatedConnectException).
* http://michaelnielsen.org/blog/consistent-hashing/ (200) with 1 occurrences could not be migrated:
   ([https](https://michaelnielsen.org/blog/consistent-hashing/) result SSLHandshakeException).
* http://nitrogenproject.com/ (200) with 2 occurrences could not be migrated:
   ([https](https://nitrogenproject.com/) result ConnectTimeoutException).
* http://proper.softlab.ntua.gr (200) with 1 occurrences could not be migrated:
   ([https](https://proper.softlab.ntua.gr) result SSLHandshakeException).
* http://rubybunny.info (200) with 1 occurrences could not be migrated:
   ([https](https://rubybunny.info) result AnnotatedConnectException).
* http://www.martinbroadhurst.com/Consistent-Hash-Ring.html (200) with 1 occurrences could not be migrated:
   ([https](https://www.martinbroadhurst.com/Consistent-Hash-Ring.html) result SSLHandshakeException).
* http://yaws.hyber.org (200) with 1 occurrences could not be migrated:
   ([https](https://yaws.hyber.org) result AnnotatedConnectException).
* http://choven.ca (503) with 1 occurrences could not be migrated:
   ([https](https://choven.ca) result ConnectTimeoutException).

# Fixed URLs

## Fixed But Review Recommended
These URLs were fixed, but the https status was not OK. However, the https status was the same as the http request or http redirected to an https URL, so they were migrated. Your review is recommended.

* http://fixprotocol.org/ (301) with 1 occurrences migrated to:
  https://fixtrading.org ([https](https://fixprotocol.org/) result SSLHandshakeException).
* http://erldb.org (UnknownHostException) with 1 occurrences migrated to:
  https://erldb.org ([https](https://erldb.org) result UnknownHostException).

## Fixed Success
These URLs were switched to an https URL with a 2xx status. While the status was successful, your review is still recommended.

* http://cloudi.org/ with 27 occurrences migrated to:
  https://cloudi.org/ ([https](https://cloudi.org/) result 200).
* http://en.wikipedia.org/wiki/Consistent_hashing with 1 occurrences migrated to:
  https://en.wikipedia.org/wiki/Consistent_hashing ([https](https://en.wikipedia.org/wiki/Consistent_hashing) result 200).
* http://erlware.org/ with 1 occurrences migrated to:
  https://erlware.org/ ([https](https://erlware.org/) result 200).
* http://inaka.github.io/cowboy-trails/ with 1 occurrences migrated to:
  https://inaka.github.io/cowboy-trails/ ([https](https://inaka.github.io/cowboy-trails/) result 200).
* http://ninenines.eu with 6 occurrences migrated to:
  https://ninenines.eu ([https](https://ninenines.eu) result 200).
* http://www.actordb.com/ with 2 occurrences migrated to:
  https://www.actordb.com/ ([https](https://www.actordb.com/) result 200).
* http://www.cs.kent.ac.uk/projects/wrangler/Home.html with 1 occurrences migrated to:
  https://www.cs.kent.ac.uk/projects/wrangler/Home.html ([https](https://www.cs.kent.ac.uk/projects/wrangler/Home.html) result 200).
* http://www.rabbitmq.com/plugins.html with 1 occurrences migrated to:
  https://www.rabbitmq.com/plugins.html ([https](https://www.rabbitmq.com/plugins.html) result 200).
* http://www.rebar3.org with 1 occurrences migrated to:
  https://www.rebar3.org ([https](https://www.rebar3.org) result 200).
* http://contributor-covenant.org with 1 occurrences migrated to:
  https://contributor-covenant.org ([https](https://contributor-covenant.org) result 301).
* http://contributor-covenant.org/version/1/3/0/ with 1 occurrences migrated to:
  https://contributor-covenant.org/version/1/3/0/ ([https](https://contributor-covenant.org/version/1/3/0/) result 301).
* http://inaka.github.com/apns4erl with 1 occurrences migrated to:
  https://inaka.github.com/apns4erl ([https](https://inaka.github.com/apns4erl) result 301).
* http://inaka.github.com/edis/ with 1 occurrences migrated to:
  https://inaka.github.com/edis/ ([https](https://inaka.github.com/edis/) result 301).
* http://lasp-lang.org/ with 1 occurrences migrated to:
  https://lasp-lang.org/ ([https](https://lasp-lang.org/) result 301).
* http://saleyn.github.com/erlexec with 1 occurrences migrated to:
  https://saleyn.github.com/erlexec ([https](https://saleyn.github.com/erlexec) result 301).
* http://www.mozilla.org/MPL/ with 3 occurrences migrated to:
  https://www.mozilla.org/MPL/ ([https](https://www.mozilla.org/MPL/) result 301).
* http://zhongwencool.github.io/observer_cli with 1 occurrences migrated to:
  https://zhongwencool.github.io/observer_cli ([https](https://zhongwencool.github.io/observer_cli) result 301).
2019-03-20 03:13:58 -05:00
Michael Klishin 3051e234ad Travis: use Erlang 21.3, Elixir 1.8.1 2019-03-20 10:33:13 +03:00
Michael Klishin 7b395083de Update rabbitmq-components.mk 2019-03-15 05:32:50 +03:00
Michael Klishin f5d3cd0c40 Update rabbitmq-components.mk 2019-03-15 05:05:40 +03:00
Michael Klishin 3bda05f884 Update rabbitmq-components.mk 2019-02-12 23:23:50 +03:00
Michael Klishin 839ca669ff .travis.yml: use 21.2 as 21.2.2 builds are no available 2019-01-14 01:20:19 +03:00
Michael Klishin 9be0c69cd2 .travis.yml: sync Erlang and Elixir versions, require Erlang/OTP 21.2 2019-01-13 05:49:58 +03:00
Luke Bakken 822ade6127 Update rabbitmq-components.mk 2019-01-09 12:54:59 -08:00
Luke Bakken d557a222b7 Update rabbitmq-components.mk 2019-01-08 11:58:25 -08:00
Diana Corbacho 1b75f7beec More tests for #40
Use served-named queues
2019-01-07 14:03:19 +00:00
Diana Corbacho 81836b3531 Refactor test code
References #40.
2019-01-07 13:20:23 +00:00
Michael Klishin 17451f05de Make sure binding's bucket keys are sorted as they form a range
maps:keys/1 does not offer any ordering guarantees as don't Erlang
maps themselves.

Pair: @dcorbacho.

References #40.
2019-01-07 15:56:23 +03:00
Michael Klishin 235bacb3b4 More tests for #40 2019-01-07 15:56:17 +03:00
Michael Klishin 93626ee0c8 Add a failing test for the scenario outlined in #40 2019-01-07 15:24:12 +03:00
Michael Klishin ce9524fba6 Update rabbitmq-components.mk 2018-12-06 18:28:43 +03:00
Michael Klishin c0d9473823 Update rabbitmq-components.mk 2018-12-06 03:15:09 +03:00
Jean-Sébastien Pédron 7f09033b8c Update rabbitmq-components.mk 2018-11-30 14:56:47 +01:00
Jean-Sébastien Pédron 4a79cab6ce Update rabbitmq-components.mk 2018-11-22 09:19:32 +01:00
kjnilsson ce0c8e2f1c Update rabbitmq-components.mk 2018-10-29 12:43:17 +00:00
Jean-Sébastien Pédron 46ab013619 Update rabbitmq-components.mk 2018-09-19 10:38:13 +02:00
Jean-Sébastien Pédron c27d57bdba Update rabbitmq-components.mk 2018-09-11 14:22:26 +02:00
Michael Klishin 0638c70552 Make chi squared test an observation we log, not an assertion
Due to randomness of the inputs and other characteristics that vary
beetween environments it doesn't always end up being < the expected
value but there's plenty of evidence that in most environments
the resulting distribution is very uniform (for all intents and
purposes of this plugin anyway).

References #37, #39.
2018-08-31 23:51:36 +02:00
Michael Klishin fde2479d00 Merge pull request #39 from rabbitmq/hash-ring-management-fixes
Store hash ring state in a single table
2018-08-30 10:37:56 +03:00
Diana Corbacho 075ea3817d Remove table locking 2018-08-29 15:31:12 +01:00
Diana Corbacho 7143b03c46 Create a new map with the updated buckets
Previous implementation would update/delete more than once some keys
delivering an inconsistent ring
2018-08-29 12:08:52 +01:00
Luke Bakken a6dc0f337c Update git-commit-msgs link 2018-08-28 16:18:10 -07:00
Michael Klishin 6ace19d972 Use only a subset of queues in routing tests 2018-08-28 20:01:25 +03:00
Michael Klishin 9883618853 Commit a new header file 2018-08-28 20:00:52 +03:00
Michael Klishin 0b1776d59d More tests, more idempotent binding management operations
[#159822323]
2018-08-28 19:53:52 +03:00
Michael Klishin a2d4f0bd87 Store hash ring state in a single table
This implementation is significantly simpler and doesn't
perform nearly as many Mnesia operations.

Pair: @dcorbacho.

References #37, #38.

[#159822323]
2018-08-28 16:40:38 +03:00
Michael Klishin e5e9316b9e Ignore debug/* 2018-08-28 16:40:26 +03:00
Michael Klishin 1788aafe2e Explain 2018-08-24 12:20:28 +03:00
Michael Klishin 02ce15af54 Update bucket numbers for each queue just once
There can be more than one bucket per queue, so we ended up with
potential extra updates that resulted in incorrect (negative) bucket numbers.

As part of this we considered using an alternative data model:
folding two tables into one that looks like this:

exchange => map(bucket => queue)

This would greatly simplify binding management and be roughly as
efficient for routing except for one thing: updating the rest of the ring
will be a linear operation over all buckets. So that change alone
would be insufficient but remains to be an idea to improve on
in the future (e.g. by using a tree).

Pair: @dcorbacho.

References #37, #38.

[#159822323]
2018-08-24 12:12:10 +03:00
Michael Klishin 0585c01789 WIP More hash ring management fixes
* Update bucket counts for queues in the updated "downstream" segment
   of the ring.

 * Delete binding buckets for the given {exchange, queue} when removing
   a binding
2018-08-24 11:00:11 +03:00
Michael Klishin b3b05e7e70 Comments 2018-08-24 03:04:14 +03:00
Michael Klishin 7efdf48dd6 WIP: Hash ring management fixes
remove_binding/1 was iterating over a table while also modifying it
and was running into "duplicate" keys (from buckets with lower numbers
that are being deleted).

There are two changes that move us in the right direction:

 * Delete buckets of the binding being deleted first, then update those
   with higher ranges on the right (downstream buckets?)
 * Avoid concurrent iteration and modification

One remaining item is: updating bucket_numbers for all the affected
bindings/pairs.

Pair: @lukebakken.
2018-08-24 02:47:23 +03:00
Michael Klishin a4d575ba6d Correct a PR template typo 2018-08-21 18:54:19 +03:00
Michael Klishin 7d4fa1c778 Link to the examples directory 2018-08-21 18:49:18 +03:00
Michael Klishin 393956d71d Examples: correct plugin repo links 2018-08-21 18:48:07 +03:00
Michael Klishin f281cb473f Minor README improvements 2018-08-21 18:41:57 +03:00
Michael Klishin 491e71d0b1 A typo 2018-08-21 18:37:13 +03:00
Michael Klishin c4f112e393 Executable examples for Ruby 2018-08-21 18:36:15 +03:00
Michael Klishin 5babddcd0a README: Unify Ruby examples 2018-08-21 18:31:56 +03:00
Michael Klishin 6ab23f79d3 Executable examples for Python 2018-08-21 18:05:13 +03:00
Michael Klishin b368ee922e Increase sample count to pass Chi squared test in more environments, reorganise tests
We still depend on the PRNG to provide a reasonably uniform distribution
of inputs (e.g. routing keys) but things pass in at least 3 different environments
reliably with 150K iterations.

Pair: @dcorbacho.

References #37, #38.
2018-08-21 16:40:21 +03:00
Michael Klishin ab5f54ee8f Bring back the Chi squared test assertion, bump the number of samples 2018-08-21 16:23:10 +03:00
Michael Klishin 67fe821b79 Fix a warning 2018-08-21 16:03:18 +03:00
Michael Klishin eeba8685f5 Use an OTP 19.3 compatible seed generation algorithm
References #37, #38.
2018-08-21 15:56:36 +03:00
Michael Klishin 05e7cc756f Don't assert on Chi squared test value
In some environments, namely our Concourse containers, with *some* iterations
of the test the value exceeds the reference value of p-value = 0.01.

This may be specific to OTP 19.3 or certain platforms. This is not
something that I can reproduce in a number of OTP 21 environments.

References #37, #38.
2018-08-21 06:50:04 +03:00
Michael Klishin 219fc225c2 Erlang/OTP 19.3 compatibility for #37
19.3 does not provide math:floor/1 or erlang:floor/1.
2018-08-20 20:01:16 +03:00
Michael Klishin d6e9fd9b9e Test suite improvements
* Use publisher confirms, that's what the test really needs
 * Clean up exchanges before setting up topology to make sure failing tests
   do not leave anything behind
2018-08-20 19:47:43 +03:00
Michael Klishin 0faf2ba272 Don't try to parse every routing key as an integer
References #37.
2018-08-20 19:29:42 +03:00
Michael Klishin e132a0a865 A typo 2018-08-20 18:31:56 +03:00
Michael Klishin 25ae2b3fe9 A typo 2018-08-20 15:13:28 +03:00
Michael Klishin 4adfec92e8 Add Ch-squared test results to implementation details 2018-08-20 15:12:27 +03:00
Michael Klishin b887efdcf3 Extract a few test helpers 2018-08-20 15:05:01 +03:00
Michael Klishin 7cc7b376ad More code examples in Java, minor doc edits 2018-08-20 14:10:23 +03:00
Michael Klishin 989efa604a Add a Java code example 2018-08-20 13:39:35 +03:00
Diana Corbacho f319c84343 Test different bucket sizes 2018-08-20 11:07:12 +01:00
Diana Corbacho 02c5be2d54 Test - and fix - binding cleanup 2018-08-20 08:57:35 +01:00
Michael Klishin 9bde2bc88a More code examples, minor edits 2018-08-19 16:59:30 +03:00
Michael Klishin cb4610d29d Code examples in more languages for the header routing example 2018-08-19 16:35:53 +03:00
Michael Klishin fc1ed981f5 Provide examples in Python and Ruby
with more examples to come as time permits
2018-08-19 15:59:31 +03:00
Michael Klishin e76e1c7667 More documentation updates 2018-08-19 14:20:39 +03:00
Michael Klishin 9b5fe90736 Documentation updates 2018-08-19 14:09:00 +03:00