Commit Graph

338 Commits

Author SHA1 Message Date
Michael Klishin 40db671e0b
Merge pull request #3633 from wrobell/rabbitmq-streams-doc-protocol-hex
Use hex values in RabbitMQ Streams protocol description
2021-11-02 21:58:42 +03:00
wrobell 08c417fdb7 Use hex values in RabbitMQ Streams protocol description
The protocol documentation uses decimal values for error and request key
codes.

Let's use hex values instead. This helps when looking at a request and
its response - 0x0006 and 0x8006 vs. 6 and 32774.

Also, when looking at output of protocol analysis tools like Wireshark,
a hexadecimal value will be printed, for example:

    "Nov  1, 2021 23:05:19.395825508 GMT","60216,5552","00000009000600010000000701"
    "Nov  1, 2021 23:05:19.396069528 GMT","5552,60216","0000000a80060001000000070001"

Above, we can visually identify delete publisher request and response
(0x0006 and 0x8006) and easily match them in the documentation of the
protocol.

Finally, above argument applies to logging as it is common to log
hex values, not decimal.
2021-11-02 18:31:30 +00:00
Philip Kuryloski 76287fb30c format with bazel files with buildifier 2021-11-02 18:24:26 +01:00
wrobell e74823d68c Fix use of "RabbitMQ Streams" in protocol documentation
The following RabbitMQ page uses "RabbitMQ Streams" or "streams",
therefore use it in the protocol documentation as well

    https://blog.rabbitmq.com/posts/2021/07/rabbitmq-streams-overview
2021-10-29 19:06:06 +01:00
wrobell 57ff5dbf4a Fix RabbitMQ Streams protocol documentation
1. Response for publisher declaration request does not contain
   publisher id.

2. Add mechanism entry to the details of SASL handshake request.

3. SASL handshake response contains list of mechanisms, not just single
   mechanism.
2021-10-28 17:53:22 +01:00
Arnaud Cogoluègnes c0f1e23b17
Disable stream test in mixed versions mode
Not worth testing in this mode, as the stream plugin
does not start without the stream_queue feature flag.
2021-10-20 15:16:45 +02:00
Arnaud Cogoluègnes 030a08fa54
Enable stream_queue flag with env variable in test suite
For mixed version mode, where feature flags are
not explictely enabled.
2021-10-20 15:08:05 +02:00
Arnaud Cogoluègnes d87fa91a52
Enable stream_queue flag in test suite
For mixed version mode, where feature flags are
not explictely enabled.
2021-10-20 14:55:54 +02:00
Arnaud Cogoluègnes 6aabfdbbb4
Format stream plugin 2021-10-20 14:53:08 +02:00
Michael Klishin 2140562209
Merge pull request #3562 from rabbitmq/disable_stream_plugin_3557
Disable the stream_plugin if stream feature flag is not enabled
2021-10-14 18:45:26 +03:00
Arnaud Cogoluègnes 6d0dc3f6b4
Attach add/delete super stream command to streams scope only
Not to ctl.

References #3503
2021-10-14 17:09:18 +02:00
Michael Klishin 2fbc5fab25
Formatting, wording 2021-10-14 17:20:59 +03:00
Gabriele Santomaggio f2df98aef9 change message 2021-10-13 12:01:21 +02:00
Gabriele Santomaggio 2eb625531d Disable the stream_plugin and stream management if the feature flag
is not enabled.
fixes https://github.com/rabbitmq/rabbitmq-server/issues/3557
2021-10-13 11:59:11 +02:00
Gabriele Santomaggio c270ebf037 Disable the stream_plugin if the feature flag
is not enabled.
fixes https://github.com/rabbitmq/rabbitmq-server/issues/3557
2021-10-12 19:09:34 +02:00
Arnaud Cogoluègnes 6b9589bae4
Handle stream arguments in add_super_stream command
max-age, leader-locator, etc.
2021-10-11 16:50:03 +02:00
Arnaud Cogoluègnes ecbd969834
Embed RPC call function in test 2021-10-11 16:50:02 +02:00
Arnaud Cogoluègnes 8ae60bef73
Fix group spec in test 2021-10-11 16:50:02 +02:00
Arnaud Cogoluègnes b0bd5f8a00
Add delete_super_stream CLI command 2021-10-11 16:50:02 +02:00
Arnaud Cogoluègnes a73b1a3d0d
Add add_super_stream CLI command 2021-10-11 16:50:01 +02:00
Arnaud Cogoluègnes 147659093f
Add functions to create/delete super stream in manager 2021-10-11 16:50:01 +02:00
Michael Klishin 72c2e152ef
Merge pull request #3499 from rabbitmq/cla-url
Update CLA URL
2021-10-01 00:16:03 +03:00
Arnaud Cogoluègnes e8d0d380a2
Bump dependencies in stream Java tests 2021-09-28 14:02:10 +02:00
Arnaud Cogoluègnes fc80138204
Fallback to rabbit_stream:host/0 if advertised_tls_host not set
The advertised_host must also be tried for TLS connections.

References #3514
2021-09-28 14:00:41 +02:00
Carl Hörberg 52791c677b
Support for advertising different hostname for TLS stream connections
Use case: Allow plain connections over one (internal IP), and TLS
connections over another IP (eg. internet routable IP). Without this
patch a cluster can only support access over one or the other IP, not
both.

(cherry picked from commit b9e6aad035)
2021-09-28 13:47:22 +02:00
Michal Kuratczyk 1a2126676d Update CLA URL 2021-09-23 14:29:36 +02:00
Michael Klishin 0f6a9dac27
Introduce rabbit_nodes:all/0 2021-09-20 22:24:25 +03:00
Arnaud Cogoluègnes ab2c565b49
Merge pull request #3423 from rabbitmq/streams-sort-partitions
Sort stream partitions using binding parameter
2021-09-15 14:56:17 +02:00
Arnaud Cogoluègnes faee72552a
Polish test 2021-09-15 11:56:53 +02:00
Arnaud Cogoluègnes 4bbed58352
Add Bazel rule for rabbit_stream_utils_SUITE 2021-09-15 11:38:17 +02:00
Arnaud Cogoluègnes 5b83dceb87
Return only streams for partition-related commands
The stream partition metadata is based on bindings,
so we make sure to return only streams from the binding
information.
2021-09-15 11:33:04 +02:00
Arnaud Cogoluègnes b140395864
Fall back to HOSTNAME(1) in stream Java test suite
In case the Java way fails (only on new CI).
2021-09-15 10:36:27 +02:00
Arnaud Cogoluègnes 04a0653571
Sort stream partitions using binding parameter
If present. To make the partition order stable.
2021-09-14 18:02:22 +02:00
Michael Klishin c8d483809e
Resolve a missed conflict 2021-09-13 20:33:11 +03:00
Michael Klishin f79bc1c935
Merge branch 'master' into stream-reader-close-in-terminate
Conflicts:
	deps/rabbitmq_stream/src/rabbit_stream_reader.erl
2021-09-13 20:27:58 +03:00
Arnaud Cogoluègnes 8f207e3c5f
Make stream protocol route command return several streams
We expect to have 1 stream for each routing key, but
as binding can return several queues for a given key we
let that possibility open in the stream protocol.
2021-09-13 17:53:25 +02:00
Karl Nilsson 9e4506041d fix build warnings 2021-09-13 11:38:41 +01:00
Karl Nilsson 135575b3ff Stream reader: close osiris logs and sockets in terminate
Instead of injecting it into varios places inside the code.

When the osiris log is closed it will decrement the global "readers"
counter which is why it is much safer to do this in terminate.
2021-09-13 11:23:35 +01:00
Karl Nilsson 3b1714cbe3 formatting 2021-09-10 15:26:26 +01:00
Karl Nilsson f10db03b4d Gracefully terminate stream reaader
when the client forcefully terminates TCP connection

Also improve logging.
2021-09-10 15:24:29 +01:00
Karl Nilsson d6301a3e11 Handle closed connections in stream reader
and throw and stop gracefully.
2021-09-10 10:15:59 +01:00
Karl Nilsson 3513fa0ea8 rabbitmq_stream formatting 2021-09-09 09:45:13 +01:00
Federico Caprari 2246727428
Fix store offset parameters
As you can see 860333a088/deps/rabbitmq_stream_common/src/rabbit_stream_core.erl (L239)

There is the stream name and not the subscription id in this message.
2021-09-01 22:13:07 +02:00
Arnaud Cogoluègnes 902fa429dd
Use awaitMatch to check global counters
Assertion fails on CI environment.
2021-09-01 09:51:06 +02:00
Karl Nilsson c240ec2985
Fix function_clause error in stream reader
When the server initiate connection close.
2021-08-31 15:29:16 +01:00
Gerhard Lazu 6c0ba03d61
Test that we start from 0 publishers & consumers
Pair: @kjnilsson

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2021-08-31 15:29:16 +01:00
Gerhard Lazu 0ecf3d4eeb
Test stream publisher & consumer counters
Pair: @kjnilsson

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2021-08-31 15:29:16 +01:00
Gerhard Lazu dad0025088
Perform stream reader cleanup in terminate
Otherwise metrics will not get cleaned up correctly when processes crash.

It's also tidier to do this in a single place, in terminate/3

Pair: @kjnilsson

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2021-08-31 15:29:15 +01:00
Arnaud Cogoluègnes 8dc024089c
Bump dependencies in Java stream tests 2021-08-31 08:32:53 +02:00
Michael Klishin ace3ee9cd9
Bump Java stream client used in tests 2021-08-10 11:32:57 +03:00