Commit Graph

352 Commits

Author SHA1 Message Date
Arnaud Cogoluègnes 1bcca0745e
Increase timeout in stream test
Some CI environment may need a bit more time.
2021-12-10 12:08:02 +01:00
Arnaud Cogoluègnes 4c397c54fa
Use AssertJ conditions in Java stream test suite 2021-12-10 11:52:54 +01:00
Arnaud Cogoluègnes bd4771addb
Return appropriate response when stream leader not available
Fixes #3874
2021-12-10 10:58:48 +01:00
Arnaud Cogoluègnes 8f40e534ed
Bump Java test dependencies in stream test suites 2021-12-10 09:18:22 +01:00
Arnaud Cogoluègnes b6f5b1c21a
Merge pull request #3842 from rabbitmq/rabbitmq-server-3841-stream-consumer-nested-record
Create configuration nested record in stream consumer record
2021-12-03 11:11:40 +01:00
Arnaud Cogoluègnes 854efb3e85
Declare variable for better formatting
References #3841
2021-12-03 10:59:45 +01:00
Arnaud Cogoluègnes 81b9632535
Create configuration nested record in stream consumer record
Only a couple of fields of the stream consumer record change
very frequently (credits and Osiris log reference), so this commit
introduces a nested record in the main consumer record that
contains the immutable fields. This potentially avoids producing
a lot of garbage, especially when the consumer state contains
several properties (consumer name, or single active consumer information
in the future).

Fixes #3841
2021-12-03 10:34:51 +01:00
Arnaud Cogoluègnes 17d0ba9317
Return "no offset" (19) code when stored offset is undefined
Fixes #3783
2021-12-01 17:59:14 +01:00
Michael Klishin ce497a5b52
Merge pull request #3798 from wrobell/rabbitmq-streams-doc-protocol-timestamp
Link to most recent description of Osiris chunk format
2021-11-24 21:48:07 +03:00
wrobell 07dc62af7c Link to most recent description of Osiris chunk format
The most recent description of Osiris chunk format does not reference
the timestamp field to be "posix-ish" anymore. This was bit misleading
as it is Erlang's system time.

Add link to Erlang system time documentation to the subscription command
description to avoid confusion about the timestamp field.
2021-11-24 18:34:55 +00:00
Michael Klishin 01f68b80c9
Merge pull request #3723 from wrobell/rabbitmq-streams-doc-protocol-deliver-fix
Fix description of "deliver" command in RabbitMQ Streams protocol documentation
2021-11-14 21:59:12 +03:00
wrobell b4f39031f6 Fix description of "deliver" command in RabbitMQ Streams protocol documentation
Add missing fields - chunk type, timestamp, trailer length and reserved.
The fields are added after looking at

    3fd348d225/src/main/java/com/rabbitmq/stream/impl/ServerFrameHandler.java (L267)
    https://github.com/qweeze/rstream/blob/master/rstream/schema.py#L376

Change type of "version" field - it shall be uint16 like in other
commands.
2021-11-14 14:51:35 +00:00
Philip Kuryloski e4505bea3d Use unique group names in rabbit_stream_SUITE
Technically duplicate names is supported by common test, but we have
seen it contribute to flakiness in our suite in practice

(cherry picked from commit 513446b6d1)
2021-11-09 13:52:41 +01:00
Arnaud Cogoluègnes ba5cfb1bae
Use version range for stream client Maven dependency
In test suite. Note a snapshot for 1.0-SNAPSHOT has been
pushed by mistake a while ago, so this one must be
excluded. It's unlikely it will be erased, as the snapshots
for the first stable version should be 1.0.0-SNAPSHOT.
2021-11-03 14:05:23 +01:00
Michael Klishin 40db671e0b
Merge pull request #3633 from wrobell/rabbitmq-streams-doc-protocol-hex
Use hex values in RabbitMQ Streams protocol description
2021-11-02 21:58:42 +03:00
wrobell 08c417fdb7 Use hex values in RabbitMQ Streams protocol description
The protocol documentation uses decimal values for error and request key
codes.

Let's use hex values instead. This helps when looking at a request and
its response - 0x0006 and 0x8006 vs. 6 and 32774.

Also, when looking at output of protocol analysis tools like Wireshark,
a hexadecimal value will be printed, for example:

    "Nov  1, 2021 23:05:19.395825508 GMT","60216,5552","00000009000600010000000701"
    "Nov  1, 2021 23:05:19.396069528 GMT","5552,60216","0000000a80060001000000070001"

Above, we can visually identify delete publisher request and response
(0x0006 and 0x8006) and easily match them in the documentation of the
protocol.

Finally, above argument applies to logging as it is common to log
hex values, not decimal.
2021-11-02 18:31:30 +00:00
Philip Kuryloski 76287fb30c format with bazel files with buildifier 2021-11-02 18:24:26 +01:00
wrobell e74823d68c Fix use of "RabbitMQ Streams" in protocol documentation
The following RabbitMQ page uses "RabbitMQ Streams" or "streams",
therefore use it in the protocol documentation as well

    https://blog.rabbitmq.com/posts/2021/07/rabbitmq-streams-overview
2021-10-29 19:06:06 +01:00
wrobell 57ff5dbf4a Fix RabbitMQ Streams protocol documentation
1. Response for publisher declaration request does not contain
   publisher id.

2. Add mechanism entry to the details of SASL handshake request.

3. SASL handshake response contains list of mechanisms, not just single
   mechanism.
2021-10-28 17:53:22 +01:00
Arnaud Cogoluègnes c0f1e23b17
Disable stream test in mixed versions mode
Not worth testing in this mode, as the stream plugin
does not start without the stream_queue feature flag.
2021-10-20 15:16:45 +02:00
Arnaud Cogoluègnes 030a08fa54
Enable stream_queue flag with env variable in test suite
For mixed version mode, where feature flags are
not explictely enabled.
2021-10-20 15:08:05 +02:00
Arnaud Cogoluègnes d87fa91a52
Enable stream_queue flag in test suite
For mixed version mode, where feature flags are
not explictely enabled.
2021-10-20 14:55:54 +02:00
Arnaud Cogoluègnes 6aabfdbbb4
Format stream plugin 2021-10-20 14:53:08 +02:00
Michael Klishin 2140562209
Merge pull request #3562 from rabbitmq/disable_stream_plugin_3557
Disable the stream_plugin if stream feature flag is not enabled
2021-10-14 18:45:26 +03:00
Arnaud Cogoluègnes 6d0dc3f6b4
Attach add/delete super stream command to streams scope only
Not to ctl.

References #3503
2021-10-14 17:09:18 +02:00
Michael Klishin 2fbc5fab25
Formatting, wording 2021-10-14 17:20:59 +03:00
Gabriele Santomaggio f2df98aef9 change message 2021-10-13 12:01:21 +02:00
Gabriele Santomaggio 2eb625531d Disable the stream_plugin and stream management if the feature flag
is not enabled.
fixes https://github.com/rabbitmq/rabbitmq-server/issues/3557
2021-10-13 11:59:11 +02:00
Gabriele Santomaggio c270ebf037 Disable the stream_plugin if the feature flag
is not enabled.
fixes https://github.com/rabbitmq/rabbitmq-server/issues/3557
2021-10-12 19:09:34 +02:00
Arnaud Cogoluègnes 6b9589bae4
Handle stream arguments in add_super_stream command
max-age, leader-locator, etc.
2021-10-11 16:50:03 +02:00
Arnaud Cogoluègnes ecbd969834
Embed RPC call function in test 2021-10-11 16:50:02 +02:00
Arnaud Cogoluègnes 8ae60bef73
Fix group spec in test 2021-10-11 16:50:02 +02:00
Arnaud Cogoluègnes b0bd5f8a00
Add delete_super_stream CLI command 2021-10-11 16:50:02 +02:00
Arnaud Cogoluègnes a73b1a3d0d
Add add_super_stream CLI command 2021-10-11 16:50:01 +02:00
Arnaud Cogoluègnes 147659093f
Add functions to create/delete super stream in manager 2021-10-11 16:50:01 +02:00
Michael Klishin 72c2e152ef
Merge pull request #3499 from rabbitmq/cla-url
Update CLA URL
2021-10-01 00:16:03 +03:00
Arnaud Cogoluègnes e8d0d380a2
Bump dependencies in stream Java tests 2021-09-28 14:02:10 +02:00
Arnaud Cogoluègnes fc80138204
Fallback to rabbit_stream:host/0 if advertised_tls_host not set
The advertised_host must also be tried for TLS connections.

References #3514
2021-09-28 14:00:41 +02:00
Carl Hörberg 52791c677b
Support for advertising different hostname for TLS stream connections
Use case: Allow plain connections over one (internal IP), and TLS
connections over another IP (eg. internet routable IP). Without this
patch a cluster can only support access over one or the other IP, not
both.

(cherry picked from commit b9e6aad035)
2021-09-28 13:47:22 +02:00
Michal Kuratczyk 1a2126676d Update CLA URL 2021-09-23 14:29:36 +02:00
Michael Klishin 0f6a9dac27
Introduce rabbit_nodes:all/0 2021-09-20 22:24:25 +03:00
Arnaud Cogoluègnes ab2c565b49
Merge pull request #3423 from rabbitmq/streams-sort-partitions
Sort stream partitions using binding parameter
2021-09-15 14:56:17 +02:00
Arnaud Cogoluègnes faee72552a
Polish test 2021-09-15 11:56:53 +02:00
Arnaud Cogoluègnes 4bbed58352
Add Bazel rule for rabbit_stream_utils_SUITE 2021-09-15 11:38:17 +02:00
Arnaud Cogoluègnes 5b83dceb87
Return only streams for partition-related commands
The stream partition metadata is based on bindings,
so we make sure to return only streams from the binding
information.
2021-09-15 11:33:04 +02:00
Arnaud Cogoluègnes b140395864
Fall back to HOSTNAME(1) in stream Java test suite
In case the Java way fails (only on new CI).
2021-09-15 10:36:27 +02:00
Arnaud Cogoluègnes 04a0653571
Sort stream partitions using binding parameter
If present. To make the partition order stable.
2021-09-14 18:02:22 +02:00
Michael Klishin c8d483809e
Resolve a missed conflict 2021-09-13 20:33:11 +03:00
Michael Klishin f79bc1c935
Merge branch 'master' into stream-reader-close-in-terminate
Conflicts:
	deps/rabbitmq_stream/src/rabbit_stream_reader.erl
2021-09-13 20:27:58 +03:00
Arnaud Cogoluègnes 8f207e3c5f
Make stream protocol route command return several streams
We expect to have 1 stream for each routing key, but
as binding can return several queues for a given key we
let that possibility open in the stream protocol.
2021-09-13 17:53:25 +02:00