8f207e3c5f
Make stream protocol route command return several streams
...
We expect to have 1 stream for each routing key, but
as binding can return several queues for a given key we
let that possibility open in the stream protocol.
2021-09-13 17:53:25 +02:00
3513fa0ea8
rabbitmq_stream formatting
2021-09-09 09:45:13 +01:00
902fa429dd
Use awaitMatch to check global counters
...
Assertion fails on CI environment.
2021-09-01 09:51:06 +02:00
6c0ba03d61
Test that we start from 0 publishers & consumers
...
Pair: @kjnilsson
Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2021-08-31 15:29:16 +01:00
0ecf3d4eeb
Test stream publisher & consumer counters
...
Pair: @kjnilsson
Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2021-08-31 15:29:16 +01:00
8dc024089c
Bump dependencies in Java stream tests
2021-08-31 08:32:53 +02:00
ace3ee9cd9
Bump Java stream client used in tests
2021-08-10 11:32:57 +03:00
863b899079
Remove TEST macro
...
since it fails with Bazel.
As discussed with @pjk25, let's set this value via application env,
make it configurable to the test, but not configurable to the user.
2021-07-19 16:42:54 +02:00
13b03b8530
Remove unused variable warning
2021-07-19 14:36:22 +02:00
3964da37b4
Close TCP connection when stream reader times out
...
Add state timeouts.
If the client takes more than 10s for a single step in the authentication
protocol, make the server close the TCP connection.
Also close the TCP connection if the server times out in state
close_sent. That's the case when the client sends an invalid command
(after successful authentication), the server requests the client to
close the connection, but the client doesn't respond anymore.
2021-07-15 19:29:24 +02:00
b636ad2565
Rename protocol error counters to _total
2021-06-30 12:46:41 +02:00
58e36b6417
Add specific stream protocol counters to track protocol errors
2021-06-29 12:50:00 +02:00
a3c97d491f
Update additional test skipping for 3.8/3.9 mixed versions
2021-06-25 11:17:46 +02:00
bb75157fc1
Mark deps/rabbitmq_stream:commands_SUITE as flaky
2021-06-24 12:56:20 +02:00
38f474688f
Stream common library
2021-06-11 17:24:00 +02:00
0d84dc1e48
Merge pull request #3098 from rabbitmq/unexpected-socket-input
...
Close stream socket if client doesn't follow authentication protocol
2021-06-11 03:25:45 +03:00
fcc8dbeab6
Close client connections that don't follow authentication protocol
...
Before this commit sending garbarge data to the server stream port
caused the RabbitMQ node to eat more and more memory.
In this commit, we fix it by expecting the client to go through the
proper authentication sequence. Otherwise, the server closes the socket.
Co-authored-by: Michal Kuratczyk <mkuratczyk@pivotal.io>
2021-06-10 15:44:51 +02:00
dcd65572a0
Remove correlation ID from commit_offset
...
In stream protocol. commit_offset is asynchronous and does
not expect a response, so the correlation ID is not required.
2021-06-10 15:21:38 +02:00
7d5c8f402a
Merge pull request #3071 from rabbitmq/subscribe-props
...
Fix properties binary construction
2021-06-03 18:09:52 +02:00
761af0a7a0
Extract publishing IDs from batch publishing
...
In stream plugin, to e.g. send publish errors in case the stream
does not exist. Batches were not taken into account.
2021-05-31 15:35:10 +02:00
65c9dae53f
Fix properties binary construction
2021-05-28 15:45:19 +02:00
f36751aa6d
make rabbit_stream_SUITE more reliable
...
By having rabbit_stream_core cache it's incoming command internally.
2021-05-27 13:15:55 +01:00
f251815002
Replace rabbitmq_stream test helper with common version
...
from rabbitmq_ct_helpers
and update default app env for bazel, to match Makefile
2021-05-27 12:26:51 +02:00
2ab5cb22ca
Expose TLS info for stream connections (CLI, REST API)
2021-05-27 10:43:33 +02:00
69ad6969e6
Add stream.advertised_tls_port setting
2021-05-26 11:08:43 +02:00
05bd6dd838
Test multiple chunks
2021-05-25 14:25:38 +02:00
4a9d8115f8
rebase fixes
...
post rebase test fixes
Make socket initialisation more lenient
correct return types
fix
remove commented code
2021-05-24 15:53:10 +01:00
3fefa8e8d4
Use ssl option when initialising data reader
2021-05-21 17:13:15 +01:00
8f54150867
Add stream TLS test
2021-05-21 17:10:55 +01:00
03063f2eed
Merge pull request #3043 from rabbitmq/streams-consumer-lag-metrics
...
Add consumer offset_lag to rabbitmq-stream CLI command & Management
2021-05-21 16:38:50 +01:00
080b0771cf
Fix test_gc_consumers test
...
Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2021-05-21 13:02:48 +01:00
8f8e08a9a9
Send advertised host and port in open
...
More secure than in peer properties frame, which is just
at the beginning of the connection opening.
2021-05-21 13:03:38 +02:00
c42930acb0
Set stream plugin default port to 5552
2021-05-19 15:38:52 +02:00
ef52b92390
Make stream consumer arg parsing return error
...
when receiving unexpected input
2021-05-19 12:00:10 +01:00
67546a60ca
Fix stream test suite
2021-05-18 17:03:21 +02:00
28f3de7f2a
Support properties in stream subscription
...
To add metadata. Can be used to map a subscription to
the commit offset reference.
2021-05-18 10:15:26 +02:00
79502ef663
fix tests
2021-05-17 10:39:55 +01:00
a5cc3b28e4
remove unused function
2021-05-17 10:39:55 +01:00
b21966cf86
stream plugin: use core module in test suite
...
this implements a basic "client" and tests the core module on both server and client.
2021-05-17 10:39:55 +01:00
a1ce34c1b8
Extract stream protocol framing and binary generation
...
into a separate module.
2021-05-17 10:39:52 +01:00
e5101c2bfd
Bump JUnit in stream-related Java test suite
2021-05-17 09:32:30 +02:00
3b69f4b630
Handle 4 reserved bytes in osiris chunk
...
References rabbitmq/osiris#22
2021-03-30 09:45:29 +02:00
e46216b5a8
Check PID on leader lookup in stream plugin
...
To make sure the PID is alive, as the mnesia record can stale after a
failure.
Make also the local PID lookup in the stream coordinator do a consistent
query over the cluster if the PID is not alive.
Co-authored-by: Karl Nilsson <kjnilsson@users.noreply.github.com>
2021-03-12 15:04:40 +00:00
e80db9d46a
Adapt stream plugin after coordinator refactoring
...
Do not use mnesia record for topology info, use coordinator instead.
2021-03-12 15:04:35 +00:00
9f0bec8def
Start constants at 1 for stream plugin
2021-02-26 11:48:45 +01:00
768a61792e
Use MSB in keys to distinguish requests/responses
...
In stream protocol.
2021-02-26 11:03:36 +01:00
b39f4a6d1d
Set stream plugin default port to 5551
...
Instead of 5555, which is already registered.
2021-02-25 18:16:32 +01:00
d63756ed67
Set correct offset metrics on stream consumer creation
2021-02-09 11:36:20 +01:00
3de198845f
Disable stream failure test
2021-01-27 17:54:35 +01:00
ce7851b470
Disable stream failure test until coordinator is fixed
2021-01-21 16:35:13 +01:00
e87581f14c
Remove unnecessary dependencies
2021-01-19 15:04:59 +01:00
2f903843e0
Add list_stream_publishers CLI command
2021-01-19 14:49:30 +01:00
1e6244990c
Add list_stream_consumers CLI command
2021-01-19 11:31:39 +01:00
0d73b58be0
Re-order stream command constants
...
The order is more usage-oriented.
2021-01-18 11:25:21 +01:00
52aa2963f5
Bump test dependencies in stream plugins
2021-01-14 17:58:38 +01:00
5f0df3bdb0
Format stream and stream management plugins code
2021-01-06 15:47:27 +01:00
07b9e68ff5
Add logs to some stream tests
2021-01-06 14:38:02 +01:00
c0bb7ec4a7
Add offset & consumed metrics to stream consumer
2020-12-15 15:37:26 +01:00
e5ed53c5e2
Merge branch 'master' into rabbitmq-stream-management
2020-12-08 11:41:22 +01:00
87837f857d
Fix failure tests in stream plugin
2020-12-08 11:40:38 +01:00
846b36bec6
Fix metrics table GC test in stream plugin
2020-12-08 09:47:38 +01:00
1faeaad499
Add publisher REST endpoint to stream consumer management
2020-12-04 17:22:14 +01:00
224e9914b2
Merge branch 'master' into rabbitmq-stream-management
2020-12-04 10:26:42 +01:00
c51e060a07
Wait longer for replicas to come up in stream plugin
2020-12-04 09:48:56 +01:00
66867a4062
Add GC to stream consumer monitoring table
2020-12-02 18:12:45 +01:00
08891a734e
Merge branch 'master' into rabbitmq-stream-management
2020-11-30 09:42:54 +01:00
67308be13c
Remove stream field from publish command
...
Only the publisher ID is necessary now the publisher declaration is
mandatory before publishing.
2020-11-27 11:07:10 +01:00
9859714f0e
Declare publisher in stream test
2020-11-24 18:18:14 +01:00
584811582b
Add first tests for stream management
...
They are using Java to have a proper stream client.
2020-11-23 11:23:29 +01:00
a1f6006356
Support leader locator strategies
...
References rabbitmq/rabbitmq-server#2471
2020-10-21 15:25:31 +02:00
cc030ac195
Support initial-cluster-size argument on creation
...
See rabbitmq/rabbitmq-server#2467
2020-10-16 14:07:19 +02:00
27d06e8021
Add CLI command to list stream connections
2020-10-16 09:18:36 +02:00
685164be64
Add INI-type configuration support
2020-10-12 17:29:38 +02:00
b8bdb5ae7b
Bump dependencies in Java test suite
...
And apply Google Java Format with Maven Spotless plugin.
2020-09-29 12:05:29 +02:00
03a11e0771
Adapt failure tests
...
They are a bit more defensive. The subscription is also now more
reliable by returning a stream-not-available code if necessary.
Using also Aten poll interval to 1 second (bumped to 5 seconds in master
now).
2020-09-29 11:29:56 +02:00
c25d89d67a
Adapt test to additional chunk type field in chunk
2020-09-03 13:46:57 +02:00
21d9aea96c
Add publisher ID
2020-09-02 15:13:37 +02:00
db93e14932
Use byte for subscription ID
...
Instead of integer. This is to be consistent with the introduction of a
similar ID for publishers (which will be a byte as well).
2020-09-01 18:07:27 +02:00
58f6ebff2e
Refactor Java tests after Client package change
2020-08-06 14:01:02 +02:00
02d87cab5d
Fix frame reading in test
2020-07-23 11:40:20 +02:00
7740fee13e
Pass in monitor reference, not stream name
2020-07-09 11:33:44 +02:00
4714f682e5
Add consumer failure test
...
The node the consumer is connected to fails, the consumer connects
and starts consuming where it left off.
2020-07-06 17:42:46 +02:00
4f01776a65
Add publishing failure test
...
Stream leader node fails while publisher is publishing, publisher
reconnects on the new leader. A consumer should read all the confirmed
messages afterwards.
2020-07-03 16:10:38 +02:00
1597bdf14a
Disable failure test
2020-06-29 10:51:31 +02:00
30e51ca4cb
Add failure test
2020-06-25 16:25:02 +02:00
1d3978ae40
Change response code label
...
From "stream deleted" to "stream not available". It covers now the
deletion of a stream and the unavailibility due to a failure. This is up
to the client to find out what to do (typically send a metadata request
about the stream and see if it's still there).
2020-06-23 10:51:34 +02:00
5bd352737c
Pass in node information to Java test suite
2020-06-22 16:35:07 +02:00
50581b2255
Fix cluster tests
2020-06-11 18:12:31 +02:00
d3aa774156
Add Java-based tests for clustering
2020-06-11 15:38:07 +02:00
c39852fa72
Introduce PeerProperties command
...
To exchange client/server properties at the beginning of the connection.
2020-06-10 10:42:29 +02:00
d2a29c5aef
Support client properties
2020-06-09 17:48:23 +02:00
6ef10c8573
Use MPL 2.0
2020-06-08 09:54:33 +02:00
a56cf4013c
Support first/last/next/offset/timestamp offset spec
2020-06-02 16:35:00 +02:00
b54026aa32
Add arguments for stream creation
...
To handle retention settings.
2020-05-19 18:18:30 +02:00
2d19e85925
Use stream instead of target
2020-05-06 09:15:16 +02:00
204f61399b
Handle client heartbeat
2020-04-01 10:08:32 +02:00
ece2a4770f
Refactoring before introducing heartbeat
...
Introduce a level of supervisors to have a parent PID for the heartbeat
process.
2020-03-31 10:12:35 +02:00
e899bdfb3e
Refactor tune
...
Use int32 for both max frame size and bump max frame size to 1 MB.
2020-03-27 15:55:57 +01:00
670f5f639b
Add test for close
2020-03-27 09:26:02 +01:00
9667525cd7
Update test for opening sequence
2020-03-26 17:52:54 +01:00
1fd5270d01
Use constants in tests
2020-03-25 10:22:16 +01:00
b84de96d0b
Add test for authentication
2020-03-25 10:08:10 +01:00
4e8a801e8d
Add license information
2020-03-17 10:59:37 +01:00
ab98aecd17
Initial import
2020-03-16 16:40:02 +01:00
Arnaud Cogoluègnes
Karl Nilsson
Gerhard Lazu
Michael Klishin
David Ansari
dcorbacho
Philip Kuryloski
Michael Klishin
kjnilsson