8868ad97b3
Add logging to stream subscription
2021-02-11 16:16:26 +01:00
d63756ed67
Set correct offset metrics on stream consumer creation
2021-02-09 11:36:20 +01:00
3de198845f
Disable stream failure test
2021-01-27 17:54:35 +01:00
b921ac11a8
Merge pull request #2712 from rabbitmq/rabbitmq-stream-prometheus
...
Add stream prometheus plugin
2021-01-27 16:46:37 +01:00
268bc7257c
Merge pull request #2729 from rabbitmq/rabbitmq-stream-consumers-in-queue-details
...
Report stream consumers as regular consumers in management
2021-01-27 16:06:59 +01:00
ce7851b470
Disable stream failure test until coordinator is fixed
2021-01-21 16:35:13 +01:00
78b723d7c3
Report stream consumers as regular consumers in management
...
This helps to see that a queue is consumed from.
WIP
2021-01-20 15:36:16 +01:00
e87581f14c
Remove unnecessary dependencies
2021-01-19 15:04:59 +01:00
2f903843e0
Add list_stream_publishers CLI command
2021-01-19 14:49:30 +01:00
1e6244990c
Add list_stream_consumers CLI command
2021-01-19 11:31:39 +01:00
a559c79b8c
Add response codes to stream protocol page
2021-01-18 14:25:24 +01:00
b5315c0166
Merge branch 'master' into rabbitmq-stream-prometheus
2021-01-18 11:26:06 +01:00
0d73b58be0
Re-order stream command constants
...
The order is more usage-oriented.
2021-01-18 11:25:21 +01:00
52aa2963f5
Bump test dependencies in stream plugins
2021-01-14 17:58:38 +01:00
bf72683eb2
Add stream prometheus plugin
2021-01-11 16:49:56 +01:00
5f0df3bdb0
Format stream and stream management plugins code
2021-01-06 15:47:27 +01:00
07b9e68ff5
Add logs to some stream tests
2021-01-06 14:38:02 +01:00
fc88f3ba21
Check that the queue exists before calling declare
...
Rabbit channels are responsible of this check before calling declare,
skipping it on the manager meant that the queue was partly redeclared
and a new data directory created. The old one was still on disk with
a different timestamp, but from the user point of view the queue data
has been erased.
2020-12-23 12:18:33 +01:00
c0bb7ec4a7
Add offset & consumed metrics to stream consumer
2020-12-15 15:37:26 +01:00
c4d07467da
Merge branch 'master' into rabbitmq-stream-management
2020-12-09 12:00:56 +01:00
9a590647b3
Add chunk header parameter to send_file callback
2020-12-09 11:33:28 +01:00
e5ed53c5e2
Merge branch 'master' into rabbitmq-stream-management
2020-12-08 11:41:22 +01:00
87837f857d
Fix failure tests in stream plugin
2020-12-08 11:40:38 +01:00
846b36bec6
Fix metrics table GC test in stream plugin
2020-12-08 09:47:38 +01:00
2b45d8635b
Add publisher reference to stream endpoint
2020-12-07 10:17:01 +01:00
1faeaad499
Add publisher REST endpoint to stream consumer management
2020-12-04 17:22:14 +01:00
224e9914b2
Merge branch 'master' into rabbitmq-stream-management
2020-12-04 10:26:42 +01:00
c51e060a07
Wait longer for replicas to come up in stream plugin
2020-12-04 09:48:56 +01:00
88b4ddb6c2
Track publishers in stream plugin
2020-12-03 17:57:48 +01:00
c14c9bb6da
Fix some dialyzer warnings in stream plugin
2020-12-03 10:09:29 +01:00
66867a4062
Add GC to stream consumer monitoring table
2020-12-02 18:12:45 +01:00
41160210ab
Start stream consumer REST endpoint
2020-12-01 18:23:22 +01:00
08891a734e
Merge branch 'master' into rabbitmq-stream-management
2020-11-30 09:42:54 +01:00
67308be13c
Remove stream field from publish command
...
Only the publisher ID is necessary now the publisher declaration is
mandatory before publishing.
2020-11-27 11:07:10 +01:00
bfa541a095
Include publisher state management
2020-11-26 18:18:43 +01:00
8cf5fb0775
Enforce publisher ID uniqueness
2020-11-26 15:19:02 +01:00
59bc60fe36
Add query publisher sequence
2020-11-26 10:02:13 +01:00
9859714f0e
Declare publisher in stream test
2020-11-24 18:18:14 +01:00
8f97ea400a
Start adding publishing dedup support for streams
2020-11-24 17:48:41 +01:00
584811582b
Add first tests for stream management
...
They are using Java to have a proper stream client.
2020-11-23 11:23:29 +01:00
580beb21bf
Include publish dedup in stream protocol
2020-11-20 11:43:04 +01:00
c8249a304f
Filter stream connections where metrics are disabled
...
This implied defining the protocol field in tracked connection to be
able to filter out non-stream connections.
2020-11-20 09:29:55 +01:00
e516aab737
Display only stream connections in plugin
2020-11-19 17:02:35 +01:00
23d7e8114c
Introduce stream management plugin
2020-11-19 14:48:25 +01:00
1ed8f76eed
Check if member processes are alive in metadata command
...
In case the Mnesia record is stale.
2020-11-17 10:15:09 +01:00
29e74cbfca
Move some code to utils module
2020-11-05 15:33:13 +01:00
fa586140d3
Call rabbit_log:warning/2
2020-11-03 18:04:22 +01:00
d2eda4cb79
Update rabbitmq-components.mk
2020-11-03 14:27:50 +01:00
0f569d705a
Return Erlang node long name in metadata
...
Fixes #2
2020-11-03 11:45:22 +01:00
2813356682
Use non_amqp_connection name for force_refresh_event
2020-10-23 15:37:07 +02:00
a2f05f5b75
Support shutdown call from management plugin
2020-10-23 11:14:04 +02:00
bd191cf40d
Register connections and handle force_event_refresh
...
For existing connections that must emit stats when the management plugin
is enabled.
References rabbitmq/rabbitmq-server#2481
2020-10-23 10:17:39 +02:00
bb510b0e14
Merge branch 'master' into management-integration
...
Conflicts:
src/rabbit_stream_manager.erl
2020-10-22 15:41:42 +02:00
dd8b3ea008
Check stats timer in listen loop
2020-10-22 14:27:06 +02:00
43898e59b1
Support max-age in stream creation
2020-10-22 13:42:33 +02:00
a1f6006356
Support leader locator strategies
...
References rabbitmq/rabbitmq-server#2471
2020-10-21 15:25:31 +02:00
20681aca63
Add event notification during authentication
2020-10-20 18:20:03 +02:00
184f177d78
Emit stats for management
...
WIP
2020-10-20 15:09:34 +02:00
cc030ac195
Support initial-cluster-size argument on creation
...
See rabbitmq/rabbitmq-server#2467
2020-10-16 14:07:19 +02:00
b704e2f8ef
Check stream name before creation
...
Should not start with "amq.", be empty. Strips also newline
and carriage return characters.
2020-10-16 11:22:16 +02:00
27d06e8021
Add CLI command to list stream connections
2020-10-16 09:18:36 +02:00
9e6008a362
Add rabbitmq_ct_client_helpers dependency
2020-10-12 17:58:42 +02:00
c85f98cb7f
Add ct_client_helpers dependency
2020-10-12 17:51:17 +02:00
685164be64
Add INI-type configuration support
2020-10-12 17:29:38 +02:00
4420c77e8b
Set default heartbeat to 60 seconds
2020-10-12 16:05:34 +02:00
5e80aa9f39
Add correlation ID field in commit offset frame
...
Not used yet, but can avoid a command version change if the command
should have confirm response.
2020-10-02 14:16:22 +02:00
b8bdb5ae7b
Bump dependencies in Java test suite
...
And apply Google Java Format with Maven Spotless plugin.
2020-09-29 12:05:29 +02:00
03a11e0771
Adapt failure tests
...
They are a bit more defensive. The subscription is also now more
reliable by returning a stream-not-available code if necessary.
Using also Aten poll interval to 1 second (bumped to 5 seconds in master
now).
2020-09-29 11:29:56 +02:00
04e981e9e6
Use rabbit_durable_queue to check if stream exists
...
This is done after checking rabbit_queue and if it returns that the
queue does not exist. The coordinator may be recovering the queue, so
thanks to this double check we know the queue exists but is not
available, instead of thinking it does not exist at all.
2020-09-17 09:30:20 +02:00
cf5e99cd29
Add offset tracking commands
2020-09-15 17:52:05 +02:00
c25d89d67a
Adapt test to additional chunk type field in chunk
2020-09-03 13:46:57 +02:00
21d9aea96c
Add publisher ID
2020-09-02 15:13:37 +02:00
b2f30a55ba
Use byte for sub ID in credit notification
2020-09-01 18:14:07 +02:00
9a788362dd
Change sub ID to byte in protocol reference
2020-09-01 18:13:41 +02:00
db93e14932
Use byte for subscription ID
...
Instead of integer. This is to be consistent with the introduction of a
similar ID for publishers (which will be a byte as well).
2020-09-01 18:07:27 +02:00
ea2636cbb3
Add debug log during deletion
2020-08-07 16:43:34 +02:00
58f6ebff2e
Refactor Java tests after Client package change
2020-08-06 14:01:02 +02:00
40f2db4291
Add function to kill connection based on name
...
For test purposes.
2020-08-05 14:43:07 +02:00
02d87cab5d
Fix frame reading in test
2020-07-23 11:40:20 +02:00
a886cd8872
Send heartbeat from the reader process
...
If the heartbeat frame is sent from a dedicated process, it interleaves
between 2 socket calls from the reader process. Frames are typically
sent in one call, so this is fine, but a chunk is delivered with 2
calls, one for the frame header and one send_file for the chunk. So the
heartbeat frame can sneak in between these 2 calls, which makes clients
fail to parse frames.
2020-07-10 09:42:00 +02:00
92ebb825cb
Handle error case when creating a stream
2020-07-10 09:24:24 +02:00
7740fee13e
Pass in monitor reference, not stream name
2020-07-09 11:33:44 +02:00
4714f682e5
Add consumer failure test
...
The node the consumer is connected to fails, the consumer connects
and starts consuming where it left off.
2020-07-06 17:42:46 +02:00
4f01776a65
Add publishing failure test
...
Stream leader node fails while publisher is publishing, publisher
reconnects on the new leader. A consumer should read all the confirmed
messages afterwards.
2020-07-03 16:10:38 +02:00
af2dcf7f57
Handle publishing sub-entries (batch)
2020-06-30 17:29:28 +02:00
1597bdf14a
Disable failure test
2020-06-29 10:51:31 +02:00
7270125c0b
Fix closing sequence
...
When server sends close, it would not listen on the socket for the
client confirmation.
2020-06-26 16:33:39 +02:00
30e51ca4cb
Add failure test
2020-06-25 16:25:02 +02:00
4a05b6b62a
Ignore non-responding brokers in metadata command
2020-06-25 15:32:33 +02:00
1d3978ae40
Change response code label
...
From "stream deleted" to "stream not available". It covers now the
deletion of a stream and the unavailibility due to a failure. This is up
to the client to find out what to do (typically send a metadata request
about the stream and see if it's still there).
2020-06-23 10:51:34 +02:00
7b3d8c4ad0
Use Erlang monitor to detect stream deletion/failure
2020-06-22 18:33:54 +02:00
5bd352737c
Pass in node information to Java test suite
2020-06-22 16:35:07 +02:00
2fd743f2bd
Add credit response to protocol reference
2020-06-19 15:27:46 +02:00
f8074a9f93
Notify client when it credits an unknown subscription
2020-06-19 15:19:39 +02:00
27c4e1e707
Add authorisation on publishing
2020-06-18 17:00:02 +02:00
5f69ef7cf7
Add authorisation for subscription
2020-06-18 16:13:14 +02:00
397c5ca078
Add authorisation for creation/deletion
2020-06-18 15:25:50 +02:00
3c18498734
Clean manager listener code
...
Now the manager uses the coordinator to know about deleted streams.
2020-06-18 10:57:01 +02:00
ad165d78da
Adjust licensing
2020-06-18 09:44:30 +02:00
a1314a13c9
Use coordinator to get notified on stream deletion
2020-06-18 08:58:59 +02:00
625eaec004
Use rabbit_log
2020-06-15 11:17:27 +02:00
50581b2255
Fix cluster tests
2020-06-11 18:12:31 +02:00
157808ca8f
Make max_frame consistent between makefile and application
2020-06-11 16:53:14 +02:00
d3aa774156
Add Java-based tests for clustering
2020-06-11 15:38:07 +02:00
6a07d385ae
Add specs t stream manager
2020-06-11 11:05:02 +02:00
bb5efad9ef
Fix dialyzer errors
2020-06-10 18:13:55 +02:00
76162b18a2
Add type specifications
2020-06-10 16:24:18 +02:00
19a1eb4013
Remove some unnecessary comments
2020-06-10 15:06:51 +02:00
16288a2596
Separate changing state from (almost) immutable state
2020-06-10 15:04:31 +02:00
c39852fa72
Introduce PeerProperties command
...
To exchange client/server properties at the beginning of the connection.
2020-06-10 10:42:29 +02:00
d2a29c5aef
Support client properties
2020-06-09 17:48:23 +02:00
121f0e3575
Improve readme
2020-06-09 13:53:49 +02:00
6ef10c8573
Use MPL 2.0
2020-06-08 09:54:33 +02:00
a56cf4013c
Support first/last/next/offset/timestamp offset spec
2020-06-02 16:35:00 +02:00
9536d74f9d
Update link to osiris for data format
2020-06-02 11:13:19 +02:00
ecf47a8a42
Fix typo
2020-06-02 10:47:46 +02:00
2ab7e864dc
Consume from a local member
2020-05-28 11:59:13 +02:00
efc6fbc089
Add advertised host/port settings
...
This allows to communicate whatever it's appropriate to clients when
they issue the metadata command.
2020-05-28 10:47:30 +02:00
2f8559ae41
Polish documentation
2020-05-25 17:56:02 +02:00
838ef21b0d
Document authentication sequence
2020-05-25 17:53:34 +02:00
4faa853b00
Fix link in documentation
2020-05-25 17:04:00 +02:00
0f86e54b61
Cover all commands in protocol documentation
2020-05-25 17:02:43 +02:00
b7f70c6598
Fix links in documentation
2020-05-25 15:57:16 +02:00
848c95a181
Start documenting protocol
2020-05-25 15:55:51 +02:00
b54026aa32
Add arguments for stream creation
...
To handle retention settings.
2020-05-19 18:18:30 +02:00
55adca1b44
Make sure queue is stream queue
2020-05-07 11:00:45 +02:00
c2e35eb55f
Use stream coordinator to manage streams
2020-05-06 17:35:41 +02:00
2d19e85925
Use stream instead of target
2020-05-06 09:15:16 +02:00
b61a79b9ff
Credit value is signed
2020-04-03 11:00:35 +02:00
204f61399b
Handle client heartbeat
2020-04-01 10:08:32 +02:00
8c97d442ce
Add heartbeat
2020-03-31 14:43:06 +02:00
ece2a4770f
Refactoring before introducing heartbeat
...
Introduce a level of supervisors to have a parent PID for the heartbeat
process.
2020-03-31 10:12:35 +02:00
b693f71c5d
Add todo task
2020-03-30 17:46:27 +02:00
47d46d145b
Send close when client sends frame too large
2020-03-30 15:18:57 +02:00
e899bdfb3e
Refactor tune
...
Use int32 for both max frame size and bump max frame size to 1 MB.
2020-03-27 15:55:57 +01:00
670f5f639b
Add test for close
2020-03-27 09:26:02 +01:00
9667525cd7
Update test for opening sequence
2020-03-26 17:52:54 +01:00
4e035dfbdb
Add support to send close
2020-03-26 16:56:53 +01:00
a26962cfcb
Handle close command from client
2020-03-26 14:52:19 +01:00
7c33112f25
Check virtual host access in open
2020-03-26 09:53:28 +01:00
728a017f9e
Add skeleton for tune and open
2020-03-25 17:06:04 +01:00
1fd5270d01
Use constants in tests
2020-03-25 10:22:16 +01:00
b84de96d0b
Add test for authentication
2020-03-25 10:08:10 +01:00
55a836b488
Refactor to pre/post authentication frame handling
2020-03-24 17:42:10 +01:00
6ae44768f3
Handle null binary in SASL authenticate
2020-03-24 16:40:05 +01:00
22dc72c9d9
Handle SASL authenticate
2020-03-24 11:47:50 +01:00
e2f11fdafc
Handle SASL handshake
2020-03-23 14:54:59 +01:00
1f42faeab8
Refactor after some Osiris changes
2020-03-23 10:48:46 +01:00
9f64d3d490
Add instructions to build and run
2020-03-17 11:37:04 +01:00
4e8a801e8d
Add license information
2020-03-17 10:59:37 +01:00
ab98aecd17
Initial import
2020-03-16 16:40:02 +01:00
b9de5bc80b
Initial commit
2020-03-16 16:21:18 +01:00
Arnaud Cogoluègnes
dcorbacho