2022-02-17 14:35:36 +08:00
|
|
|
<?xml version="1.0"?>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<!DOCTYPE suppressions PUBLIC
|
|
|
|
|
"-//Puppy Crawl//DTD Suppressions 1.1//EN"
|
|
|
|
|
"http://www.puppycrawl.com/dtds/suppressions_1_1.dtd">
|
2022-02-17 14:35:36 +08:00
|
|
|
<!--
|
|
|
|
|
Licensed to the Apache Software Foundation (ASF) under one or more
|
|
|
|
|
contributor license agreements. See the NOTICE file distributed with
|
|
|
|
|
this work for additional information regarding copyright ownership.
|
|
|
|
|
The ASF licenses this file to You under the Apache License, Version 2.0
|
|
|
|
|
(the "License"); you may not use this file except in compliance with
|
|
|
|
|
the License. You may obtain a copy of the License at
|
|
|
|
|
|
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
2017-03-01 06:55:46 +08:00
|
|
|
|
2022-02-17 14:35:36 +08:00
|
|
|
Unless required by applicable law or agreed to in writing, software
|
|
|
|
|
distributed under the License is distributed on an "AS IS" BASIS,
|
|
|
|
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
|
|
|
See the License for the specific language governing permissions and
|
|
|
|
|
limitations under the License.
|
|
|
|
|
-->
|
2017-03-01 06:55:46 +08:00
|
|
|
<suppressions>
|
|
|
|
|
|
2017-06-28 08:47:00 +08:00
|
|
|
<!-- Note that [/\\] must be used as the path separator for cross-platform support -->
|
|
|
|
|
|
2019-09-25 23:58:54 +08:00
|
|
|
<!-- Generator -->
|
|
|
|
|
<suppress checks="CyclomaticComplexity|BooleanExpressionComplexity"
|
2022-05-26 07:25:16 +08:00
|
|
|
files="(SchemaGenerator|MessageDataGenerator|FieldSpec|FieldType).java"/>
|
2019-09-25 23:58:54 +08:00
|
|
|
<suppress checks="NPathComplexity"
|
2020-05-28 14:49:57 +08:00
|
|
|
files="(MessageDataGenerator|FieldSpec|WorkerSinkTask).java"/>
|
2019-09-25 23:58:54 +08:00
|
|
|
<suppress checks="JavaNCSS"
|
2021-08-31 06:39:25 +08:00
|
|
|
files="(ApiMessageType|FieldSpec|MessageDataGenerator|KafkaConsumerTest).java"/>
|
2019-11-21 08:40:18 +08:00
|
|
|
<suppress checks="MethodLength"
|
2021-01-12 06:17:52 +08:00
|
|
|
files="(FieldSpec|MessageDataGenerator).java"/>
|
2020-08-06 04:52:50 +08:00
|
|
|
<suppress id="dontUseSystemExit"
|
|
|
|
|
files="MessageGenerator.java"/>
|
2019-09-25 23:58:54 +08:00
|
|
|
|
2020-11-18 14:49:04 +08:00
|
|
|
<!-- core -->
|
|
|
|
|
<suppress checks="(NPathComplexity|ClassFanOutComplexity|CyclomaticComplexity|ClassDataAbstractionCoupling|FinalLocalVariable|LocalVariableName|MemberName|ParameterName|MethodLength|JavaNCSS|AvoidStarImport)"
|
|
|
|
|
files="core[\\/]src[\\/](generated|generated-test)[\\/].+.java$"/>
|
MINOR: Make ReplicaManager, LogManager, KafkaApis easier to construct (#11320)
The ReplicaManager, LogManager, and KafkaApis class all have many
constructor parameters. It is often difficult to add or remove a
parameter, since there are so many locations that need to be updated. In
order to address this problem, we should use named parameters when
constructing these objects from Scala code. This will make it easy to
add new optional parameters without modifying many test cases. It will
also make it easier to read git diffs and PRs, since the parameters will
have names next to them. Since Java does not support named paramters,
this PR adds several Builder classes which can be used to achieve the
same effect.
ReplicaManager also had a secondary constructor, which this PR removes.
The function of the secondary constructor was just to provide some
default parameters for the main constructor. However, it is simpler just
to actually use default parameters.
Reviewers: David Arthur <mumrah@gmail.com>
2021-09-18 05:12:31 +08:00
|
|
|
<suppress checks="NPathComplexity" files="(ClusterTestExtensions|KafkaApisBuilder).java"/>
|
2020-11-18 14:49:04 +08:00
|
|
|
|
2017-03-01 06:55:46 +08:00
|
|
|
<!-- Clients -->
|
2020-08-06 04:52:50 +08:00
|
|
|
<suppress id="dontUseSystemExit"
|
|
|
|
|
files="Exit.java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
<suppress checks="ClassFanOutComplexity"
|
2020-12-22 10:15:15 +08:00
|
|
|
files="(Fetcher|Sender|SenderTest|ConsumerCoordinator|KafkaConsumer|KafkaProducer|Utils|TransactionManager|TransactionManagerTest|KafkaAdminClient|NetworkClient|Admin|KafkaRaftClient|KafkaRaftClientTest|RaftClientTestContext).java"/>
|
2017-09-16 00:16:29 +08:00
|
|
|
<suppress checks="ClassFanOutComplexity"
|
|
|
|
|
files="(SaslServerAuthenticator|SaslAuthenticatorTest).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
<suppress checks="ClassFanOutComplexity"
|
2017-06-28 08:47:00 +08:00
|
|
|
files="Errors.java"/>
|
2017-03-25 03:38:36 +08:00
|
|
|
<suppress checks="ClassFanOutComplexity"
|
2017-06-28 08:47:00 +08:00
|
|
|
files="Utils.java"/>
|
2017-04-07 18:22:09 +08:00
|
|
|
<suppress checks="ClassFanOutComplexity"
|
2017-06-28 08:47:00 +08:00
|
|
|
files="AbstractRequest.java"/>
|
2017-04-07 18:22:09 +08:00
|
|
|
<suppress checks="ClassFanOutComplexity"
|
2017-06-28 08:47:00 +08:00
|
|
|
files="AbstractResponse.java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="MethodLength"
|
KAFKA-12984: make AbstractStickyAssignor resilient to invalid input, utilize generation in cooperative, and fix assignment bug (#10985)
1) Bring the generation field back to the CooperativeStickyAssignor so we don't need to rely so heavily on the ConsumerCoordinator properly updating its SubscriptionState after eg falling out of the group. The plain StickyAssignor always used the generation since it had to, so we just make sure the CooperativeStickyAssignor has this tool as well
2) In case of unforeseen problems or further bugs that slip past the generation field safety net, the assignor will now explicitly look out for partitions that are being claimed by multiple consumers as owned in the same generation. Such a case should never occur, but if it does, we have to invalidate this partition from the ownedPartitions of both consumers, since we can't tell who, if anyone, has the valid claim to this partition.
3) Fix a subtle bug that I discovered while writing tests for the above two fixes: in the constrained algorithm, we compute the exact number of partitions each consumer should end up with, and keep track of the "unfilled" members who must -- or might -- require more partitions to hit their quota. The problem was that members at the minQuota were being considered as "unfilled" even after we had already hit the maximum number of consumers allowed to go up to the maxQuota, meaning those minQuota members could/should not accept any more partitions beyond that. I believe this was introduced in #10509, so it shouldn't be in any released versions and does not need to be backported.
Reviewers: Guozhang Wang <guozhang@apache.org>, Luke Chen <showuon@gmail.com>
2021-07-14 09:29:31 +08:00
|
|
|
files="(KerberosLogin|RequestResponseTest|ConnectMetricsRegistry|KafkaConsumer|AbstractStickyAssignor).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="ParameterNumber"
|
2020-09-23 02:32:44 +08:00
|
|
|
files="(NetworkClient|FieldSpec|KafkaRaftClient).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
<suppress checks="ParameterNumber"
|
|
|
|
|
files="KafkaConsumer.java"/>
|
|
|
|
|
<suppress checks="ParameterNumber"
|
|
|
|
|
files="Fetcher.java"/>
|
2017-09-06 08:36:53 +08:00
|
|
|
<suppress checks="ParameterNumber"
|
|
|
|
|
files="Sender.java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
<suppress checks="ParameterNumber"
|
|
|
|
|
files="ConfigDef.java"/>
|
2017-03-25 03:38:36 +08:00
|
|
|
<suppress checks="ParameterNumber"
|
|
|
|
|
files="DefaultRecordBatch.java"/>
|
2017-08-26 01:38:15 +08:00
|
|
|
<suppress checks="ParameterNumber"
|
2021-09-17 00:17:15 +08:00
|
|
|
files="MemoryRecordsBuilder.java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="ClassDataAbstractionCoupling"
|
2020-09-23 02:32:44 +08:00
|
|
|
files="(KafkaConsumer|ConsumerCoordinator|Fetcher|KafkaProducer|AbstractRequest|AbstractResponse|TransactionManager|Admin|KafkaAdminClient|MockAdminClient|KafkaRaftClient|KafkaRaftClientTest).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
<suppress checks="ClassDataAbstractionCoupling"
|
2018-04-16 15:51:33 +08:00
|
|
|
files="(Errors|SaslAuthenticatorTest|AgentTest|CoordinatorTest).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="BooleanExpressionComplexity"
|
KAFKA-7862 & KIP-345 part-one: Add static membership logic to JoinGroup protocol (#6177)
This is the first diff for the implementation of JoinGroup logic for static membership. The goal of this diff contains:
* Add group.instance.id to be unique identifier for consumer instances, provided by end user;
Modify group coordinator to accept JoinGroupRequest with/without static membership, refactor the logic for readability and code reusability.
* Add client side support for incorporating static membership changes, including new config for group.instance.id, apply stream thread client id by default, and new join group exception handling.
* Increase max session timeout to 30 min for more user flexibility if they are inclined to tolerate partial unavailability than burdening rebalance.
* Unit tests for each module changes, especially on the group coordinator logic. Crossing the possibilities like:
6.1 Dynamic/Static member
6.2 Known/Unknown member id
6.3 Group stable/unstable
6.4 Leader/Follower
The rest of the 345 change will be broken down to 4 separate diffs:
* Avoid kicking out members through rebalance.timeout, only do the kick out through session timeout.
* Changes around LeaveGroup logic, including version bumping, broker logic, client logic, etc.
* Admin client changes to add ability to batch remove static members
* Deprecate group.initial.rebalance.delay
Reviewers: Liquan Pei <liquanpei@gmail.com>, Stanislav Kozlovski <familyguyuser192@windowslive.com>, Jason Gustafson <jason@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
2019-04-27 02:44:38 +08:00
|
|
|
files="(Utils|Topic|KafkaLZ4BlockOutputStream|AclData|JoinGroupRequest).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="CyclomaticComplexity"
|
KAFKA-13111: Re-evaluate Fetch Sessions when using topic IDs (#11331)
With the changes for topic IDs, we have a different flow. When a broker receives a request, it uses a map to convert the topic ID to topic names. If the topic ID is not found in the map, we return a top level error and close the session. This decision was motivated by the difficulty to store “unresolved” partitions in the session. In earlier iterations we stored an “unresolved” partition object in the cache, but it was somewhat hard to reason about and required extra logic to try to resolve the topic ID on each incremental request and add to the session. It also required extra logic to forget the topic (either by topic ID if the topic name was never known or by topic name if it was finally resolved when we wanted to remove from the session.)
One helpful simplifying factor is that we only allow one type of request (uses topic ID or does not use topic ID) in the session. That means we can rely on a session continuing to have the same information. We don’t have to worry about converting topics only known by name to topic ID for a response and we won’t need to convert topics only known by ID to name for a response.
This PR introduces a change to store the "unresolved partitions" in the cached partition object. If a version 13+ request is sent with a topic ID that is unknown, a cached partition will be created with that fetch request data and a null topic name. On subsequent incremental requests, unresolved partitions may be resolved with the new IDs found in the metadata cache. When handling the request, getting all partitions will return a TopicIdPartition object that will be used to handle the request and build the response. Since we can rely on only one type of request (with IDs or without), the cached partitions map will have different keys depending on what fetch request version is being used.
This PR involves changes both in FetchSessionHandler and FetchSession. Some major changes are outlined below.
1. FetchSessionHandler: Forgetting a topic and adding a new topic with the same name - We may have a case where there is a topic foo with ID 1 in the session. Upon a subsequent metadata update, we may have topic foo with ID 2. This means that topic foo has been deleted and recreated. When sending fetch requests version 13+ we will send a request to add foo ID 2 to the session and remove foo ID 1. Otherwise, we will fall back to the same behavior for versions 12 and below
2. FetchSession: Resolving in Incremental Sessions - Incremental sessions contain two distinct sets of partitions. Partitions that are sent in the latest request that are new/updates/forgotten partitions and the partitions already in the session. If we want to resolve unknown topic IDs we will need to handle both cases.
* Partitions in the request - These partitions are either new or updating/forgetting previous partitions in the session. The new partitions are trivial. We either have a resolved partition or create a partition that is unresolved. For the other cases, we need to be a bit more careful.
* For updated partitions we have a few cases – keep in mind, we may not programmatically know if a partition is an update:
1. partition in session is resolved, update is resolved: trivial
2. partition in session is unresolved, update is unresolved: in code, this is equivalent to the case above, so trivial as well
3. partition in session is unresolved, update is resolved: this means the partition in the session does not have a name, but the metadata cache now contains the name – to fix this we can check if there exists a cached partition with the given ID and update it both with the partition update and with the topic name.
4. partition in session is resolved, update is unresolved: this means the partition in the session has a name, but the update was unable to be resolved (ie, the topic is deleted) – this is the odd case. We will look up the partition using the ID. We will find the old version with a name but will not replace the name. This will lead to an UNKNOWN_TOPIC_OR_PARTITION or INCONSISTENT_TOPIC_ID error which will be handled with a metadata update. Likely a future request will forget the partition, and we will be able to do so by ID.
5. Two partitions in the session have IDs, but they are different: only one topic ID should exist in the metadata at a time, so likely only one topic ID is in the fetch set. The other one should be in the toForget. We will be able to remove this partition from the session. If for some reason, we don't try to forget this partition — one of the partitions in the session will cause an inconsistent topic ID error and the metadata for this partition will be refreshed — this should result in the old ID being removed from the session. This should not happen if the FetchSessionHandler is correctly in sync.
* For the forgotten partitions we have the same cases:
1. partition in session is resolved, forgotten is resolved: trivial
2. partition in session is unresolved, forgotten is unresolved: in code, this is equivalent to the case above, so trivial as well
3. partition in session is unresolved, forgotten is resolved: this means the partition in the session does not have a name, but the metadata cache now contains the name – to fix this we can check if there exists a cached partition with the given ID and try to forget it before we check the resolved name case.
4. partition in session is resolved, update is unresolved: this means the partition in the session has a name, but the update was unable to be resolved (ie, the topic is deleted) We will look up the partition using the ID. We will find the old version with a name and be able to delete it.
5. both partitions in the session have IDs, but they are different: This should be the same case as described above. If we somehow do not have the ID in the session, no partition will be removed. This should not happen unless the Fetch Session Handler is out of sync.
* Partitions in the session - there may be some partitions in the session already that are unresolved. We can resolve them in forEachPartition using a method that checks if the partition is unresolved and tries to resolve it using a topicName map from the request. The partition will be resolved before the function using the cached partition is applied.
Reviewers: David Jacot <djacot@confluent.io>
2021-11-15 17:04:43 +08:00
|
|
|
files="(ConsumerCoordinator|Fetcher|KafkaProducer|ConfigDef|KerberosLogin|AbstractRequest|AbstractResponse|Selector|SslFactory|SslTransportLayer|SaslClientAuthenticator|SaslClientCallbackHandler|SaslServerAuthenticator|AbstractCoordinator|TransactionManager|AbstractStickyAssignor|DefaultSslEngineFactory|Authorizer|RecordAccumulator|MemoryRecords|FetchSessionHandler).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="JavaNCSS"
|
2021-07-24 12:54:44 +08:00
|
|
|
files="(AbstractRequest|AbstractResponse|KerberosLogin|WorkerSinkTaskTest|TransactionManagerTest|SenderTest|KafkaAdminClient|ConsumerCoordinatorTest|KafkaAdminClientTest|KafkaRaftClientTest).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="NPathComplexity"
|
2021-07-08 07:02:37 +08:00
|
|
|
files="(ConsumerCoordinator|BufferPool|Fetcher|MetricName|Node|ConfigDef|RecordBatch|SslFactory|SslTransportLayer|MetadataResponse|KerberosLogin|Selector|Sender|Serdes|TokenInformation|Agent|Values|PluginUtils|MiniTrogdorCluster|TasksRequest|KafkaProducer|AbstractStickyAssignor|KafkaRaftClient|Authorizer|FetchSessionHandler).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
2019-01-25 01:26:51 +08:00
|
|
|
<suppress checks="(JavaNCSS|CyclomaticComplexity|MethodLength)"
|
|
|
|
|
files="CoordinatorClient.java"/>
|
2019-10-04 06:59:31 +08:00
|
|
|
<suppress checks="(UnnecessaryParentheses|BooleanExpressionComplexity|CyclomaticComplexity|WhitespaceAfter|LocalVariableName)"
|
|
|
|
|
files="Murmur3.java"/>
|
2019-01-25 01:26:51 +08:00
|
|
|
|
2020-08-27 06:10:09 +08:00
|
|
|
<suppress checks="(NPathComplexity|ClassFanOutComplexity|CyclomaticComplexity|ClassDataAbstractionCoupling|LocalVariableName|MemberName|ParameterName|MethodLength|JavaNCSS|AvoidStarImport)"
|
2019-09-25 23:58:54 +08:00
|
|
|
files="clients[\\/]src[\\/](generated|generated-test)[\\/].+.java$"/>
|
|
|
|
|
|
|
|
|
|
<suppress checks="NPathComplexity"
|
2021-07-07 18:55:00 +08:00
|
|
|
files="MessageTest.java|OffsetFetchRequest.java"/>
|
2019-09-25 23:58:54 +08:00
|
|
|
|
2020-08-06 04:52:50 +08:00
|
|
|
<!-- Clients tests -->
|
2017-03-01 06:55:46 +08:00
|
|
|
<suppress checks="ClassDataAbstractionCoupling"
|
2020-01-23 05:48:36 +08:00
|
|
|
files="(Sender|Fetcher|KafkaConsumer|Metrics|RequestResponse|TransactionManager|KafkaAdminClient|Message|KafkaProducer)Test.java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="ClassFanOutComplexity"
|
2020-01-23 05:48:36 +08:00
|
|
|
files="(ConsumerCoordinator|KafkaConsumer|RequestResponse|Fetcher|KafkaAdminClient|Message|KafkaProducer)Test.java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
2017-12-08 08:16:54 +08:00
|
|
|
<suppress checks="ClassFanOutComplexity"
|
|
|
|
|
files="MockAdminClient.java"/>
|
|
|
|
|
|
2021-12-11 01:54:52 +08:00
|
|
|
<suppress checks="CyclomaticComplexity"
|
|
|
|
|
files="RequestResponseTest.java"/>
|
|
|
|
|
|
2017-04-03 10:41:44 +08:00
|
|
|
<suppress checks="JavaNCSS"
|
2020-05-30 02:25:20 +08:00
|
|
|
files="RequestResponseTest.java|FetcherTest.java|KafkaAdminClientTest.java"/>
|
2017-04-03 10:41:44 +08:00
|
|
|
|
2018-05-22 14:17:42 +08:00
|
|
|
<suppress checks="NPathComplexity"
|
2021-03-02 22:20:47 +08:00
|
|
|
files="MemoryRecordsTest|MetricsTest|RequestResponseTest|TestSslUtils|AclAuthorizerBenchmark"/>
|
2018-05-22 14:17:42 +08:00
|
|
|
|
2019-10-04 06:59:31 +08:00
|
|
|
<suppress checks="(WhitespaceAround|LocalVariableName|ImportControl|AvoidStarImport)"
|
|
|
|
|
files="Murmur3Test.java"/>
|
|
|
|
|
|
2017-03-01 06:55:46 +08:00
|
|
|
<!-- Connect -->
|
|
|
|
|
<suppress checks="ClassFanOutComplexity"
|
2021-07-01 12:13:07 +08:00
|
|
|
files="(AbstractHerder|DistributedHerder|Worker).java"/>
|
2020-05-28 09:18:36 +08:00
|
|
|
<suppress checks="ClassFanOutComplexity"
|
|
|
|
|
files="Worker(|Test).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
<suppress checks="MethodLength"
|
2022-06-07 15:16:48 +08:00
|
|
|
files="(DistributedHerder|KafkaConfigBackingStore|Values|IncrementalCooperativeAssignor).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
<suppress checks="ParameterNumber"
|
2020-05-27 13:07:34 +08:00
|
|
|
files="Worker(SinkTask|SourceTask|Coordinator).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
<suppress checks="ParameterNumber"
|
|
|
|
|
files="ConfigKeyInfo.java"/>
|
|
|
|
|
|
|
|
|
|
<suppress checks="ClassDataAbstractionCoupling"
|
|
|
|
|
files="(RestServer|AbstractHerder|DistributedHerder).java"/>
|
|
|
|
|
|
|
|
|
|
<suppress checks="BooleanExpressionComplexity"
|
|
|
|
|
files="JsonConverter.java"/>
|
|
|
|
|
|
|
|
|
|
<suppress checks="CyclomaticComplexity"
|
2020-05-27 13:07:34 +08:00
|
|
|
files="(FileStreamSourceTask|DistributedHerder|KafkaConfigBackingStore).java"/>
|
KAFKA-5142: Add Connect support for message headers (KIP-145)
**[KIP-145](https://cwiki.apache.org/confluence/display/KAFKA/KIP-145+-+Expose+Record+Headers+in+Kafka+Connect) has been accepted, and this PR implements KIP-145 except without the SMTs.**
Changed the Connect API and runtime to support message headers as described in [KIP-145](https://cwiki.apache.org/confluence/display/KAFKA/KIP-145+-+Expose+Record+Headers+in+Kafka+Connect).
The new `Header` interface defines an immutable representation of a Kafka header (key-value pair) with support for the Connect value types and schemas. This interface provides methods for easily converting between many of the built-in primitive, structured, and logical data types.
The new `Headers` interface defines an ordered collection of headers and is used to track all headers associated with a `ConnectRecord` (and thus `SourceRecord` and `SinkRecord`). This does allow multiple headers with the same key. The `Headers` contains methods for adding, removing, finding, and modifying headers. Convenience methods allow connectors and transforms to easily use and modify the headers for a record.
A new `HeaderConverter` interface is also defined to enable the Connect runtime framework to be able to serialize and deserialize headers between the in-memory representation and Kafka’s byte[] representation. A new `SimpleHeaderConverter` implementation has been added, and this serializes to strings and deserializes by inferring the schemas (`Struct` header values are serialized without the schemas, so they can only be deserialized as `Map` instances without a schema.) The `StringConverter`, `JsonConverter`, and `ByteArrayConverter` have all been extended to also be `HeaderConverter` implementations. Each connector can be configured with a different header converter, although by default the `SimpleHeaderConverter` is used to serialize header values as strings without schemas.
Unit and integration tests are added for `ConnectHeader` and `ConnectHeaders`, the two implementation classes for headers. Additional test methods are added for the methods added to the `Converter` implementations. Finally, the `ConnectRecord` object is already used heavily, so only limited tests need to be added while quite a few of the existing tests already cover the changes.
Author: Randall Hauch <rhauch@gmail.com>
Reviewers: Arjun Satish <arjun@confluent.io>, Ted Yu <yuzhihong@gmail.com>, Magesh Nandakumar <magesh.n.kumar@gmail.com>, Konstantine Karantasis <konstantine@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes #4319 from rhauch/kafka-5142-b
2018-02-01 02:40:24 +08:00
|
|
|
<suppress checks="CyclomaticComplexity"
|
2021-04-12 17:49:24 +08:00
|
|
|
files="(JsonConverter|Values|ConnectHeaders).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="JavaNCSS"
|
2022-02-23 21:48:43 +08:00
|
|
|
files="(KafkaConfigBackingStore|Values|ConnectMetricsRegistry).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
2020-05-24 21:56:27 +08:00
|
|
|
<suppress checks="NPathComplexity"
|
2020-05-27 13:07:34 +08:00
|
|
|
files="(DistributedHerder|RestClient|RestServer|JsonConverter|KafkaConfigBackingStore|FileStreamSourceTask|TopicAdmin).java"/>
|
2020-05-24 21:56:27 +08:00
|
|
|
|
2017-03-01 06:55:46 +08:00
|
|
|
<!-- connect tests-->
|
|
|
|
|
<suppress checks="ClassDataAbstractionCoupling"
|
2021-03-19 22:03:36 +08:00
|
|
|
files="(DistributedHerder|KafkaBasedLog|WorkerSourceTaskWithTopicCreation|WorkerSourceTask)Test.java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
2019-09-26 00:23:03 +08:00
|
|
|
<suppress checks="ClassFanOutComplexity"
|
2020-05-27 13:07:34 +08:00
|
|
|
files="(WorkerSink|WorkerSource|ErrorHandling)Task(|WithTopicCreation)Test.java"/>
|
|
|
|
|
<suppress checks="ClassFanOutComplexity"
|
|
|
|
|
files="DistributedHerderTest.java"/>
|
|
|
|
|
|
|
|
|
|
<suppress checks="MethodLength"
|
|
|
|
|
files="(RequestResponse|WorkerSinkTask)Test.java"/>
|
2019-09-26 00:23:03 +08:00
|
|
|
|
2021-05-06 05:11:15 +08:00
|
|
|
<suppress checks="JavaNCSS"
|
|
|
|
|
files="DistributedHerderTest.java"/>
|
|
|
|
|
|
2017-03-01 06:55:46 +08:00
|
|
|
<!-- Streams -->
|
|
|
|
|
<suppress checks="ClassFanOutComplexity"
|
2022-04-19 23:29:28 +08:00
|
|
|
files="(KafkaStreams|KStreamImpl|KTableImpl|InternalTopologyBuilder|StreamsPartitionAssignor|StreamThread|IQv2StoreIntegrationTest|KStreamImplTest).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="MethodLength"
|
2020-06-18 08:04:43 +08:00
|
|
|
files="KTableImpl.java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
2020-02-05 13:06:39 +08:00
|
|
|
<suppress checks="ParameterNumber"
|
|
|
|
|
files="StreamThread.java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="ClassDataAbstractionCoupling"
|
2021-05-04 04:53:35 +08:00
|
|
|
files="(KafkaStreams|KStreamImpl|KTableImpl).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="CyclomaticComplexity"
|
2022-04-16 02:28:43 +08:00
|
|
|
files="(KafkaStreams|StreamsPartitionAssignor|StreamThread|TaskManager|PartitionGroup|SubscriptionWrapperSerde|AssignorConfiguration).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
2020-05-28 04:55:29 +08:00
|
|
|
<suppress checks="StaticVariableName"
|
|
|
|
|
files="StreamsMetricsImpl.java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="NPathComplexity"
|
2022-03-31 07:24:01 +08:00
|
|
|
files="(GlobalStateManagerImpl|KafkaStreams|KStreamImplJoin|StreamsPartitionAssignor|StreamThread|TaskManager|TopologyConfig).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
2019-09-20 07:38:18 +08:00
|
|
|
<suppress checks="(FinalLocalVariable|UnnecessaryParentheses|BooleanExpressionComplexity|CyclomaticComplexity|WhitespaceAfter|LocalVariableName)"
|
|
|
|
|
files="Murmur3.java"/>
|
|
|
|
|
|
2020-09-10 03:02:19 +08:00
|
|
|
<suppress checks="(NPathComplexity|CyclomaticComplexity)"
|
|
|
|
|
files="KStreamSlidingWindowAggregate.java"/>
|
|
|
|
|
|
2018-08-04 04:19:46 +08:00
|
|
|
<!-- suppress FinalLocalVariable outside of the streams package. -->
|
|
|
|
|
<suppress checks="FinalLocalVariable"
|
|
|
|
|
files="^(?!.*[\\/]org[\\/]apache[\\/]kafka[\\/]streams[\\/].*$)"/>
|
|
|
|
|
|
2020-08-06 04:52:50 +08:00
|
|
|
<!-- Generated code -->
|
2020-08-27 06:10:09 +08:00
|
|
|
<suppress checks="(NPathComplexity|ClassFanOutComplexity|CyclomaticComplexity|ClassDataAbstractionCoupling|FinalLocalVariable|LocalVariableName|MemberName|ParameterName|MethodLength|JavaNCSS|AvoidStarImport)"
|
KAFKA-8868: Generate SubscriptionInfo protocol message (#7248)
Rather than maintain hand coded protocol serialization code, Streams could use the same code-generation framework as Clients/Core.
There isn't a perfect match, since the code generation framework includes an assumption that you're generating "protocol messages", rather than just arbitrary blobs, but I think it's close enough to justify using it, and improving it over time.
Using the code generation allows us to drop a lot of detail-oriented, brittle, and hard-to-maintain serialization logic in favor of a schema spec.
Reviewers: Colin P. McCabe <cmccabe@apache.org>, Boyang Chen <boyang@confluent.io>, Bill Bejeck <bill@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
2019-11-02 01:03:55 +08:00
|
|
|
files="streams[\\/]src[\\/](generated|generated-test)[\\/].+.java$"/>
|
2020-09-23 02:32:44 +08:00
|
|
|
<suppress checks="(NPathComplexity|ClassFanOutComplexity|CyclomaticComplexity|ClassDataAbstractionCoupling|FinalLocalVariable|LocalVariableName|MemberName|ParameterName|MethodLength|JavaNCSS|AvoidStarImport)"
|
|
|
|
|
files="raft[\\/]src[\\/](generated|generated-test)[\\/].+.java$"/>
|
2021-05-05 22:48:52 +08:00
|
|
|
<suppress checks="(NPathComplexity|ClassFanOutComplexity|CyclomaticComplexity|ClassDataAbstractionCoupling|FinalLocalVariable|LocalVariableName|MemberName|ParameterName|MethodLength|JavaNCSS|AvoidStarImport)"
|
|
|
|
|
files="storage[\\/]src[\\/](generated|generated-test)[\\/].+.java$"/>
|
KAFKA-8868: Generate SubscriptionInfo protocol message (#7248)
Rather than maintain hand coded protocol serialization code, Streams could use the same code-generation framework as Clients/Core.
There isn't a perfect match, since the code generation framework includes an assumption that you're generating "protocol messages", rather than just arbitrary blobs, but I think it's close enough to justify using it, and improving it over time.
Using the code generation allows us to drop a lot of detail-oriented, brittle, and hard-to-maintain serialization logic in favor of a schema spec.
Reviewers: Colin P. McCabe <cmccabe@apache.org>, Boyang Chen <boyang@confluent.io>, Bill Bejeck <bill@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
2019-11-02 01:03:55 +08:00
|
|
|
|
2020-07-31 01:29:39 +08:00
|
|
|
<suppress checks="ImportControl" files="FetchResponseData.java"/>
|
2020-08-14 00:52:23 +08:00
|
|
|
<suppress checks="ImportControl" files="RecordsSerdeTest.java"/>
|
2020-07-31 01:29:39 +08:00
|
|
|
|
2018-01-30 09:21:48 +08:00
|
|
|
<!-- Streams tests -->
|
2017-03-01 06:55:46 +08:00
|
|
|
<suppress checks="ClassFanOutComplexity"
|
2021-06-08 06:38:12 +08:00
|
|
|
files="(StreamsPartitionAssignorTest|StreamThreadTest|StreamTaskTest|TaskManagerTest|TopologyTestDriverTest).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="MethodLength"
|
2021-04-29 04:22:15 +08:00
|
|
|
files="(EosIntegrationTest|EosV2UpgradeIntegrationTest|KStreamKStreamJoinTest|RocksDBWindowStoreTest).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="ClassDataAbstractionCoupling"
|
2017-06-28 08:47:00 +08:00
|
|
|
files=".*[/\\]streams[/\\].*test[/\\].*.java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="CyclomaticComplexity"
|
2021-04-29 04:22:15 +08:00
|
|
|
files="(EosV2UpgradeIntegrationTest|KStreamKStreamJoinTest|KTableKTableForeignKeyJoinIntegrationTest|RocksDBGenericOptionsToDbOptionsColumnFamilyOptionsAdapterTest|RelationalSmokeTest|MockProcessorContextStateStoreTest).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="JavaNCSS"
|
2021-04-29 04:22:15 +08:00
|
|
|
files="(EosV2UpgradeIntegrationTest|KStreamKStreamJoinTest|TaskManagerTest).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
|
|
|
|
<suppress checks="NPathComplexity"
|
2021-04-29 04:22:15 +08:00
|
|
|
files="(EosV2UpgradeIntegrationTest|EosTestDriver|KStreamKStreamJoinTest|KTableKTableForeignKeyJoinIntegrationTest|RelationalSmokeTest|MockProcessorContextStateStoreTest).java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
2019-09-20 07:38:18 +08:00
|
|
|
<suppress checks="(FinalLocalVariable|WhitespaceAround|LocalVariableName|ImportControl|AvoidStarImport)"
|
|
|
|
|
files="Murmur3Test.java"/>
|
|
|
|
|
|
2020-09-12 05:38:17 +08:00
|
|
|
<suppress checks="MethodLength"
|
2021-04-29 08:57:28 +08:00
|
|
|
files="(KStreamSlidingWindowAggregateTest|KStreamKStreamLeftJoinTest|KStreamKStreamOuterJoinTest).java"/>
|
2020-09-12 05:38:17 +08:00
|
|
|
|
2020-10-01 11:07:23 +08:00
|
|
|
<suppress checks="ClassFanOutComplexity"
|
|
|
|
|
files="StreamTaskTest.java"/>
|
|
|
|
|
|
2020-08-06 04:52:50 +08:00
|
|
|
<!-- Streams test-utils -->
|
2018-01-30 09:21:48 +08:00
|
|
|
<suppress checks="ClassFanOutComplexity"
|
|
|
|
|
files="TopologyTestDriver.java"/>
|
|
|
|
|
<suppress checks="ClassDataAbstractionCoupling"
|
|
|
|
|
files="TopologyTestDriver.java"/>
|
|
|
|
|
|
2020-08-06 04:52:50 +08:00
|
|
|
<!-- Streams examples -->
|
|
|
|
|
<suppress id="dontUseSystemExit"
|
|
|
|
|
files="PageViewTypedDemo.java|PipeDemo.java|TemperatureDemo.java|WordCountDemo.java|WordCountProcessorDemo.java|WordCountTransformerDemo.java"/>
|
|
|
|
|
|
2017-03-01 06:55:46 +08:00
|
|
|
<!-- Tools -->
|
|
|
|
|
<suppress checks="ClassDataAbstractionCoupling"
|
|
|
|
|
files="VerifiableConsumer.java"/>
|
|
|
|
|
<suppress checks="CyclomaticComplexity"
|
2017-08-26 03:29:40 +08:00
|
|
|
files="(StreamsResetter|ProducerPerformance|Agent).java"/>
|
2019-10-04 06:59:31 +08:00
|
|
|
<suppress checks="BooleanExpressionComplexity"
|
|
|
|
|
files="StreamsResetter.java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
<suppress checks="NPathComplexity"
|
2020-02-13 04:34:12 +08:00
|
|
|
files="(ProducerPerformance|StreamsResetter|Agent|TransactionalMessageCopier).java"/>
|
2017-08-26 03:29:40 +08:00
|
|
|
<suppress checks="ImportControl"
|
|
|
|
|
files="SignalLogger.java"/>
|
|
|
|
|
<suppress checks="IllegalImport"
|
|
|
|
|
files="SignalLogger.java"/>
|
2018-03-20 21:51:45 +08:00
|
|
|
<suppress checks="ParameterNumber"
|
|
|
|
|
files="ProduceBenchSpec.java"/>
|
2020-12-19 05:03:59 +08:00
|
|
|
<suppress checks="ParameterNumber"
|
|
|
|
|
files="ConsumeBenchSpec.java"/>
|
2019-09-09 10:49:13 +08:00
|
|
|
<suppress checks="ParameterNumber"
|
|
|
|
|
files="SustainedConnectionSpec.java"/>
|
2020-08-06 04:52:50 +08:00
|
|
|
<suppress id="dontUseSystemExit"
|
|
|
|
|
files="VerifiableConsumer.java"/>
|
|
|
|
|
<suppress id="dontUseSystemExit"
|
|
|
|
|
files="VerifiableProducer.java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
2021-02-10 06:11:35 +08:00
|
|
|
<!-- Shell -->
|
|
|
|
|
<suppress checks="CyclomaticComplexity"
|
2021-10-27 04:23:20 +08:00
|
|
|
files="(GlobComponent|MetadataNodeManager).java"/>
|
2021-02-10 06:11:35 +08:00
|
|
|
|
2017-03-01 06:55:46 +08:00
|
|
|
<!-- Log4J-Appender -->
|
|
|
|
|
<suppress checks="CyclomaticComplexity"
|
|
|
|
|
files="KafkaLog4jAppender.java"/>
|
|
|
|
|
|
|
|
|
|
<suppress checks="NPathComplexity"
|
|
|
|
|
files="KafkaLog4jAppender.java"/>
|
2017-04-07 05:51:09 +08:00
|
|
|
<suppress checks="JavaNCSS"
|
|
|
|
|
files="RequestResponseTest.java"/>
|
2017-03-01 06:55:46 +08:00
|
|
|
|
2021-01-15 01:58:52 +08:00
|
|
|
<!-- metadata -->
|
KAFKA-12276: Add the quorum controller code (#10070)
The quorum controller stores metadata in the KIP-500 metadata log, not in Apache
ZooKeeper. Each controller node is a voter in the metadata quorum. The leader of the
quorum is the active controller, which processes write requests. The followers are standby
controllers, which replay the operations written to the log. If the active controller goes away,
a standby controller can take its place.
Like the ZooKeeper-based controller, the quorum controller is based on an event queue
backed by a single-threaded executor. However, unlike the ZK-based controller, the quorum
controller can have multiple operations in flight-- it does not need to wait for one operation
to be finished before starting another. Therefore, calls into the QuorumController return
CompleteableFuture objects which are completed with either a result or an error when the
operation is done. The QuorumController will also time out operations that have been
sitting on the queue too long without being processed. In this case, the future is completed
with a TimeoutException.
The controller uses timeline data structures to store multiple "versions" of its in-memory
state simultaneously. "Read operations" read only committed state, which is slightly older
than the most up-to-date in-memory state. "Write operations" read and write the latest
in-memory state. However, we can not return a successful result for a write operation until
its state has been committed to the log. Therefore, if a client receives an RPC response, it
knows that the requested operation has been performed, and can not be undone by a
controller failover.
Reviewers: Jun Rao <junrao@gmail.com>, Ron Dagostino <rdagostino@confluent.io>
2021-02-20 10:03:23 +08:00
|
|
|
<suppress checks="ClassDataAbstractionCoupling"
|
2021-12-11 04:29:58 +08:00
|
|
|
files="(QuorumController|QuorumControllerTest|ReplicationControlManager|ReplicationControlManagerTest).java"/>
|
KAFKA-12276: Add the quorum controller code (#10070)
The quorum controller stores metadata in the KIP-500 metadata log, not in Apache
ZooKeeper. Each controller node is a voter in the metadata quorum. The leader of the
quorum is the active controller, which processes write requests. The followers are standby
controllers, which replay the operations written to the log. If the active controller goes away,
a standby controller can take its place.
Like the ZooKeeper-based controller, the quorum controller is based on an event queue
backed by a single-threaded executor. However, unlike the ZK-based controller, the quorum
controller can have multiple operations in flight-- it does not need to wait for one operation
to be finished before starting another. Therefore, calls into the QuorumController return
CompleteableFuture objects which are completed with either a result or an error when the
operation is done. The QuorumController will also time out operations that have been
sitting on the queue too long without being processed. In this case, the future is completed
with a TimeoutException.
The controller uses timeline data structures to store multiple "versions" of its in-memory
state simultaneously. "Read operations" read only committed state, which is slightly older
than the most up-to-date in-memory state. "Write operations" read and write the latest
in-memory state. However, we can not return a successful result for a write operation until
its state has been committed to the log. Therefore, if a client receives an RPC response, it
knows that the requested operation has been performed, and can not be undone by a
controller failover.
Reviewers: Jun Rao <junrao@gmail.com>, Ron Dagostino <rdagostino@confluent.io>
2021-02-20 10:03:23 +08:00
|
|
|
<suppress checks="ClassFanOutComplexity"
|
2022-05-19 03:08:36 +08:00
|
|
|
files="(QuorumController|QuorumControllerTest|ReplicationControlManager|ReplicationControlManagerTest).java"/>
|
2022-04-15 01:04:32 +08:00
|
|
|
<suppress checks="(ParameterNumber|ClassDataAbstractionCoupling)"
|
KAFKA-12467: Implement QuorumController snapshot generation (#10366)
Implement controller-side snapshot generation.Implement QuorumController snapshot
generation. Note that this PR does not handle KRaft integration, just the internal
snapshot record generation and consumption logic.
Reading a snapshot is relatively straightforward. When the QuorumController
starts up, it loads the most recent snapshot. This is just a series of records
that we replay, plus a log offset ("snapshot epoch") that we advance to.
Writing a snapshot is more complex. There are several components:
the SnapshotWriter which persists the snapshot, the SnapshotGenerator
which manages writing each batch of records, and the SnapshotGeneratorManager
which interfaces the preceding two classes with the event queue.
Controller snapshots are done incrementally. In order to avoid blocking the
controller thread for a long time, we pull a few record batches at a time from
our record batch iterators. These iterators are implemented by controller
manager classes such as ReplicationControlManager, ClusterControlManager, etc.
Finally, this PR adds ControllerTestUtils#deepSortRecords and
ControllerTestUtils#assertBatchIteratorContains, which make it easier to write
unit tests. Since records are often constructed from unsorted data structures,
it is often useful to sort them before comparing them.
Reviewers: David Arthur <mumrah@gmail.com>
2021-04-07 01:18:06 +08:00
|
|
|
files="(QuorumController).java"/>
|
KAFKA-12276: Add the quorum controller code (#10070)
The quorum controller stores metadata in the KIP-500 metadata log, not in Apache
ZooKeeper. Each controller node is a voter in the metadata quorum. The leader of the
quorum is the active controller, which processes write requests. The followers are standby
controllers, which replay the operations written to the log. If the active controller goes away,
a standby controller can take its place.
Like the ZooKeeper-based controller, the quorum controller is based on an event queue
backed by a single-threaded executor. However, unlike the ZK-based controller, the quorum
controller can have multiple operations in flight-- it does not need to wait for one operation
to be finished before starting another. Therefore, calls into the QuorumController return
CompleteableFuture objects which are completed with either a result or an error when the
operation is done. The QuorumController will also time out operations that have been
sitting on the queue too long without being processed. In this case, the future is completed
with a TimeoutException.
The controller uses timeline data structures to store multiple "versions" of its in-memory
state simultaneously. "Read operations" read only committed state, which is slightly older
than the most up-to-date in-memory state. "Write operations" read and write the latest
in-memory state. However, we can not return a successful result for a write operation until
its state has been committed to the log. Therefore, if a client receives an RPC response, it
knows that the requested operation has been performed, and can not be undone by a
controller failover.
Reviewers: Jun Rao <junrao@gmail.com>, Ron Dagostino <rdagostino@confluent.io>
2021-02-20 10:03:23 +08:00
|
|
|
<suppress checks="CyclomaticComplexity"
|
KAFKA-13646; Implement KIP-801: KRaft authorizer (#11649)
Currently, when using KRaft mode, users still have to have an Apache ZooKeeper instance if they want to use AclAuthorizer. We should have a built-in Authorizer for KRaft mode that does not depend on ZooKeeper. This PR introduces such an authorizer, called StandardAuthorizer. See KIP-801 for a full description of the new Authorizer design.
Authorizer.java: add aclCount API as described in KIP-801. StandardAuthorizer is currently the only authorizer that implements it, but eventually we may implement it for AclAuthorizer and others as well.
ControllerApis.scala: fix a bug where createPartitions was authorized using CREATE on the topic resource rather than ALTER on the topic resource as it should have been.
QuorumTestHarness: rename the controller endpoint to CONTROLLER for consistency (the brokers already called it that). This is relevant in AuthorizerIntegrationTest where we are examining endpoint names. Also add the controllerServers call.
TestUtils.scala: adapt the ACL functions to be usable from KRaft, by ensuring that they use the Authorizer from the current active controller.
BrokerMetadataPublisher.scala: add broker-side ACL application logic.
Controller.java: add ACL APIs. Also add a findAllTopicIds API in order to make junit tests that use KafkaServerTestHarness#getTopicNames and KafkaServerTestHarness#getTopicIds work smoothly.
AuthorizerIntegrationTest.scala: convert over testAuthorizationWithTopicExisting (more to come soon)
QuorumController.java: add logic for replaying ACL-based records. This means storing them in the new AclControlManager object, and integrating them into controller snapshots. It also means applying the changes in the Authorizer, if one is configured. In renounce, when reverting to a snapshot, also set newBytesSinceLastSnapshot to 0.
Reviewers: YeonCheol Jang <YeonCheolGit@users.noreply.github.com>, Jason Gustafson <jason@confluent.io>
2022-02-10 02:38:52 +08:00
|
|
|
files="(ClientQuotasImage|MetadataDelta|QuorumController|ReplicationControlManager).java"/>
|
2021-02-05 06:46:57 +08:00
|
|
|
<suppress checks="NPathComplexity"
|
2022-04-15 01:04:32 +08:00
|
|
|
files="(ClientQuotasImage|KafkaEventQueue|ReplicationControlManager|FeatureControlManager).java"/>
|
2021-01-15 01:58:52 +08:00
|
|
|
<suppress checks="(NPathComplexity|ClassFanOutComplexity|CyclomaticComplexity|ClassDataAbstractionCoupling|LocalVariableName|MemberName|ParameterName|MethodLength|JavaNCSS|AvoidStarImport)"
|
|
|
|
|
files="metadata[\\/]src[\\/](generated|generated-test)[\\/].+.java$"/>
|
KAFKA-13646; Implement KIP-801: KRaft authorizer (#11649)
Currently, when using KRaft mode, users still have to have an Apache ZooKeeper instance if they want to use AclAuthorizer. We should have a built-in Authorizer for KRaft mode that does not depend on ZooKeeper. This PR introduces such an authorizer, called StandardAuthorizer. See KIP-801 for a full description of the new Authorizer design.
Authorizer.java: add aclCount API as described in KIP-801. StandardAuthorizer is currently the only authorizer that implements it, but eventually we may implement it for AclAuthorizer and others as well.
ControllerApis.scala: fix a bug where createPartitions was authorized using CREATE on the topic resource rather than ALTER on the topic resource as it should have been.
QuorumTestHarness: rename the controller endpoint to CONTROLLER for consistency (the brokers already called it that). This is relevant in AuthorizerIntegrationTest where we are examining endpoint names. Also add the controllerServers call.
TestUtils.scala: adapt the ACL functions to be usable from KRaft, by ensuring that they use the Authorizer from the current active controller.
BrokerMetadataPublisher.scala: add broker-side ACL application logic.
Controller.java: add ACL APIs. Also add a findAllTopicIds API in order to make junit tests that use KafkaServerTestHarness#getTopicNames and KafkaServerTestHarness#getTopicIds work smoothly.
AuthorizerIntegrationTest.scala: convert over testAuthorizationWithTopicExisting (more to come soon)
QuorumController.java: add logic for replaying ACL-based records. This means storing them in the new AclControlManager object, and integrating them into controller snapshots. It also means applying the changes in the Authorizer, if one is configured. In renounce, when reverting to a snapshot, also set newBytesSinceLastSnapshot to 0.
Reviewers: YeonCheol Jang <YeonCheolGit@users.noreply.github.com>, Jason Gustafson <jason@confluent.io>
2022-02-10 02:38:52 +08:00
|
|
|
<suppress checks="BooleanExpressionComplexity"
|
|
|
|
|
files="(MetadataImage).java"/>
|
2022-05-03 07:27:52 +08:00
|
|
|
<suppress checks="ImportControl"
|
|
|
|
|
files="ApiVersionsResponse.java"/>
|
2022-05-19 03:08:36 +08:00
|
|
|
<suppress checks="AvoidStarImport"
|
|
|
|
|
files="MetadataVersionTest.java"/>
|
2021-06-19 00:38:42 +08:00
|
|
|
|
|
|
|
|
<!-- Storage -->
|
|
|
|
|
<suppress checks="(CyclomaticComplexity|ParameterNumber)"
|
|
|
|
|
files="(RemoteLogManagerConfig).java"/>
|
|
|
|
|
|
2021-07-08 07:02:37 +08:00
|
|
|
<!-- benchmarks -->
|
2022-06-04 00:12:06 +08:00
|
|
|
<suppress checks="(ClassDataAbstractionCoupling|ClassFanOutComplexity)"
|
2021-07-08 07:02:37 +08:00
|
|
|
files="(ReplicaFetcherThreadBenchmark).java"/>
|
|
|
|
|
|
2017-03-01 06:55:46 +08:00
|
|
|
</suppressions>
|
