This PR addresses minor grammar and clarity issues in upgrade.html doc.
Reviewers: mingdaoy <mingdaoy@gmail.com>, Colin P. McCabe <cmccabe@apache.org>, TengYao Chi <kitingiao@gmail.com>, Ken Huang <s7133700@gmail.com>, Jhen-Yung Hsu <jhenyunghsu@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
Running `bin/kafka-features.sh upgrade --release-version 4.0` results in
the following error. This PR fixes the issue by adding the required
argument.
`kafka-features: error: one of the arguments --bootstrap-server
--bootstrap-controller is required.`
Reviewers: Colin P. McCabe <cmccabe@apache.org>
Add information about setting dynamic log levels and dynamic
configurations on KRaft controllers.
Reviewers: Ismael Juma <ismael@juma.me.uk>, Chia-Ping Tsai <chia7712@gmail.com>
In the documentation today, we have the following sentence:
By default, if no ResourcePatterns match a specific Resource R, then R
has no associated ACLs, and therefore no one other than super users is
allowed to access R. If you want to change that behavior, you can
include the following in server.properties.
Reviewers: TengYao Chi <kitingiao@gmail.com>, Andrew Schofield <aschofield@confluent.io>
Although, it is correct, I have observed users being confused by it. I
think could me made clearer that default is deny and this property is a
way to change default.
Change
Replace the above with the following:
Default Behavior Without ACLs:
If a resource (R) does not have any ACLs defined—that is, if no ACL
matches the resource—Kafka will restrict access to that resource. In
this situation, only super users are allowed to access it.
Changing the Default Behavior:
If you prefer that resources without any ACLs be accessible by all users
(instead of just super users), you can change the default behavior. To
do this, add the following line to your server.properties file:
allow.everyone.if.no.acl.found=true
With this setting enabled, if a resource does not have any ACLs defined,
Kafka will allow access to everyone. If a resource has one or more ACLs
defined, those ACL rules will be enforced as usual, regardless of the
setting.
Using `bin/kafka-features.sh upgrade --metadata 4.0` to finalize the
upgrade is deprecated and furthermore it does not bump all the other
feature flags. Hence we should recommend to use `bin/kafka-features.sh
upgrade --release-version 4.0` in the documentation. This change was
introduced in [KIP-1022: Formatting and Updating
Features](https://cwiki.apache.org/confluence/display/KAFKA/KIP-1022%3A+Formatting+and+Updating+Features).
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Ismael Juma <ismael@juma.me.uk>
In KRaft mode, custom KafkaPrincipalBuilder instances must implement KafkaPrincipalSerde. This PR updates all related documentation to highlight this requirement.
Reviewers: Ken Huang <s7133700@gmail.com>, David Jacot <djacot@confluent.io>, TengYao Chi <kitingiao@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
1. Updated JavaDoc to reflect that CreateTopicPolicy and AlterConfigPolicy run on the controller in KRaft mode.
2. Modified Behavioral Change Reference in the HTML docs to include this change.
3. add warning message to KafkaConfig if the config of broker node has policy configs
Reviewers: TengYao Chi <kitingiao@gmail.com>, Ken Huang <s7133700@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
This PR addresses minor grammar and clarity issues in zk2kraft.html doc.
Reviewers: TengYao Chi <kitingiao@gmail.com>, Colin P. McCabe <cmccabe@apache.org>
Add client upgrade documentation, for more information, please refer to https://cwiki.apache.org/confluence/x/y4kgF
Reviewers: TengYao Chi <kitingiao@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
1. add prefix "config" to the commands' properties
2. add missed sections (6.11 and 6.12)
3. fix some incorrect commands
Reviewers: David Jacot <djacot@confluent.io>, Ken Huang <s7133700@gmail.com>, TengYao Chi <kitingiao@gmail.com>, Jun Rao <junrao@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
JIRA: KAFKA-18908
`ZooKeeper` mode allows large configuration values to be created through
an append operation. `KRaft` mode restricts configuration values to a
maximum size of `Short.MAX_VALUE`.
We should document this behavioral change.
Reviewers: Ken Huang <s7133700@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
This patch filters out the topic describe unauthorized topics from the
ConsumerGroupHeartbeat and ConsumerGroupDescribe response.
In ConsumerGroupHeartbeat,
- if the request has `subscribedTopicNames` set, we directly check the
authz in `KafkaApis` and return a topic auth failure in the response if
any of the topics is denied.
- Otherwise, we check the authz only if a regex refresh is triggered and
we do it based on the acl of the consumer that triggered the refresh. If
any of the topic is denied, we filter it out from the resolved
subscription.
In ConsumerGroupDescribe, we check the authz of the coordinator
response. If any of the topic in the group is denied, we remove the
described info and add a topic auth failure to the described group.
(similar to the group auth failure)
Reviewers: David Jacot <djacot@confluent.io>, Lianet Magrans
<lmagrans@confluent.io>, Rajini Sivaram <rajinisivaram@googlemail.com>,
Chia-Ping Tsai <chia7712@gmail.com>, TaiJuWu <tjwu1217@gmail.com>,
TengYao Chi <kitingiao@gmail.com>
Link to the older version of the documentation for people who care about
the old message format.
Reviewers: Jun Rao <junrao@apache.org>, Chia-Ping Tsai <chia7712@apache.org>
Pull request to implement KIP-1111, aims to add a configuration that
prevents a Kafka Streams application from starting if any of its
internal topics have auto-generated names, thereby enforcing explicit
naming for all internal topics and enhancing the stability of the
application’s topology.
- Repartition Topics:
All repartition topics are created in the
KStreamImpl.createRepartitionedSource(...) static method. This method
either receives a name explicitly provided by the user or null and then
builds the final repartition topic name.
- Changelog Topics and State Store Names:
There are several scenarios where these are created:
In the MaterializedInternal constructor.
During KStream/KStream joins.
During KStream/KTable joins with grace periods.
With key-value buffers are used in suppressions.
Reviewers: Lucas Brutschy <lbrutschy@confluent.io>, Sophie Blee-Goldman <sophie@responsive.dev>
Details:
1. Upgrades to 4.0.x are only supported from 3.3.x and for kraft mode clusters
2. Add rolling upgrade instructions for 4.0.x
3. Clarify the message for zk to kraft migrations
4. Remove all the upgrade instructions for older versions (they can still be found in the upgrade notes for older releases)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, David Jacot <djacot@confluent.io>
3.3.0 was the first KRaft release that was deemed production-ready and also
when KIP-778 (KRaft to KRaft upgrades) landed. Given that, it's reasonable
for 4.x to only support upgrades from 3.3.0 or newer (the metadata version also
needs to be set to "3.3" or newer before upgrading).
Noteworthy changes:
1. `AlterPartition` no longer includes topic names, which makes it possible to
simplify `AlterParitionManager` logic.
2. Metadata versions older than `IBP_3_3_IV3` have been removed and
`IBP_3_3_IV3` is now the minimum version.
3. `MINIMUM_BOOTSTRAP_VERSION` has been removed.
4. Removed `isLeaderRecoverySupported`, `isNoOpsRecordSupported`,
`isKRaftSupported`, `isBrokerRegistrationChangeRecordSupported` and
`isInControlledShutdownStateSupported` - these are always `true` now.
Also removed related conditional code.
5. Removed default metadata version or metadata version fallbacks in
multiple places - we now fail-fast instead of potentially using an incorrect
metadata version.
6. Update `MetadataBatchLoader.resetToImage` to set `hasSeenRecord`
based on whether image is empty - this was a previously existing issue that
became more apparent after the changes in this PR.
7. Remove `ibp` parameter from `BootstrapDirectory`
8. A number of tests were not useful anymore and have been removed.
I will update the upgrade notes via a separate PR as there are a few things that
need changing and it would be easier to do so that way.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Jun Rao <junrao@gmail.com>, David Arthur <mumrah@gmail.com>, Colin P. McCabe <cmccabe@apache.org>, Justine Olshan <jolshan@confluen.io>, Ken Huang <s7133700@gmail.com>
This patch changes the default value of `group.coordinator.threads` to `4` and sets it priority to `HIGH`. This change makes it consistent with how we handle `num.network.threads` and `num.io.threads`. The patch also tweaks the upgrade notes.
Reviewers: Ismael Juma <ismael@juma.me.uk>
KAFKA-16720 aims to add the support for the AlterShareGroupOffsets AdminClient. Key Changes in the PR:
1. Added handing of alterShareGroupOffsets() in KafkaAdminClient and introduce AlterShareGroupOffsetRequest/AlterShareGroupOffsetResponse/AlterShareGroupOffsetsOptions classes.
2. Corresponding test in KafkaAdminClientTest.
3. Added ALTER_SHARE_GROUP_OFFSETS API (will finish it in next PR and the share coordinator pieces)
Reviewers: poorv Mittal <apoorvmittal10@gmail.com>, Andrew Schofield <aschofield@confluent.io>, Chia-Ping Tsai <chia7712@gmail.com>
Most of the changes are obvious clean-ups/fixes. A couple of noteworthy items:
1. Support for non LTS versions is clarified (we were incorrectly stating full support
for Java 23).
2. TLS version negotiation details are clarified.
Reviewers: Matthias J. Sax <matthias@confluent.io>, Chia-Ping Tsai <chia7712@gmail.com>
Mahsa Seifikar
Nick Guo
Ken Huang
TengYao Chi
Colin Patrick McCabe
Iamoshione
David Jacot
mingdaoy
co63oc
S.Y. Wang
Logan Zhu
Kuan-Po Tseng
Xuan-Zhang Gong
Dongnuo Lyu
Ismael Juma
Calvin Liu
Sebastien Viale
Kaushik Raina
Chirag Wadhwa
Andrew Schofield
Luke Chen
Jimmy Wang
Justine Olshan
Lianet Magrans
Jhen-Yung Hsu
PoAn Yang
Albert
Ming-Yen Chung
Almog Gavra
Mehari Beyene
Joao Pedro Fonseca Dantas