root
/
kafka
mirror of https://github.com/apache/kafka.git
1
0
Fork 0
Code Issues Actions 4 Packages Projects Releases Wiki Activity

MINOR: Move upgraded docs from site to kafak docs (#9532) 

Reviewers: Matthias J. Sax <mjsax@apache.org>
This commit is contained in:
Bill Bejeck 2020-10-31 07:52:14 -04:00 committed by GitHub
parent a8247023a7
commit d39382d3e2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 151 additions and 29 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

43
docs/documentation.html
View File

@ -22,51 +22,56 @@
<div class="content documentation documentation--current">
<!--#include virtual="../includes/_nav.htm" -->
<div class="right">
<!--#include virtual="../includes/_docs_banner.htm" -->
<!--#include virtual="../includes/_nav.htm" -->
<div class="toc-handle-container">
<div class="toc-handle">&lt;</div>
</div>
<div class="docs-nav">
<!--#include virtual="toc.html" -->
</div>
<div class="right">
<!--//#include virtual="../includes/_docs_banner.htm" -->
<h1>Documentation</h1>
<h3>Kafka 2.8 Documentation</h3>
Prior releases: <a href="/07/documentation.html">0.7.x</a>, <a href="/08/documentation.html">0.8.0</a>, <a href="/081/documentation.html">0.8.1.X</a>, <a href="/082/documentation.html">0.8.2.X</a>, <a href="/090/documentation.html">0.9.0.X</a>, <a href="/0100/documentation.html">0.10.0.X</a>, <a href="/0101/documentation.html">0.10.1.X</a>, <a href="/0102/documentation.html">0.10.2.X</a>, <a href="/0110/documentation.html">0.11.0.X</a>, <a href="/10/documentation.html">1.0.X</a>, <a href="/11/documentation.html">1.1.X</a>, <a href="/20/documentation.html">2.0.X</a>, <a href="/21/documentation.html">2.1.X</a>, <a href="/22/documentation.html">2.2.X</a>, <a href="/23/documentation.html">2.3.X</a>, <a href="/24/documentation.html">2.4.X</a>, <a href="/25/documentation.html">2.5.X</a>, <a href="/26/documentation.html">2.6.X</a>, <a href="/27/documentation.html">2.7.X</a>.
<!--#include virtual="toc.html" -->
<h2><a id="gettingStarted" href="#gettingStarted">1. Getting Started</a></h2>
<h3><a id="introduction" href="#introduction">1.1 Introduction</a></h3>
<h2 class="anchor-heading"><a id="gettingStarted" class="anchor-link"></a><a href="#gettingStarted">1. Getting Started</a></h2>
<h3 class="anchor-heading"><a id="introduction" class="anchor-link"></a><a href="#introduction">1.1 Introduction</a></h3>
<!--#include virtual="introduction.html" -->
<h3><a id="uses" href="#uses">1.2 Use Cases</a></h3>
<h3 class="anchor-heading"><a id="uses" class="anchor-link"></a><a href="#uses">1.2 Use Cases</a></h3>
<!--#include virtual="uses.html" -->
<h3><a id="quickstart" href="#quickstart">1.3 Quick Start</a></h3>
<!--#include virtual="quickstart.html" -->
<h3><a id="ecosystem" href="#ecosystem">1.4 Ecosystem</a></h3>
<h3 class="anchor-heading"><a id="quickstart" class="anchor-link"></a><a href="#quickstart">1.3 Quick Start</a></h3>
<!--#include virtual="quickstart-zookeeper.html" -->
<h3 class="anchor-heading"><a id="ecosystem" class="anchor-link"></a><a href="#ecosystem">1.4 Ecosystem</a></h3>
<!--#include virtual="ecosystem.html" -->
<h3><a id="upgrade" href="#upgrade">1.5 Upgrading From Previous Versions</a></h3>
<h3 class="anchor-heading"><a id="upgrade" class="anchor-link"></a><a href="#upgrade">1.5 Upgrading From Previous Versions</a></h3>
<!--#include virtual="upgrade.html" -->
<h2><a id="api" href="#api">2. APIs</a></h2>
<h2 class="anchor-heading"><a id="api" class="anchor-link"></a><a href="#api">2. APIs</a></h2>
<!--#include virtual="api.html" -->
<h2><a id="configuration" href="#configuration">3. Configuration</a></h2>
<h2 class="anchor-heading"><a id="configuration" class="anchor-link"></a><a href="#configuration">3. Configuration</a></h2>
<!--#include virtual="configuration.html" -->
<h2><a id="design" href="#design">4. Design</a></h2>
<h2 class="anchor-heading"><a id="design" class="anchor-link"></a><a href="#design">4. Design</a></h2>
<!--#include virtual="design.html" -->
<h2><a id="implementation" href="#implementation">5. Implementation</a></h2>
<h2 class="anchor-heading"><a id="implementation" class="anchor-link"></a><a href="#implementation">5. Implementation</a></h2>
<!--#include virtual="implementation.html" -->
<h2><a id="operations" href="#operations">6. Operations</a></h2>
<h2 class="anchor-heading"><a id="operations" class="anchor-link"></a><a href="#operations">6. Operations</a></h2>
<!--#include virtual="ops.html" -->
<h2><a id="security" href="#security">7. Security</a></h2>
<h2 class="anchor-heading"><a id="security" class="anchor-link"></a><a href="#security">7. Security</a></h2>
<!--#include virtual="security.html" -->
<h2><a id="connect" href="#connect">8. Kafka Connect</a></h2>
<h2 class="anchor-heading"><a id="connect" class="anchor-link"></a><a href="#connect">8. Kafka Connect</a></h2>
<!--#include virtual="connect.html" -->
<h2><a id="streams" href="/documentation/streams">9. Kafka Streams</a></h2>

137
docs/upgrade.html
View File

@ -21,6 +21,48 @@
<h5><a id="upgrade_280_notable" href="#upgrade_280_notable">Notable changes in 2.8.0</a></h5>
<h4><a id="upgrade_2_7_0" href="#upgrade_2_7_0">Upgrading to 2.7.0 from any version 0.8.x through 2.6.x</a></h4>
<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to store consumer offsets.
Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>
<p><b>For a rolling upgrade:</b></p>
<ol>
<li> Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you
are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously
overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior
to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.
<ul>
<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.6</code>, <code>2.5</code>, etc.)</li>
<li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact
following the upgrade</a> for the details on what this configuration does.)</li>
</ul>
If you are upgrading from version 0.11.0.x or above, and you have not overridden the message format, then you only need to override
the inter-broker protocol version.
<ul>
<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.6</code>, <code>2.5</code>, etc.)</li>
</ul>
</li>
<li> Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the
brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.
It is still possible to downgrade at this point if there are any problems.
</li>
<li> Once the cluster's behavior and performance has been verified, bump the protocol version by editing
<code>inter.broker.protocol.version</code> and setting it to <code>2.7</code>.
</li>
<li> Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest
protocol version, it will no longer be possible to downgrade the cluster to an older version.
</li>
<li> If you have overridden the message format version as instructed above, then you need to do one more rolling restart to
upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,
change log.message.format.version to 2.7 on each broker and restart them one by one. Note that the older Scala clients,
which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs
(or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),
the newer Java clients must be used.
</li>
</ol>
<h5><a id="upgrade_270_notable" href="#upgrade_270_notable">Notable changes in 2.7.0</a></h5>
<ul>
<li>
@ -93,8 +135,49 @@
<a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-616%3A+Rename+implicit+Serdes+instances+in+kafka-streams-scala">KIP-616</a>
</li>
</ul>
<h4><a id="upgrade_2_6_0" href="#upgrade_2_6_0">Upgrading to 2.6.0 from any version 0.8.x through 2.5.x</a></h4>
<h5><a id="upgrade_260_notable" href="#upgrade_260_notable">Notable changes in 2.6.0</a></h5>
<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to store consumer offsets.
Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>
<p><b>For a rolling upgrade:</b></p>
<ol>
<li> Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you
are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously
overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior
to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.
<ul>
<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.5</code>, <code>2.4</code>, etc.)</li>
<li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact
following the upgrade</a> for the details on what this configuration does.)</li>
</ul>
If you are upgrading from version 0.11.0.x or above, and you have not overridden the message format, then you only need to override
the inter-broker protocol version.
<ul>
<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.5</code>, <code>2.4</code>, etc.)</li>
</ul>
</li>
<li> Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the
brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.
It is still possible to downgrade at this point if there are any problems.
</li>
<li> Once the cluster's behavior and performance has been verified, bump the protocol version by editing
<code>inter.broker.protocol.version</code> and setting it to <code>2.6</code>.
</li>
<li> Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest
protocol version, it will no longer be possible to downgrade the cluster to an older version.
</li>
<li> If you have overridden the message format version as instructed above, then you need to do one more rolling restart to
upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,
change log.message.format.version to 2.6 on each broker and restart them one by one. Note that the older Scala clients,
which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs
(or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),
the newer Java clients must be used.
</li>
</ol>
<h5 class="anchor-heading"><a id="upgrade_260_notable" class="anchor-link"></a><a href="#upgrade_260_notable">Notable changes in 2.6.0</a></h5>
<ul>
<li>Kafka Streams adds a new processing mode (requires broker 2.5 or newer) that improves application
scalability using exactly-once guarantees
@ -114,17 +197,51 @@
Fetch requests and other requests intended only for the leader or follower return NOT_LEADER_OR_FOLLOWER(6) instead of REPLICA_NOT_AVAILABLE(9)
if the broker is not a replica, ensuring that this transient error during reassignments is handled by all clients as a retriable exception.
</li>
<li>There are several notable changes to the reassignment tool <code>kafka-reassign-partitions.sh</code>
following the completion of
<a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-455%3A+Create+an+Administrative+API+for+Replica+Reassignment">KIP-455</a>.
This tool now requires the <code>--additional</code> flag to be provided when changing the throttle of an
active reassignment. Reassignment cancellation is now possible using the
<code>--cancel</code> command. Finally, reassignment with <code>--zookeeper</code>
has been deprecated in favor of <code>--bootstrap-server</code>. See the KIP for more detail.
</li>
</ul>
<h5><a id="upgrade_250_notable" href="#upgrade_250_notable">Notable changes in 2.5.0</a></h5>
<h4><a id="upgrade_2_5_0" href="#upgrade_2_5_0">Upgrading to 2.5.0 from any version 0.8.x through 2.4.x</a></h4>
<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to store consumer offsets.
Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>
<p><b>For a rolling upgrade:</b></p>
<ol>
<li> Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you
are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously
overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior
to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.
<ul>
<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.4</code>, <code>2.3</code>, etc.)</li>
<li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact
following the upgrade</a> for the details on what this configuration does.)</li>
</ul>
If you are upgrading from version 0.11.0.x or above, and you have not overridden the message format, then you only need to override
the inter-broker protocol version.
<ul>
<li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.4</code>, <code>2.3</code>, etc.)</li>
</ul>
</li>
<li> Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the
brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.
It is still possible to downgrade at this point if there are any problems.
</li>
<li> Once the cluster's behavior and performance has been verified, bump the protocol version by editing
<code>inter.broker.protocol.version</code> and setting it to <code>2.5</code>.
</li>
<li> Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest
protocol version, it will no longer be possible to downgrade the cluster to an older version.
</li>
<li> If you have overridden the message format version as instructed above, then you need to do one more rolling restart to
upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,
change log.message.format.version to 2.5 on each broker and restart them one by one. Note that the older Scala clients,
which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs
(or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),
the newer Java clients must be used.
</li>
</ol>
<h5 class="anchor-heading"><a id="upgrade_250_notable" class="anchor-link"></a><a href="#upgrade_250_notable">Notable changes in 2.5.0</a></h5>
<ul>
<li>When <code>RebalanceProtocol#COOPERATIVE</code> is used, <code>Consumer#poll</code> can still return data
while it is in the middle of a rebalance for those partitions still owned by the consumer; in addition