mirror of https://github.com/apache/kafka.git
KAFKA-9226: Updated documentation section on log deletion policies. (#7738)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
This commit is contained in:
parent
88448f6255
commit
d7513693a3
|
@ -219,7 +219,15 @@
|
|||
</pre>
|
||||
<h4><a id="impl_deletes" href="#impl_deletes">Deletes</a></h4>
|
||||
<p>
|
||||
Data is deleted one log segment at a time. The log manager allows pluggable delete policies to choose which files are eligible for deletion. The current policy deletes any log with a modification time of more than <i>N</i> days ago, though a policy which retained the last <i>N</i> GB could also be useful. To avoid locking reads while still allowing deletes that modify the segment list we use a copy-on-write style segment list implementation that provides consistent views to allow a binary search to proceed on an immutable static snapshot view of the log segments while deletes are progressing.
|
||||
Data is deleted one log segment at a time. The log manager applies two metrics to identify segments which are
|
||||
eligible for deletion: time and size. For time-based policies, the record timestamps are considered, with the
|
||||
largest timestamp in a segment file (order of records is not relevant) defining the retention time for the
|
||||
entire segment. Size-based retention is disabled by default. When enabled the log manager keeps deleting the
|
||||
oldest segment file until the overall size of the partition is within the configured limit again. If both
|
||||
policies are enabled at the same time, a segment that is eligible for deletion due to either policy will be
|
||||
deleted. To avoid locking reads while still allowing deletes that modify the segment list we use a copy-on-write
|
||||
style segment list implementation that provides consistent views to allow a binary search to proceed on an
|
||||
immutable static snapshot view of the log segments while deletes are progressing.
|
||||
</p>
|
||||
<h4><a id="impl_guarantees" href="#impl_guarantees">Guarantees</a></h4>
|
||||
<p>
|
||||
|
|
Loading…
Reference in New Issue