mirror of https://github.com/apache/kafka.git
351203873d
In the original algorithm, standby tasks are assigned to a process that previously owned the task only if it is “load-balanced”, meaning the process has fewer tasks that members, or it is the least loaded process. This is strong requirement, and will cause standby tasks to often not get assigned to process that previously owned it. Furthermore, the condition “is the least loaded process” is hard to evaluate efficiently in this context. We propose to instead use the same “below-quota” condition as in active task assignment. We compute a quota for active and standby tasks, by definiing numOfTasks = numberOfActiveTasks+numberOfStandbyTasks and defining the quota as numOfTasks/numberOfMembers rounded up. Whenever a member becomes “full” (its assigned number of tasks is equal to numOfTasks) we deduct its tasks from numOfTasks and decrement numberOfMembers and recompute the quota, which means that the quota may be reduced by one during the assignment process, to avoid uneven assignments. A standby task can be assigned to a process that previously owned it, whenever the process has fewer than numOfMembersOfProcess*quota. This condition will, again, prioritize standby stickyness, and can be evaluated in constant time. In our worst-performing benchmark, this improves the runtime by 2.5x on top of the previous optimizations, but 5x on the more important incremental assignment case. Reviewers: Bill Bejeck <bbejeck@apache.org> |
||
|---|---|---|
| .. | ||
| group-coordinator-api/src/main/java/org/apache/kafka/coordinator/group/api/assignor | ||
| src | ||
| .gitignore | ||