MiniTrogdorCluster spins up agents from a different thread when scheduling them, but does not use volatiles in these objects. It's not clear that the updated fields are visible.
Reviewers: Colin P. McCabe <cmccabe@apache.org>, Kvicii <Karonazaba@gmail.com>, David Jacot <djacot@confluent.io>
The trogdor ConsumeBenchWorker allows several consumption tasks to be run in parallel, the number is configurable using the threadsPerWorker config. If one of the consumption tasks completes executing successfully due to maxMessages being consumed, then, the consumption task prematurely notifies the doneFuture causing the entire ConsumeBenchWorker to halt. This becomes a problem when more than 1 consumption task is running in parallel, because the successful completion of 1 of the tasks shuts down the entire worker while the other tasks are still running. When the worker is shut down, it kills all the active consumption tasks, though they have not consumed maxMessages yet. This commit defers notification of the doneFuture to the CloseStatusUpdater thread, which is already responsible for tracking the status of the tasks and updating their status when all of the tasks complete.
Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>
* Changes the new Throughput Generators to track messages per window
instead of making per-second calculations which can have rounding errors.
Also, one of these had a calculation error which prompted this change in
the first place.
* Fixes a couple typos.
* Fixes an error where certain JSON fields were not exposed, causing the
workloads to not behave as intended.
* Fixes a bug where we use wait not in a loop, which exits too quickly.
* Adds additional constant payload generators.
* Fixes problems with an example spec.
* Fixes several off-by-one comparisons.
Reviewers: Colin P. McCabe <cmccabe@apache.org>
Move Trogdor out of tools and into its own gradle module. This allows us to minimize
the dependencies of the tools module. We still keep Trogdor in the CLASSPATH
created by kafka-run-class.sh.
Reviewers: Colin P. McCabe <cmccabe@apache.org>
Lucas Bradstreet
Jorge Esteban Quilcate Otoya
Kevin Zhang
dengziming
Kowshik Prakasam
Scott Hendricks
Shay Elkin