root
/
kafka
mirror of https://github.com/apache/kafka.git
1
0
Fork 0
Code Issues Actions 3 Packages Projects Releases Wiki Activity

Tweaked README, changed default log collection behavior on VerifiableProducer

This commit is contained in:
Geoff Anderson 2015-07-07 11:42:19 -07:00
parent 0eb6fdc6fb
commit 2ea4e29705
3 changed files with 13 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
tests/README.md
View File

@ -15,7 +15,7 @@ This quickstart will help you run the Kafka system tests on your local machine.
* Install Vagrant Plugins: * Install Vagrant Plugins:
# Required # Required
$ vagrant plugin install vagrant-hostmanager $ vagrant plugin install vagrant-hostmanager vagrant-cachier
* Build a specific branch of Kafka * Build a specific branch of Kafka
@ -57,12 +57,15 @@ installing dependencies and updates on every vm.):
EC2 Quickstart EC2 Quickstart
-------------- --------------
This quickstart will help you run the Kafka system tests using Amazon EC2. As a convention, we'll use "kafkatest" in most names, but you can use whatever you want. This quickstart will help you run the Kafka system tests on EC2. In this setup, all logic is run
on EC2 and none on your local machine.
There are a lot of steps here, but the basic goals are to create one distinguished EC2 instance that There are a lot of steps here, but the basic goals are to create one distinguished EC2 instance that
will be our "test driver", and to set up the security groups and iam role so that the test driver will be our "test driver", and to set up the security groups and iam role so that the test driver
can create, destroy, and run ssh commands on any number of "workers". can create, destroy, and run ssh commands on any number of "workers".
As a convention, we'll use "kafkatest" in most names, but you can use whatever name you want.
Preparation Preparation
----------- -----------
In these steps, we will create an IAM role which has permission to create and destroy EC2 instances, In these steps, we will create an IAM role which has permission to create and destroy EC2 instances,

4
tests/kafkatest/services/verifiable_producer.py
View File

@ -23,7 +23,7 @@ class VerifiableProducer(BackgroundThreadService):
logs = { logs = {
"producer_log": { "producer_log": {
"path": "/mnt/producer.log", "path": "/mnt/producer.log",
"collect_default": True} "collect_default": False}
} }
def __init__(self, context, num_nodes, kafka, topic, max_messages=-1, throughput=100000): def __init__(self, context, num_nodes, kafka, topic, max_messages=-1, throughput=100000):
@ -47,8 +47,6 @@ class VerifiableProducer(BackgroundThreadService):
data = self.try_parse_json(line) data = self.try_parse_json(line)
if data is not None: if data is not None:
self.logger.debug("VerifiableProducer: " + str(data))
with self.lock: with self.lock:
if data["name"] == "producer_send_error": if data["name"] == "producer_send_error":
data["node"] = idx data["node"] = idx

9
tests/kafkatest/tests/replication_test.py
View File

@ -97,7 +97,12 @@ class ReplicationTest(Test):
self.logger.info("num consumed: %d" % len(self.consumed)) self.logger.info("num consumed: %d" % len(self.consumed))
# Check produced vs consumed # Check produced vs consumed
self.validate() success, msg = self.validate()
if not success:
self.mark_for_collect(self.producer)
assert success, msg
def clean_shutdown(self): def clean_shutdown(self):
"""Discover leader node for our topic and shut it down cleanly.""" """Discover leader node for our topic and shut it down cleanly."""
@ -142,7 +147,7 @@ class ReplicationTest(Test):
# Collect all the data logs if there was a failure # Collect all the data logs if there was a failure
self.mark_for_collect(self.kafka) self.mark_for_collect(self.kafka)
assert success, msg return success, msg
def test_clean_shutdown(self): def test_clean_shutdown(self):
self.run_with_failure(self.clean_shutdown) self.run_with_failure(self.clean_shutdown)