root
/
kafka
mirror of https://github.com/apache/kafka.git
1
0
Fork 0
Code Issues Actions 2 Packages Projects Releases Wiki Activity

KAFKA-4467: Run tests on travis-ci using docker 

ijuma ewencp cmccabe harshach Please review.
Here is a sample run:
https://travis-ci.org/raghavgautam/kafka/builds/191714520

In this run 214 tests were run and 144 tests passed.

I will open separate jiras for fixing failures.

Author: Raghav Kumar Gautam <raghav@apache.org>

Reviewers: Sriharsha Chintalapani <harsha@hortonworks.com>, Ewen Cheslack-Postava <ewen@confluent.io>

Closes #2376 from raghavgautam/trunk
This commit is contained in:
Raghav Kumar Gautam 2017-03-09 16:24:38 -08:00 committed by Ewen Cheslack-Postava
parent 65650ba4dc
commit dbcbd7920f
3 changed files with 358 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

54
.travis.yml Normal file
View File

@ -0,0 +1,54 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
sudo: required
dist: trusty
language: java
env:
- _DUCKTAPE_OPTIONS="--subset 0 --subsets 15"
- _DUCKTAPE_OPTIONS="--subset 1 --subsets 15"
- _DUCKTAPE_OPTIONS="--subset 2 --subsets 15"
- _DUCKTAPE_OPTIONS="--subset 3 --subsets 15"
- _DUCKTAPE_OPTIONS="--subset 4 --subsets 15"
- _DUCKTAPE_OPTIONS="--subset 5 --subsets 15"
- _DUCKTAPE_OPTIONS="--subset 6 --subsets 15"
- _DUCKTAPE_OPTIONS="--subset 7 --subsets 15"
- _DUCKTAPE_OPTIONS="--subset 8 --subsets 15"
- _DUCKTAPE_OPTIONS="--subset 9 --subsets 15"
- _DUCKTAPE_OPTIONS="--subset 10 --subsets 15"
- _DUCKTAPE_OPTIONS="--subset 11 --subsets 15"
- _DUCKTAPE_OPTIONS="--subset 12 --subsets 15"
- _DUCKTAPE_OPTIONS="--subset 13 --subsets 15"
- _DUCKTAPE_OPTIONS="--subset 14 --subsets 15"
jdk:
- oraclejdk8
before_install:
- gradle wrapper
script:
- ./gradlew rat
- ./gradlew releaseTarGz && /bin/bash ./tests/docker/run_tests.sh
services:
- docker
before_cache:
- rm -f $HOME/.gradle/caches/modules-2/modules-2.lock
- rm -fr $HOME/.gradle/caches/*/plugin-resolution/
cache:
directories:
- "$HOME/.m2/repository"
- "$HOME/.gradle/caches/"
- "$HOME/.gradle/wrapper/"

303
tests/README.md
View File

@ -31,6 +31,309 @@ TC_PATHS="tests/kafkatest/tests/streams tests/kafkatest/tests/tools" bash tests/
- The docker containers are named knode01, knode02 etc. - The docker containers are named knode01, knode02 etc.
These nodes can't be used for any other purpose. These nodes can't be used for any other purpose.
Examining CI run
----------------
* Set BUILD_ID is travis ci's build id. E.g. build id is 169519874 for the following build
```bash
https://travis-ci.org/apache/kafka/builds/169519874
```
* Getting number of tests that were actually run
```bash
for id in $(curl -sSL https://api.travis-ci.org/builds/$BUILD_ID | jq '.matrix|map(.id)|.[]'); do curl -sSL "https://api.travis-ci.org/jobs/$id/log.txt?deansi=true" ; done | grep -cE 'RunnerClient: Loading test'
```
* Getting number of tests that passed
```bash
for id in $(curl -sSL https://api.travis-ci.org/builds/$BUILD_ID | jq '.matrix|map(.id)|.[]'); do curl -sSL "https://api.travis-ci.org/jobs/$id/log.txt?deansi=true" ; done | grep -cE 'RunnerClient.*PASS'
```
* Getting all the logs produced from a run
```bash
for id in $(curl -sSL https://api.travis-ci.org/builds/$BUILD_ID | jq '.matrix|map(.id)|.[]'); do curl -sSL "https://api.travis-ci.org/jobs/$id/log.txt?deansi=true" ; done
```
* Explanation of curl calls to travis-ci & jq commands
- We get json information of the build using the following command
```bash
curl -sSL https://api.travis-ci.org/apache/kafka/builds/169519874
```
This produces a json about the build which looks like:
```json
{
"id": 169519874,
"repository_id": 6097916,
"number": "19",
"config": {
"sudo": "required",
"dist": "trusty",
"language": "java",
"env": [
"TC_PATHS=\"tests/kafkatest/tests/client\"",
"TC_PATHS=\"tests/kafkatest/tests/connect tests/kafkatest/tests/streams tests/kafkatest/tests/tools\"",
"TC_PATHS=\"tests/kafkatest/tests/mirror_maker\"",
"TC_PATHS=\"tests/kafkatest/tests/replication\"",
"TC_PATHS=\"tests/kafkatest/tests/upgrade\"",
"TC_PATHS=\"tests/kafkatest/tests/security\"",
"TC_PATHS=\"tests/kafkatest/tests/core\""
],
"jdk": [
"oraclejdk8"
],
"before_install": null,
"script": [
"./gradlew releaseTarGz && /bin/bash ./tests/travis/run_tests.sh"
],
"services": [
"docker"
],
"before_cache": [
"rm -f $HOME/.gradle/caches/modules-2/modules-2.lock",
"rm -fr $HOME/.gradle/caches/*/plugin-resolution/"
],
"cache": {
"directories": [
"$HOME/.m2/repository",
"$HOME/.gradle/caches/",
"$HOME/.gradle/wrapper/"
]
},
".result": "configured",
"group": "stable"
},
"state": "finished",
"result": null,
"status": null,
"started_at": "2016-10-21T13:35:43Z",
"finished_at": "2016-10-21T14:46:03Z",
"duration": 16514,
"commit": "7e583d9ea08c70dbbe35a3adde72ed203a797f64",
"branch": "trunk",
"message": "respect _DUCK_OPTIONS",
"committed_at": "2016-10-21T00:12:36Z",
"author_name": "Raghav Kumar Gautam",
"author_email": "raghav@apache.org",
"committer_name": "Raghav Kumar Gautam",
"committer_email": "raghav@apache.org",
"compare_url": "https://github.com/raghavgautam/kafka/compare/cc788ac99ca7...7e583d9ea08c",
"event_type": "push",
"matrix": [
{
"id": 169519875,
"repository_id": 6097916,
"number": "19.1",
"config": {
"sudo": "required",
"dist": "trusty",
"language": "java",
"env": "TC_PATHS=\"tests/kafkatest/tests/client\"",
"jdk": "oraclejdk8",
"before_install": null,
"script": [
"./gradlew releaseTarGz && /bin/bash ./tests/travis/run_tests.sh"
],
"services": [
"docker"
],
"before_cache": [
"rm -f $HOME/.gradle/caches/modules-2/modules-2.lock",
"rm -fr $HOME/.gradle/caches/*/plugin-resolution/"
],
"cache": {
"directories": [
"$HOME/.m2/repository",
"$HOME/.gradle/caches/",
"$HOME/.gradle/wrapper/"
]
},
".result": "configured",
"group": "stable",
"os": "linux"
},
"result": null,
"started_at": "2016-10-21T13:35:43Z",
"finished_at": "2016-10-21T14:24:50Z",
"allow_failure": false
},
{
"id": 169519876,
"repository_id": 6097916,
"number": "19.2",
"config": {
"sudo": "required",
"dist": "trusty",
"language": "java",
"env": "TC_PATHS=\"tests/kafkatest/tests/connect tests/kafkatest/tests/streams tests/kafkatest/tests/tools\"",
"jdk": "oraclejdk8",
"before_install": null,
"script": [
"./gradlew releaseTarGz && /bin/bash ./tests/travis/run_tests.sh"
],
"services": [
"docker"
],
"before_cache": [
"rm -f $HOME/.gradle/caches/modules-2/modules-2.lock",
"rm -fr $HOME/.gradle/caches/*/plugin-resolution/"
],
"cache": {
"directories": [
"$HOME/.m2/repository",
"$HOME/.gradle/caches/",
"$HOME/.gradle/wrapper/"
]
},
".result": "configured",
"group": "stable",
"os": "linux"
},
"result": 1,
"started_at": "2016-10-21T13:35:46Z",
"finished_at": "2016-10-21T14:22:05Z",
"allow_failure": false
},
...
]
}
```
- By passing this through jq filter `.matrix` we extract the matrix part of the json
```bash
curl -sSL https://api.travis-ci.org/apache/kafka/builds/169519874 | jq '.matrix'
```
The resulting json looks like:
```json
[
{
"id": 169519875,
"repository_id": 6097916,
"number": "19.1",
"config": {
"sudo": "required",
"dist": "trusty",
"language": "java",
"env": "TC_PATHS=\"tests/kafkatest/tests/client\"",
"jdk": "oraclejdk8",
"before_install": null,
"script": [
"./gradlew releaseTarGz && /bin/bash ./tests/travis/run_tests.sh"
],
"services": [
"docker"
],
"before_cache": [
"rm -f $HOME/.gradle/caches/modules-2/modules-2.lock",
"rm -fr $HOME/.gradle/caches/*/plugin-resolution/"
],
"cache": {
"directories": [
"$HOME/.m2/repository",
"$HOME/.gradle/caches/",
"$HOME/.gradle/wrapper/"
]
},
".result": "configured",
"group": "stable",
"os": "linux"
},
"result": null,
"started_at": "2016-10-21T13:35:43Z",
"finished_at": "2016-10-21T14:24:50Z",
"allow_failure": false
},
{
"id": 169519876,
"repository_id": 6097916,
"number": "19.2",
"config": {
"sudo": "required",
"dist": "trusty",
"language": "java",
"env": "TC_PATHS=\"tests/kafkatest/tests/connect tests/kafkatest/tests/streams tests/kafkatest/tests/tools\"",
"jdk": "oraclejdk8",
"before_install": null,
"script": [
"./gradlew releaseTarGz && /bin/bash ./tests/travis/run_tests.sh"
],
"services": [
"docker"
],
"before_cache": [
"rm -f $HOME/.gradle/caches/modules-2/modules-2.lock",
"rm -fr $HOME/.gradle/caches/*/plugin-resolution/"
],
"cache": {
"directories": [
"$HOME/.m2/repository",
"$HOME/.gradle/caches/",
"$HOME/.gradle/wrapper/"
]
},
".result": "configured",
"group": "stable",
"os": "linux"
},
"result": 1,
"started_at": "2016-10-21T13:35:46Z",
"finished_at": "2016-10-21T14:22:05Z",
"allow_failure": false
},
...
]
```
- By further passing this through jq filter `map(.id)` we extract the id of
the builds for each of the splits
```bash
curl -sSL https://api.travis-ci.org/apache/kafka/builds/169519874 | jq '.matrix|map(.id)'
```
The resulting json looks like:
```json
[
169519875,
169519876,
169519877,
169519878,
169519879,
169519880,
169519881
]
```
- To use these ids in for loop we want to get rid of `[]` which is done by
passing it through `.[]` filter
```bash
curl -sSL https://api.travis-ci.org/apache/kafka/builds/169519874 | jq '.matrix|map(.id)|.[]'
```
And we get
```text
169519875
169519876
169519877
169519878
169519879
169519880
169519881
```
- In the for loop we have made calls to fetch logs
```bash
curl -sSL "https://api.travis-ci.org/jobs/169519875/log.txt?deansi=true" | tail
```
which gives us
```text
[INFO:2016-10-21 14:21:12,538]: SerialTestRunner: kafkatest.tests.client.consumer_test.OffsetValidationTest.test_consumer_bounce.clean_shutdown=False.bounce_mode=rolling: test 16 of 28
[INFO:2016-10-21 14:21:12,538]: SerialTestRunner: kafkatest.tests.client.consumer_test.OffsetValidationTest.test_consumer_bounce.clean_shutdown=False.bounce_mode=rolling: setting up
[INFO:2016-10-21 14:21:30,810]: SerialTestRunner: kafkatest.tests.client.consumer_test.OffsetValidationTest.test_consumer_bounce.clean_shutdown=False.bounce_mode=rolling: running
[INFO:2016-10-21 14:24:35,519]: SerialTestRunner: kafkatest.tests.client.consumer_test.OffsetValidationTest.test_consumer_bounce.clean_shutdown=False.bounce_mode=rolling: PASS
[INFO:2016-10-21 14:24:35,519]: SerialTestRunner: kafkatest.tests.client.consumer_test.OffsetValidationTest.test_consumer_bounce.clean_shutdown=False.bounce_mode=rolling: tearing down
The job exceeded the maximum time limit for jobs, and has been terminated.
```
* Links
- [Travis-CI REST api documentation](https://docs.travis-ci.com/api)
- [jq Manual](https://stedolan.github.io/jq/manual/)
Local Quickstart Local Quickstart
---------------- ----------------
This quickstart will help you run the Kafka system tests on your local machine. Note this requires bringing up a cluster of virtual machines on your local computer, which is memory intensive; it currently requires around 10G RAM. This quickstart will help you run the Kafka system tests on your local machine. Note this requires bringing up a cluster of virtual machines on your local computer, which is memory intensive; it currently requires around 10G RAM.

2
tests/docker/Dockerfile
View File

@ -21,7 +21,7 @@ ENV MIRROR="http://apache.cs.utah.edu/"
ADD ssh /root/.ssh ADD ssh /root/.ssh
RUN chmod 600 /root/.ssh/id_rsa RUN chmod 600 /root/.ssh/id_rsa
RUN apt update && apt install -y unzip wget curl jq coreutils openssh-server net-tools vim openjdk-8-jdk python-pip python-dev libffi-dev libssl-dev RUN apt update && apt install -y unzip wget curl jq coreutils openssh-server net-tools vim python-pip python-dev libffi-dev libssl-dev
RUN pip install -U pip && pip install --upgrade cffi ducktape==0.6.0 RUN pip install -U pip && pip install --upgrade cffi ducktape==0.6.0
RUN mkdir -p "/opt/kafka-0.8.2.2" && curl -s "${MIRROR}kafka/0.8.2.2/kafka_2.10-0.8.2.2.tgz" | tar xz --strip-components=1 -C "/opt/kafka-0.8.2.2" RUN mkdir -p "/opt/kafka-0.8.2.2" && curl -s "${MIRROR}kafka/0.8.2.2/kafka_2.10-0.8.2.2.tgz" | tar xz --strip-components=1 -C "/opt/kafka-0.8.2.2"
RUN mkdir -p "/opt/kafka-0.9.0.1" && curl -s "${MIRROR}kafka/0.9.0.1/kafka_2.10-0.9.0.1.tgz" | tar xz --strip-components=1 -C "/opt/kafka-0.9.0.1" RUN mkdir -p "/opt/kafka-0.9.0.1" && curl -s "${MIRROR}kafka/0.9.0.1/kafka_2.10-0.9.0.1.tgz" | tar xz --strip-components=1 -C "/opt/kafka-0.9.0.1"