Commit Graph

324 Commits

Author SHA1 Message Date
Ismael Juma bcf447e93e KAFKA-4422; Drop support for Scala 2.10 (KIP-119)
Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Ewen Cheslack-Postava <me@ewencp.org>

Closes #2956 from ijuma/kafka-4422-drop-support-scala-2.10
2017-05-11 08:08:11 +01:00
Ismael Juma c2d17c830c MINOR: Update dependencies for 0.11
Worth special mention:

1. Update Scala to 2.11.11 and 2.12.2
2. Update Gradle to 3.5
3. Update ZooKeeper to 3.4.10
4. Update reflections to 0.9.11, which:
    * Switches to jsr305 annotations with a provided scope
    * Updates Guava from 18 to 20
    * Updates javaassist from 3.18 to 3.21

There’s a separate PR for updating RocksDb, so
I didn’t include that here.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #2872 from ijuma/update-deps-for-0.11
2017-04-20 08:55:42 +01:00
Jozef Koval d173395374 MINOR: Update rocksDB dependency to 5.0.1
Author: jozi-k <jozef.koval@protonmail.ch>

Reviewers: Ismael Juma, Guozhang Wang

Closes #2292 from jozi-k/update-rocksdb-4.13.5
2017-01-11 09:37:06 -08:00
Ben Stopford 044b56b8b5 MINOR: Add Replication Quotas Test Rig
This test rig lives in the other.kafka package so isn't part of our standard tests. It provides a convenient mechanism for measuring throttling performance over time. Measurements for each experiment are charted and presented to the user in an html file. The output looks like this:

**Experiment4**
- BrokerCount: 25
- PartitionCount: 100
- Throttle: 4,000,000 B/s
- MsgCount: 1,000
- MsgSize: 100,000
- TargetBytesPerBrokerMB: 400

![image](https://cloud.githubusercontent.com/assets/1297498/19070450/3251bc52-8a23-11e6-88fe-94de6b9147c2.png)
![image](https://cloud.githubusercontent.com/assets/1297498/19070467/4c19f38e-8a23-11e6-986a-ba19d16819ca.png)

Author: Ben Stopford <benstopford@gmail.com>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #1957 from benstopford/throttling-test-rig
2017-01-10 00:42:41 -08:00
Ismael Juma 01d58ad8e0 MINOR: Upgrade Gradle to 3.2.1 and Scala to 2.12.1
There were a couple of important issues fixed in Gradle 3.2.1:
* [GRADLE-3582] - Gradle wrapper fails to escape arguments with nested quotes
* [GRADLE-3583] - Newlines in JAVA_OPTS breaks application plugin shell script in Gradle 3.2

And a lot of important issues fixed in Scala 2.12.1:
* http://www.scala-lang.org/news/2.12.1

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Ewen Cheslack-Postava <me@ewencp.org>

Closes #2216 from ijuma/gradle-3.2.1-and-scala-2.12.1
2016-12-08 23:46:05 +00:00
Ismael Juma 079ea89b36 KAFKA-4417: Update build dependencies for 0.10.2 cycle
Notes on the updates:
* Gradle to 3.2: better incremental build and faster IDE import times (https://docs.gradle.org/3.2/release-notes)
* zkclient to 0.10: it now uses slf4j-api instead of log4j
* zookeeper to 3.4.9: a few important bug fixes (http://zookeeper.apache.org/doc/r3.4.9/releasenotes.html)
* jackson to 2.8.5: lots of updates (https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.6, https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.7, https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.8)
* jersey to 2.24: bug fixes (https://jersey.java.net/release-notes/2.23.html, https://jersey.java.net/release-notes/2.24.html)
* jopt to 5.0.3: minor improvements, the major version bump is due to requiring Java 7 instead of 6 (https://pholser.github.io/jopt-simple/changes.html)
* argparse4j to 0.7.0: minor tweaks and improvements (https://github.com/tatsuhiro-t/argparse4j/blob/argparse4j-0.6.0/NEWS, https://github.com/tatsuhiro-t/argparse4j/blob/argparse4j-0.7.0/NEWS)
* Gradle plugins
* bcpkix to 1.55: quite a few additions, but nothing that matters to us (http://www.bouncycastle.org/releasenotes.html)

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #2144 from ijuma/gradle-3.2-and-deps
2016-11-17 14:43:00 -08:00
Bernard Leach f3aad3b54b KAFKA-4376; Cross compile to Scala 2.12.0
Author: Bernard Leach <leachbj@bouncycastle.org>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #2113 from leachbj/2.12.0-trunk-build
2016-11-17 02:58:13 +00:00
Jozef Koval 6bd866b92f MINOR: Update rocksDB dependency to 4.11.2
There are 32 failing tests on both trunk and my branch.

Author: jozi-k <jozef.koval@protonmail.ch>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #2036 from jozi-k/update-rocksdb-4.11.2
2016-10-19 14:17:33 -07:00
Mathieu Fenniak 08bdffbe4d MINOR: Update rocksDB dependency to 4.9.0
rocksdbjni version 4.9.0 now includes support for running on Windows; this PR updates Kafka Stream's dependency to that version.  Tests pass locally, except for a timeout in testReprocessingFromScratchAfterReset that doesn't seem related; it happens with and without this change.

This contribution is my original work and I license the work to the project under the project's open source license.

Author: Mathieu Fenniak <mathieu.fenniak@replicon.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #1783 from mfenniak/update-rocksdb-4.9
2016-08-25 04:04:40 +01:00
Guozhang Wang d903babb72 MINOR: add slf4jlog4j to streams example
Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #1731 from guozhangwang/Kminor-log4j-streams-examples
2016-08-22 23:52:50 +01:00
Ismael Juma 44723caec7 MINOR: Upgrade RocksDB to 4.8.0
Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #1647 from ijuma/kafka-rocksdb-4.8
2016-07-21 16:11:20 -07:00
Filipe Azevedo 4c6d7ed95a KAFKA-3838; Update zkClient to 0.9 and Zookeeper to 3.4.8
Author: Filipe Azevedo <filipe@azevedo.io>

Reviewers: Flavio Junqueira <fpj@apache.org>, Ismael Juma <ismael@juma.me.uk>

Closes #1512 from mangas/trunk
2016-06-17 08:51:55 +02:00
Grant Henke 27cb6686fd KAFKA-3789; Upgrade Snappy to fix snappy decompression errors
Author: Grant Henke <granthenke@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #1467 from granthenke/snappy-fix
2016-06-04 22:43:54 +01:00
Ismael Juma 4fa456bc6e KAFKA-3557; Update rocksdb to 4.4.1 and patch updates to snappy and slf4j
* The hope is that RocksDb 4.4.1 is more stable than 4.1.0 (occasional segfaults) and 4.2.0 (very frequent segfaults), release notes for 4.4.1: https://www.facebook.com/groups/rocksdb.dev/permalink/925995520832296/
* slf4j 1.7.21 includes thread-safety fixes: http://www.slf4j.org/news.html
* snappy 1.1.2.4 includes performance improvements requested by Spark, which apply to our usage: https://github.com/xerial/snappy-java/blob/master/Milestone.md

I ran the stream tests several times and they passed every time while 4.2.0 segfaulted every time.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Eno Thereska <eno.thereska@gmail.com>, Guozhang Wang <wangguoz@gmail.com>

Closes #1219 from ijuma/kafka-3557-update-rocks-db-4.4.1-snappy-slf4j
2016-04-14 10:44:05 -07:00
Grant Henke 34a5944721 MINOR: Remove unused hadoop version
All dependencies on hadoop were removed with MiniKDC. This removes the left over version entry.

Author: Grant Henke <granthenke@gmail.com>

Reviewers: Ismael Juma

Closes #1214 from granthenke/remove-hadoop
2016-04-12 13:47:02 -07:00
Grant Henke 623ab1e7c6 KAFKA-3451: Add basic HTML coverage report generation to gradle
Author: Grant Henke <granthenke@gmail.com>

Reviewers: Gwen Shapira, Ismael Juma, Ewen Cheslack-Postava

Closes #1121 from granthenke/coverage
2016-03-31 10:07:54 -07:00
Ismael Juma 78d91dcd88 KAFKA-3475; Introduce our own `MiniKdc`
This also fixes KAFKA-3453 and KAFKA-2866.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Gwen Shapira

Closes #1155 from ijuma/kafka-3475-introduce-our-minikdc
2016-03-30 19:30:34 -07:00
Ewen Cheslack-Postava eb823281a5 KAFKA-3424: Add CORS support to Connect REST API
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Gwen Shapira

Closes #1099 from ewencp/cors-rest-support
2016-03-19 18:39:52 -07:00
Grant Henke cd8bd606cb KAFKA-3403: Upgrade ZkClient to 0.8
This ZkClient version adds authentication validation and a conditional delete method needed for other patches

Author: Grant Henke <granthenke@gmail.com>

Reviewers: Ismael Juma, Gwen Shapira

Closes #1084 from granthenke/zkclient-08
2016-03-16 15:07:37 -07:00
Ismael Juma ee9995583a KAFKA-3357; Update to Scala 2.11.8
Scala 2.11.8 has been released with a number of bug fixes:
* http://www.scala-lang.org/news/2.11.8/

There are a few important collection fixes:
* https://issues.scala-lang.org/browse/SI-9497
* https://github.com/scala/scala/pull/4714
* https://github.com/scala/scala/pull/4693

And also some pattern matcher fixes.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Grant Henke, Ashish Singh

Closes #1032 from ijuma/kafka-3357-update-to-scala-2.11.8
2016-03-09 17:57:24 -08:00
Grant Henke f25fe02d99 KAFKA-3286: Add plugin to quickly check for outdated dependencies
Adds a gradle task to generate a report of outdate release dependencies:
`gradle dependencyUpdates`

Updates a few minor versions.

Author: Grant Henke <granthenke@gmail.com>

Reviewers: Ismael Juma, Gwen Shapira

Closes #973 from granthenke/outdated-deps
2016-03-03 15:10:16 -08:00
Ismael Juma 46e4d40135 MINOR: Update streams RocksDb to 4.1.0
This is the latest version in Maven even though HISTORY.md includes releases all the way to 4.5.0.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Grant Henke <ghenke@cloudera.com>, Guozhang Wang <wangguoz@gmail.com>

Closes #937 from ijuma/update-rocks-db-for-streams
2016-02-22 13:03:24 -08:00
Ismael Juma f355918ec7 KAFKA-3227; Conservative update of Kafka dependencies
Patch version bumps for bouncy castle, minikdc, snappy, slf4j, scalatest and powermock. Notable fixes:

* Snappy: fixes a resource leak
* Bouncy castle: security fixes

Also update Gradle to 2.11 (where the notable change is improved IDE integration) and the grgit build dependency.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Grant Henke <granthenke@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>

Closes #903 from ijuma/kafka-3227-conservative-update-of-kafka-deps
2016-02-16 10:19:00 -05:00
Grant Henke f009c30947 KAFKA-3021: Centralize dependency version management
Author: Grant Henke <granthenke@gmail.com>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #741 from granthenke/central-deps
2016-01-11 14:16:58 -08:00