[VOTE] 2.1.0 RC1

Discussion:

[VOTE] 2.1.0 RC1

Dong Lin

2018-11-09 23:33:11 UTC

Hello Kafka users, developers and client-developers,

This is the second candidate for feature release of Apache Kafka 2.1.0.

This is a major version release of Apache Kafka. It includes 28 new KIPs and

critical bug fixes. Please see the Kafka 2.1.0 release plan for more
details:

*https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044*
<https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044>

Here are a few notable highlights:

- Java 11 support
- Support for Zstandard, which achieves compression comparable to gzip with
higher compression and especially decompression speeds(KIP-110)
- Avoid expiring committed offsets for active consumer group (KIP-211)
- Provide Intuitive User Timeouts in The Producer (KIP-91)
- Kafka's replication protocol now supports improved fencing of zombies.
Previously, under certain rare conditions, if a broker became partitioned
from Zookeeper but not the rest of the cluster, then the logs of replicated
partitions could diverge and cause data loss in the worst case (KIP-320)
- Streams API improvements (KIP-319, KIP-321, KIP-330, KIP-353, KIP-356)
- Admin script and admin client API improvements to simplify admin
operation (KIP-231, KIP-308, KIP-322, KIP-324, KIP-338, KIP-340)
- DNS handling improvements (KIP-235, KIP-302)

Release notes for the 2.1.0 release:
http://home.apache.org/~lindong/kafka-2.1.0-rc0/RELEASE_NOTES.html

*** Please download, test and vote by Thursday, Nov 15, 12 pm PT ***

* Kafka's KEYS file containing PGP keys we use to sign the release:
http://kafka.apache.org/KEYS

* Release artifacts to be voted upon (source and binary):
http://home.apache.org/~lindong/kafka-2.1.0-rc1/

* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/

* Javadoc:
http://home.apache.org/~lindong/kafka-2.1.0-rc1/javadoc/

* Tag to be voted upon (off 2.1 branch) is the 2.1.0-rc1 tag:
https://github.com/apache/kafka/tree/2.1.0-rc1

* Documentation:
*http://kafka.apache.org/21/documentation.html*
<http://kafka.apache.org/21/documentation.html>

* Protocol:
http://kafka.apache.org/21/protocol.html

* Successful Jenkins builds for the 2.1 branch:
Unit/integration tests: *https://builds.apache.org/job/kafka-2.1-jdk8/50/
<https://builds.apache.org/job/kafka-2.1-jdk8/50/>*

Please test and verify the release artifacts and submit a vote for this RC,
or report any issues so we can fix them and get a new RC out ASAP. Although
this release vote requires PMC votes to pass, testing, votes, and bug
reports are valuable and appreciated from everyone.

Cheers,
Dong

Jonathan Santilli

2018-11-11 19:33:26 UTC

Permalink

Hello,

+1

I have downloaded the release artifact from
http://home.apache.org/~lindong/kafka-2.1.0-rc1/
Executed a 3 brokers cluster. (java8 8u192b12)
Executed kafka-monitor for about 1 hour without problems.

Thanks,
--
Jonathan

Post by Dong Lin
Hello Kafka users, developers and client-developers,
This is the second candidate for feature release of Apache Kafka 2.1.0.
This is a major version release of Apache Kafka. It includes 28 new KIPs and
critical bug fixes. Please see the Kafka 2.1.0 release plan for more
*
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044*
<https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044
- Java 11 support
- Support for Zstandard, which achieves compression comparable to gzip with
higher compression and especially decompression speeds(KIP-110)
- Avoid expiring committed offsets for active consumer group (KIP-211)
- Provide Intuitive User Timeouts in The Producer (KIP-91)
- Kafka's replication protocol now supports improved fencing of zombies.
Previously, under certain rare conditions, if a broker became partitioned
from Zookeeper but not the rest of the cluster, then the logs of replicated
partitions could diverge and cause data loss in the worst case (KIP-320)
- Streams API improvements (KIP-319, KIP-321, KIP-330, KIP-353, KIP-356)
- Admin script and admin client API improvements to simplify admin
operation (KIP-231, KIP-308, KIP-322, KIP-324, KIP-338, KIP-340)
- DNS handling improvements (KIP-235, KIP-302)
http://home.apache.org/~lindong/kafka-2.1.0-rc0/RELEASE_NOTES.html
*** Please download, test and vote by Thursday, Nov 15, 12 pm PT ***
http://kafka.apache.org/KEYS
http://home.apache.org/~lindong/kafka-2.1.0-rc1/
https://repository.apache.org/content/groups/staging/
http://home.apache.org/~lindong/kafka-2.1.0-rc1/javadoc/
https://github.com/apache/kafka/tree/2.1.0-rc1
*http://kafka.apache.org/21/documentation.html*
<http://kafka.apache.org/21/documentation.html>
http://kafka.apache.org/21/protocol.html
Unit/integration tests: *https://builds.apache.org/job/kafka-2.1-jdk8/50/
<https://builds.apache.org/job/kafka-2.1-jdk8/50/>*
Please test and verify the release artifacts and submit a vote for this RC,
or report any issues so we can fix them and get a new RC out ASAP. Although
this release vote requires PMC votes to pass, testing, votes, and bug
reports are valuable and appreciated from everyone.
Cheers,
Dong

--
Santilli Jonathan

Eno Thereska

2018-11-13 09:51:16 UTC

Permalink

Built code and ran tests. Getting a single integration test failure:

kafka.log.LogCleanerParameterizedIntegrationTest >
testCleansCombinedCompactAndDeleteTopic[3] FAILED
java.lang.AssertionError: Contents of the map shouldn't change
expected:<Map(0 -> (340,340), 5 -> (345,345), 10 -> (350,350), 14 ->
(354,354), 1 -> (341,341), 6 -> (346,346), 9 -> (349,349), 13 -> (353,353),
2 -> (342,342), 17 -> (357,357), 12 -> (352,352), 7 -> (347,347), 3 ->
(343,343), 18 -> (358,358), 16 -> (356,356), 11 -> (351,351), 8 ->
(348,348), 19 -> (359,359), 4 -> (344,344), 15 -> (355,355))> but
was:<Map(0 -> (340,340), 5 -> (345,345), 10 -> (350,350), 14 -> (354,354),
1 -> (341,341), 6 -> (346,346), 9 -> (349,349), 13 -> (353,353), 2 ->
(342,342), 17 -> (357,357), 12 -> (352,352), 7 -> (347,347), 3 ->
(343,343), 18 -> (358,358), 16 -> (356,356), 11 -> (351,351), 99 ->
(299,299), 8 -> (348,348), 19 -> (359,359), 4 -> (344,344), 15 ->
(355,355))>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:118)
at
kafka.log.LogCleanerParameterizedIntegrationTest.testCleansCombinedCompactAndDeleteTopic(LogCleanerParameterizedIntegrationTest.scala:129)

Thanks
Eno

On Sun, Nov 11, 2018 at 7:34 PM Jonathan Santilli <

Post by Jonathan Santilli
Hello,
+1
I have downloaded the release artifact from
http://home.apache.org/~lindong/kafka-2.1.0-rc1/
Executed a 3 brokers cluster. (java8 8u192b12)
Executed kafka-monitor for about 1 hour without problems.
Thanks,
--
Jonathan

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044*

Post by Dong Lin
<

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044

Post by Dong Lin
- Java 11 support
- Support for Zstandard, which achieves compression comparable to gzip

with

Post by Dong Lin
higher compression and especially decompression speeds(KIP-110)
- Avoid expiring committed offsets for active consumer group (KIP-211)
- Provide Intuitive User Timeouts in The Producer (KIP-91)
- Kafka's replication protocol now supports improved fencing of zombies.
Previously, under certain rare conditions, if a broker became partitioned
from Zookeeper but not the rest of the cluster, then the logs of

replicated

Post by Dong Lin
partitions could diverge and cause data loss in the worst case (KIP-320)
- Streams API improvements (KIP-319, KIP-321, KIP-330, KIP-353, KIP-356)
- Admin script and admin client API improvements to simplify admin
operation (KIP-231, KIP-308, KIP-322, KIP-324, KIP-338, KIP-340)
- DNS handling improvements (KIP-235, KIP-302)
http://home.apache.org/~lindong/kafka-2.1.0-rc0/RELEASE_NOTES.html
*** Please download, test and vote by Thursday, Nov 15, 12 pm PT ***
http://kafka.apache.org/KEYS
http://home.apache.org/~lindong/kafka-2.1.0-rc1/
https://repository.apache.org/content/groups/staging/
http://home.apache.org/~lindong/kafka-2.1.0-rc1/javadoc/
https://github.com/apache/kafka/tree/2.1.0-rc1
*http://kafka.apache.org/21/documentation.html*
<http://kafka.apache.org/21/documentation.html>
http://kafka.apache.org/21/protocol.html
Unit/integration tests: *

https://builds.apache.org/job/kafka-2.1-jdk8/50/

Post by Dong Lin
<https://builds.apache.org/job/kafka-2.1-jdk8/50/>*
Please test and verify the release artifacts and submit a vote for this

RC,

Post by Dong Lin
or report any issues so we can fix them and get a new RC out ASAP.

Although

Post by Dong Lin
this release vote requires PMC votes to pass, testing, votes, and bug
reports are valuable and appreciated from everyone.
Cheers,
Dong

--
Santilli Jonathan

Andras Beni

2018-11-13 10:07:11 UTC

Permalink

+1 (non-binding)

Verified signatures and checksums of release artifacts
Performed quickstart steps on rc artifacts (both scala 2.11 and 2.12)

Andras

Post by Eno Thereska
kafka.log.LogCleanerParameterizedIntegrationTest >
testCleansCombinedCompactAndDeleteTopic[3] FAILED
java.lang.AssertionError: Contents of the map shouldn't change
expected:<Map(0 -> (340,340), 5 -> (345,345), 10 -> (350,350), 14 ->
(354,354), 1 -> (341,341), 6 -> (346,346), 9 -> (349,349), 13 -> (353,353),
2 -> (342,342), 17 -> (357,357), 12 -> (352,352), 7 -> (347,347), 3 ->
(343,343), 18 -> (358,358), 16 -> (356,356), 11 -> (351,351), 8 ->
(348,348), 19 -> (359,359), 4 -> (344,344), 15 -> (355,355))> but
was:<Map(0 -> (340,340), 5 -> (345,345), 10 -> (350,350), 14 -> (354,354),
1 -> (341,341), 6 -> (346,346), 9 -> (349,349), 13 -> (353,353), 2 ->
(342,342), 17 -> (357,357), 12 -> (352,352), 7 -> (347,347), 3 ->
(343,343), 18 -> (358,358), 16 -> (356,356), 11 -> (351,351), 99 ->
(299,299), 8 -> (348,348), 19 -> (359,359), 4 -> (344,344), 15 ->
(355,355))>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:118)
at
kafka.log.LogCleanerParameterizedIntegrationTest.testCleansCombinedCompactAndDeleteTopic(LogCleanerParameterizedIntegrationTest.scala:129)
Thanks
Eno
On Sun, Nov 11, 2018 at 7:34 PM Jonathan Santilli <

KIPs

Post by Jonathan Santilli

Post by Dong Lin
and
critical bug fixes. Please see the Kafka 2.1.0 release plan for more
*

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044*

Post by Jonathan Santilli

Post by Dong Lin
<

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044

Post by Jonathan Santilli

Post by Dong Lin
- Java 11 support
- Support for Zstandard, which achieves compression comparable to gzip

with

zombies.

Post by Jonathan Santilli

Post by Dong Lin
Previously, under certain rare conditions, if a broker became

partitioned

Post by Jonathan Santilli

Post by Dong Lin
from Zookeeper but not the rest of the cluster, then the logs of

replicated

Post by Dong Lin
partitions could diverge and cause data loss in the worst case

(KIP-320)

Post by Jonathan Santilli

Post by Dong Lin
- Streams API improvements (KIP-319, KIP-321, KIP-330, KIP-353,

KIP-356)

Post by Jonathan Santilli

Post by Dong Lin
- Admin script and admin client API improvements to simplify admin
operation (KIP-231, KIP-308, KIP-322, KIP-324, KIP-338, KIP-340)
- DNS handling improvements (KIP-235, KIP-302)
http://home.apache.org/~lindong/kafka-2.1.0-rc0/RELEASE_NOTES.html
*** Please download, test and vote by Thursday, Nov 15, 12 pm PT ***
http://kafka.apache.org/KEYS
http://home.apache.org/~lindong/kafka-2.1.0-rc1/
https://repository.apache.org/content/groups/staging/
http://home.apache.org/~lindong/kafka-2.1.0-rc1/javadoc/
https://github.com/apache/kafka/tree/2.1.0-rc1
*http://kafka.apache.org/21/documentation.html*
<http://kafka.apache.org/21/documentation.html>
http://kafka.apache.org/21/protocol.html
Unit/integration tests: *

https://builds.apache.org/job/kafka-2.1-jdk8/50/

Post by Dong Lin
<https://builds.apache.org/job/kafka-2.1-jdk8/50/>*
Please test and verify the release artifacts and submit a vote for this

RC,

Post by Dong Lin
or report any issues so we can fix them and get a new RC out ASAP.

Although

Post by Dong Lin
this release vote requires PMC votes to pass, testing, votes, and bug
reports are valuable and appreciated from everyone.
Cheers,
Dong

--
Santilli Jonathan

Vahid Hashemian

2018-11-15 14:54:04 UTC

Permalink

Hi Dong,

Thanks for running the release.

I built binaries from the source, ran quickstart and tests on Ubuntu (with
Java 8 & 9).

I noticed two issues:
* (minor) the documentation still mentions 2.0, and submitted a minor PR
<https://github.com/apache/kafka/pull/5916> to update it.
* the unit test
`kafka.server.DynamicBrokerReconfigurationTest.testUncleanLeaderElectionEnable`
failed for me with the following error:

```
kafka.server.DynamicBrokerReconfigurationTest >
testUncleanLeaderElectionEnable FAILED
java.lang.AssertionError: Unclean leader not elected
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
kafka.server.DynamicBrokerReconfigurationTest.testUncleanLeaderElectionEnable(DynamicBrokerReconfigurationTest.scala:487)
```

--Vahid

--
Thanks!
--Vahid

Jakub Scholz

2018-11-17 23:28:09 UTC

Permalink

+1 (non-binding) ... I used the staged artifacts and binaries and run my
tests

Satish Duggana

2018-11-18 16:25:55 UTC

Permalink

+1 (non-binding)

- Ran testAll/releaseTarGzAll on 2.1.0 successfully without any failures.
- Ran through quickstart of core/streams on builds generated from tag
- Ran few internal apps targeting to topics on 3 node cluster.

Thanks,
Satish.

Post by Jakub Scholz
+1 (non-binding) ... I used the staged artifacts and binaries and run my
tests