Pausing a partition in Kafka Consumer

Guozhang Wang

2016-08-22 17:48:45 UTC

Hi Kamal,

Thanks for reporting this. I think we can carry the paused partitions info
across rebalances so that if the partitions are assigned to the consumers
again they are still automatically paused (currently we reset such
information after rebalance).

Guozhang

Post by Kamal C
Hi all,
In the javadoc for KafkaConsumer#pause
<http://kafka.apache.org/0100/javadoc/org/apache/kafka/clients/consumer/
KafkaConsumer.html#pause(java.util.Collection)>
it's stated that,
Suspend fetching from the requested partitions. Future calls to poll(long)

Post by Kamal C
<http://kafka.apache.org/0100/javadoc/org/apache/kafka/clients/consumer/

KafkaConsumer.html#poll%28long%29>

Post by Kamal C
will not return any records from these partitions until they have been
resumed using resume(Collection)
<http://kafka.apache.org/0100/javadoc/org/apache/kafka/clients/consumer/

KafkaConsumer.html#resume%28java.util.Collection%29>.

Post by Kamal C
Note that this method does not affect partition subscription. In
particular, it does not cause a group rebalance when automatic assignment
is used.

If a new consumer instance joins the group and the paused partition
assigned to this consumer it starts to read data from that partition. How
to pause a partition even if re-balance occurs ?
Regards,
Kamal C

--
-- Guozhang