节点故障后卡夫卡消费者错误

Question

我们正在 3 实例 kafka 集群上测试节点故障场景，复制因子为 2。

删除实例后，消费者不断失败。

消费者正在使用 kafka 流读取消息

谢谢

这是消费者失败日志：

08:09:19.667 [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] ERROR o.a.k.c.c.i.ConsumerCoordinator - User provided listener org.apache.kafka.streams.processor.internals.StreamThread$RebalanceListener for group ComponentsActivityEventsStream failed on partition assignment

org.apache.kafka.streams.errors.StreamsException: Store ComponentsActivityStore's change log (ComponentsActivityEventsStream-ComponentsActivityStore-changelog) does not contain partition 0

    at org.apache.kafka.streams.processor.internals.StoreChangelogReader.validatePartitionExists(StoreChangelogReader.java:87)

    at org.apache.kafka.streams.processor.internals.ProcessorStateManager.register(ProcessorStateManager.java:165)

    at org.apache.kafka.streams.processor.internals.AbstractProcessorContext.register(AbstractProcessorContext.java:100)

    at org.apache.kafka.streams.state.internals.RocksDBSegmentedBytesStore.init(RocksDBSegmentedBytesStore.java:110)

...
08:09:19.673 [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] INFO  o.a.k.s.p.i.StreamThread - stream-thread [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] Shutting down

08:09:19.674 [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] INFO  o.a.k.c.p.KafkaProducer - Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.

08:09:19.681 [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] INFO  o.a.k.s.p.i.StreamThread - stream-thread [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] Removing all active tasks []

08:09:19.681 [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] INFO  o.a.k.s.p.i.StreamThread - stream-thread [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] Removing all standby tasks []

08:09:19.681 [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] INFO  o.a.k.s.p.i.StreamThread - stream-thread [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] Removing all standby tasks []

08:09:19.681 [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] INFO  o.a.k.s.p.i.StreamThread - stream-thread [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] Stream thread shutdown complete

08:09:19.681 [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] WARN  o.a.k.s.p.i.StreamThread - stream-thread [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] Unexpected state transition from ASSIGNING_PARTITIONS to DEAD.

08:09:19.681 [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] WARN  i.a.k.BaseEventsStream - uncaught exception in stream thread ComponentsActivityEventsStream

org.apache.kafka.streams.errors.StreamsException: stream-thread [ComponentsActivityEventsStream-608b9f05-0911-4b14-a1b1-37247747686a-StreamThread-2] Failed to rebalance.

    at org.apache.kafka.streams.processor.internals.StreamThread.pollRequests(StreamThread.java:589)

    at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:553)

    at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:527)

Caused by: org.apache.kafka.streams.errors.StreamsException: Store ComponentsActivityStore's change log (ComponentsActivityEventsStream-ComponentsActivityStore-changelog) does not contain partition 0

    at org.apache.kafka.streams.processor.internals.StoreChangelogReader.validatePartitionExists(StoreChangelogReader.java:87)

    at org.apache.kafka.streams.processor.internals.ProcessorStateManager.register(ProcessorStateManager.java:165)

Answer 1

检查您的 bootstrap-servers 列表中是否列出了所有三个经纪人。如果您只列出一个代理，而它恰好是那个发生故障的代理，那么消费者就无法获得它需要知道哪个节点是您主题中每个分区的领导者的元数据。如果它无法获得有效的元数据响应，则它不会连接到其他两个代理中的任何一个，即使它们具有数据的活动副本。

节点故障后卡夫卡消费者错误

Kafka consumer error after node failure

apache-kafka

apache-kafka-streams