卡夫卡消费者断开连接后重新连接

Question

我使用 Spring Boot 和 Spring Kafka 设置了我的项目，并且有三个消费者。查看日志，我可以看到消费者不时断开连接：

catalina.out:2019-04-27 02:19:57.962  INFO 18245 --- [ntainer#2-0-C-1] o.a.kafka.clients.FetchSessionHandler    : [Consumer clientId=consumer-2, groupId=FalconDataRiver1] Error sending fetch request (sessionId=1338157432, epoch=205630) to node 101: org.apache.kafka.common.errors.DisconnectException.
catalina.out:2019-04-27 02:19:57.962  INFO 18245 --- [ntainer#4-0-C-1] o.a.kafka.clients.FetchSessionHandler    : [Consumer clientId=consumer-6, groupId=FalconDataRiver1] Error sending fetch request (sessionId=727942178, epoch=234691) to node 101: org.apache.kafka.common.errors.DisconnectException.
catalina.out:2019-04-27 02:19:57.962  INFO 18245 --- [ntainer#0-0-C-1] o.a.kafka.clients.FetchSessionHandler    : [Consumer clientId=consumer-10, groupId=FalconDataRiver1] Error sending fetch request (sessionId=836405004, epoch=234351) to node 101: org.apache.kafka.common.errors.DisconnectException.
catalina.out:2019-04-27 02:19:58.023  INFO 18245 --- [ntainer#1-0-C-1] o.a.kafka.clients.FetchSessionHandler    : [Consumer clientId=consumer-12, groupId=FalconDataRiver1] Error sending fetch request (sessionId=1385585601, epoch=234394) to node 101: org.apache.kafka.common.errors.DisconnectException.
catalina.out:2019-04-27 02:19:58.023  INFO 18245 --- [ntainer#3-0-C-1] o.a.kafka.clients.FetchSessionHandler    : [Consumer clientId=consumer-4, groupId=FalconDataRiver1] Error sending fetch request (sessionId=452630289, epoch=201944) to node 101: org.apache.kafka.common.errors.DisconnectException.
catalina.out:2019-04-27 02:19:58.023  INFO 18245 --- [ntainer#5-0-C-1] o.a.kafka.clients.FetchSessionHandler    : [Consumer clientId=consumer-8, groupId=FalconDataRiver1] Error sending fetch request (sessionId=78802572, epoch=103) to node 101: org.apache.kafka.common.errors.DisconnectException.
catalina.out:2019-04-27 02:19:58.040  INFO 18245 --- [ntainer#2-0-C-1] o.a.kafka.clients.FetchSessionHandler    : [Consumer clientId=consumer-2, groupId=FalconDataRiver1] Error sending fetch request (sessionId=1338157432, epoch=INITIAL) to node 101: org.apache.kafka.common.errors.DisconnectException.

我没有在重新连接方面配置消费者。我知道 Kafka 文档中有两个属性：

reconnect.backoff.ms -- The maximum amount of time in milliseconds to wait when reconnecting to a broker that has repeatedly failed to connect. If provided, the backoff per host will increase exponentially for each consecutive connection failure, up to this maximum. After calculating the backoff increase, 20% random jitter is added to avoid connection storms. Default value 1000 milliseconds)

reconnect.backoff.ms -- The base amount of time to wait before attempting to reconnect to a given host. This avoids repeatedly connecting to a host in a tight loop. This backoff applies to all connection attempts by the client to a broker. Default value 50 milliseconds)

我可以看到三个消费者在上面的日志消息之后还在消费。显然他们已经从这些断开连接异常中恢复过来。困扰我的是，日志中没有任何记录重连和恢复的过程。

我是不是漏掉了什么？谢谢！

Answer 1

Kafka 会自动从这个内部错误中恢复，这就是日志级别为 INFO 的原因。显然，您的消费者仍然能够使用这些消息。

将日志级别切换到 DEBUG，以防您想获得有关导致此问题的原因的更多信息。

卡夫卡消费者断开连接后重新连接

Kafka consumer reconnection after getting disconnected

apache-kafka

kafka-consumer-api

spring-kafka