Kafka KStream - 测量消费者滞后

Kafka KStream - measuring consumer lag

由于我的基于 KStream 的应用程序不遵循传统的 Kafka 消费者路线,我应该如何跟踪消费者延迟?通常我会使用 ConsumerOffsetChecker (或类似的东西),但它需要一个消费者组名称。

我应该改用什么?

(我想跟踪这个,以便我可以判断是否/何时推出新消费者)

Kafka Streams 在内部利用 KafkaConsumer,并使用应用程序 ID 作为消费者组 ID。因此,您可以像监控任何其他消费者一样监控延迟。

http://docs.confluent.io/current/streams/developer-guide.html#required-configuration-parameters

Application Id (application.id): Each stream processing application must have a unique id. [...]

This id is used in the following places to isolate resources used by the application from others:

  • As the default Kafka consumer and producer client.id prefix
  • As the Kafka consumer group.id for coordination
  • As the name of the sub-directory in the state directory (cf. state.dir)
  • As the prefix of internal Kafka topic names