Kafka KStream - 测量消费者滞后
Kafka KStream - measuring consumer lag
由于我的基于 KStream 的应用程序不遵循传统的 Kafka 消费者路线,我应该如何跟踪消费者延迟?通常我会使用 ConsumerOffsetChecker (或类似的东西),但它需要一个消费者组名称。
我应该改用什么?
(我想跟踪这个,以便我可以判断是否/何时推出新消费者)
Kafka Streams 在内部利用 KafkaConsumer
,并使用应用程序 ID 作为消费者组 ID。因此,您可以像监控任何其他消费者一样监控延迟。
见http://docs.confluent.io/current/streams/developer-guide.html#required-configuration-parameters
Application Id (application.id): Each stream processing application must have a unique id. [...]
This id is used in the following places to isolate resources used by the application from others:
- As the default Kafka consumer and producer client.id prefix
- As the Kafka consumer group.id for coordination
- As the name of the sub-directory in the state directory (cf. state.dir)
- As the prefix of internal Kafka topic names
由于我的基于 KStream 的应用程序不遵循传统的 Kafka 消费者路线,我应该如何跟踪消费者延迟?通常我会使用 ConsumerOffsetChecker (或类似的东西),但它需要一个消费者组名称。
我应该改用什么?
(我想跟踪这个,以便我可以判断是否/何时推出新消费者)
Kafka Streams 在内部利用 KafkaConsumer
,并使用应用程序 ID 作为消费者组 ID。因此,您可以像监控任何其他消费者一样监控延迟。
见http://docs.confluent.io/current/streams/developer-guide.html#required-configuration-parameters
Application Id (application.id): Each stream processing application must have a unique id. [...]
This id is used in the following places to isolate resources used by the application from others:
- As the default Kafka consumer and producer client.id prefix
- As the Kafka consumer group.id for coordination
- As the name of the sub-directory in the state directory (cf. state.dir)
- As the prefix of internal Kafka topic names