如果我将自动提交设置为 true 和

What's the behavior if I have set the auto commit to true and

我正在 https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/connectors/datastream/kafka/#kafka-consumers-offset-committing-behaviour-configuration

读书

它说:

Checkpointing enabled: if checkpointing is enabled, the Flink Kafka Consumer will commit the offsets stored in the checkpointed states when the checkpoints are completed. This ensures that the committed offsets in Kafka brokers is consistent with the offsets in the checkpointed states. Users can choose to disable or enable offset committing by calling the setCommitOffsetsOnCheckpoints(boolean) method on the consumer (by default, the behaviour is true). Note that in this scenario, the automatic periodic offset committing settings in Properties is completely ignored.

如果我以 10 秒的间隔启用检查点,我也会将 have setCommitOffsetsOnCheckpoints 设置为 true,并在 Kafka 消费者属性中设置 enable.auto.commit=trueauto.commit.interval.ms=5000

那么偏移量提交会有什么表现呢?偏移量会每 10 秒提交 3 次吗?一次来自 flink 做检查点时,两次来自 Kafka 消费者的自动提交?

enable.auto.commit:Automatic offset submission, the configuration of this value is not the final offset submission mode, you need to consider whether the user has enabled checkpoint, Will be interpreted in the following source code analysis

consumer.setCommitOffsetsOnCheckpoints(true) ​ Explanation: After setting the checkpoint, submit the offset, that is, the oncheckpoint mode. The value is true by default. This parameter will affect the submission method of the offset. The following source code will analyze it

因此,根据您所说的引用并在互联网上进一步查看,Flink 消费者将在启用检查点时忽略 auto.commit 配置,偏移量提交行为将定义为 consumer.setCommitOffsetsOnCheckpoints 已定义(默认为 true

https://www.programmersought.com/article/84744879064/

来自 Flink 文档

Checkpointing disabled: if checkpointing is disabled, the Flink Kafka Consumer relies on the automatic periodic offset committing capability of the internally used Kafka clients. Therefore, to disable or enable offset committing, simply set the enable.auto.commit / auto.commit.interval.ms keys to appropriate values in the provided Properties configuration.

Checkpointing enabled: if checkpointing is enabled, the Flink Kafka Consumer will commit the offsets stored in the checkpointed states when the checkpoints are completed. This ensures that the committed offsets in Kafka brokers is consistent with the offsets in the checkpointed states. Users can choose to disable or enable offset committing by calling the setCommitOffsetsOnCheckpoints(boolean) method on the consumer (by default, the behaviour is true). Note that in this scenario, the automatic periodic offset committing settings in Properties is completely ignored.

请注意,在这种情况下,Properties 中的自动定期偏移提交设置将被完全忽略