只有在每个分区的副本上写入成功时,写入 Kafka 主题才会成功吗?
Is a write to a Kafka topic only successful if write on each partition's replicas is successful?
是否只有在每个分区的副本上写入成功时,写入 Kafka 主题才会成功?或者是否有可以配置的法定人数?
假设你有
- 制作人
- 带有主题 1 的服务器 1 分区 1(领导者)
- 带有主题 1 的服务器 2 分区 1(副本)
- 带有主题 1 的服务器 3 分区 1(副本)
生产者写入主题 1。是否仅当从领导者和两个副本收到确认时才存储消息?或者是否可以配置法定人数:只有领导者?
如果我对你的问题的理解正确,你正在寻找名为 acks
的 Producer API 的配置。
acks: The number of acknowledgments the producer requires the leader to have received before considering a request complete. This controls the durability of records that are sent. The following settings are allowed:
acks=0 If set to zero then the producer will not wait for any acknowledgment from the server at all. The record will be immediately added to the socket buffer and considered sent. No guarantee can be made that the server has received the record in this case, and the retries configuration will not take effect (as the client won't generally know of any failures). The offset given back for each record will always be set to -1.
acks=1 This will mean the leader will write the record to its local log but will respond without awaiting full acknowledgement from all followers. In this case should the leader fail immediately after acknowledging the record but before the followers have replicated it then the record will be lost.
acks=all This means the leader will wait for the full set of in-sync replicas to acknowledge the record. This guarantees that the record will not be lost as long as at least one in-sync replica remains alive. This is the strongest available guarantee. This is equivalent to the acks=-1 setting.
Type: stringDefault: 1Valid Values: [all, -1, 0, 1]Importance: high
检查 documentation 制作人 API 了解更多详情。
除了 acks 之外,还有一个问题是消息何时对消费者可见 - 这与生产者的 acks 无关并且取决于代理配置 - 当所有关注者都同步收到消息时会发生这种情况.如果有人落后了,那么就是 min.insync.replicas 收到消息的时候。我已经在 https://chrisg23.blogspot.com/2020/02/kafka-acks-configuration-apology.html?m=1
上写了一些关于这个的文章
是否只有在每个分区的副本上写入成功时,写入 Kafka 主题才会成功?或者是否有可以配置的法定人数?
假设你有
- 制作人
- 带有主题 1 的服务器 1 分区 1(领导者)
- 带有主题 1 的服务器 2 分区 1(副本)
- 带有主题 1 的服务器 3 分区 1(副本)
生产者写入主题 1。是否仅当从领导者和两个副本收到确认时才存储消息?或者是否可以配置法定人数:只有领导者?
如果我对你的问题的理解正确,你正在寻找名为 acks
的 Producer API 的配置。
acks: The number of acknowledgments the producer requires the leader to have received before considering a request complete. This controls the durability of records that are sent. The following settings are allowed:
acks=0 If set to zero then the producer will not wait for any acknowledgment from the server at all. The record will be immediately added to the socket buffer and considered sent. No guarantee can be made that the server has received the record in this case, and the retries configuration will not take effect (as the client won't generally know of any failures). The offset given back for each record will always be set to -1.
acks=1 This will mean the leader will write the record to its local log but will respond without awaiting full acknowledgement from all followers. In this case should the leader fail immediately after acknowledging the record but before the followers have replicated it then the record will be lost.
acks=all This means the leader will wait for the full set of in-sync replicas to acknowledge the record. This guarantees that the record will not be lost as long as at least one in-sync replica remains alive. This is the strongest available guarantee. This is equivalent to the acks=-1 setting.
Type: stringDefault: 1Valid Values: [all, -1, 0, 1]Importance: high
检查 documentation 制作人 API 了解更多详情。
除了 acks 之外,还有一个问题是消息何时对消费者可见 - 这与生产者的 acks 无关并且取决于代理配置 - 当所有关注者都同步收到消息时会发生这种情况.如果有人落后了,那么就是 min.insync.replicas 收到消息的时候。我已经在 https://chrisg23.blogspot.com/2020/02/kafka-acks-configuration-apology.html?m=1
上写了一些关于这个的文章