Kafka consumer fetch-min-size (fetch.min.bytes) 是否等待上述大小被填充?
Does Kafka consumer fetch-min-size (fetch.min.bytes) wait for the mentioned size to get filled?
假设有107条记录,每条记录1kb。如果获取大小为 15kb,则在 7 次迭代中将消耗 105kb。现在,只剩下 2kb,我会在下一次迭代中获得剩余的 2 条记录,还是会等待更多的 15kb 被累积?假设在此之后没有记录剩余。
它一直等到达到fetch.max.wait.ms
中定义的时间。默认情况下,此值设置为 500。您可以在 Consumer
上的 Kafka 文档中找到两个相关配置的描述
fetch.min.bytes
The minimum amount of data the server should return for a fetch request. If insufficient data is available the request will wait for that much data to accumulate before answering the request. The default setting of 1 byte means that fetch requests are answered as soon as a single byte of data is available or the fetch request times out waiting for data to arrive. Setting this to something greater than 1 will cause the server to wait for larger amounts of data to accumulate which can improve server throughput a bit at the cost of some additional latency.
fetch.max.wait.ms
The maximum amount of time the server will block before answering the fetch request if there isn't sufficient data to immediately satisfy the requirement given by fetch.min.bytes.
假设有107条记录,每条记录1kb。如果获取大小为 15kb,则在 7 次迭代中将消耗 105kb。现在,只剩下 2kb,我会在下一次迭代中获得剩余的 2 条记录,还是会等待更多的 15kb 被累积?假设在此之后没有记录剩余。
它一直等到达到fetch.max.wait.ms
中定义的时间。默认情况下,此值设置为 500。您可以在 Consumer
fetch.min.bytes
The minimum amount of data the server should return for a fetch request. If insufficient data is available the request will wait for that much data to accumulate before answering the request. The default setting of 1 byte means that fetch requests are answered as soon as a single byte of data is available or the fetch request times out waiting for data to arrive. Setting this to something greater than 1 will cause the server to wait for larger amounts of data to accumulate which can improve server throughput a bit at the cost of some additional latency.
fetch.max.wait.ms
The maximum amount of time the server will block before answering the fetch request if there isn't sufficient data to immediately satisfy the requirement given by fetch.min.bytes.