确定 Kafka-Client 与 kafka-broker 的兼容性

Question

在远程 kafka 云集群中，kafka 代理将更新到新版本 (5.1)，因此将应用新的 kafka 协议。

现在我应该更新我的 kafka 客户端以能够连接。现在我在我的 spring-boot 应用程序中使用以下 kafka 相关依赖项：

<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-parent</artifactId>
<version>1.3.8.RELEASE</version>

<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-kafka</artifactId>
<version>2.0.1.RELEASE</version>

<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka-test</artifactId>
<version>1.0.3.RELEASE</version>

<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
<version>1.0.3.RELEASE</version>

<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>0.9.0.1</version>

<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.11</artifactId>
<version>0.9.0.1</version>

我已经为 kafka-clients 和 kafka-brokers 搜索了合适的兼容性矩阵。

有没有人知道如何成功升级？
有人有此依赖项的兼容性矩阵吗？

Answer 1

根据文档：

Previously, the general rule was that the Kafka broker a client interacted with was required to be a newer version than the client. Now, newer versions of the Java clients and other clients that support KIP-35 such as librdkafka are able to fall back to older request types or throw appropriate errors if functionality is not available.

confluent matrix on the Spring for Apache Kafka project page 有一个 link（以及 spring-kafka/kafka-clients 兼容性）。

Answer 2

来自https://github.com/spring-cloud/spring-cloud-stream/wiki/Kafka-Client-Compatibility

兼容性矩阵

+--------+--------------+------------------+---------------+-------------+
| Spring |  Spring for  |      Spring      | kafka-clients |    Kafka    |
|  Cloud | Apache Kafka |    Integration   |               |    Broker   |
| Stream |              | for Apache Kafka |               |             |
+--------+--------------+------------------+---------------+-------------+
| 2.1.x  | 2.2.x        | 3.1.x            | 2.0.0,        | 2.0.0,      |
|        |              |                  | 1.1.x,        | 1.1.x,      |
|        |              |                  | 1.0.x(*)      | 1.0.x,      |
|        |              |                  |               | 0.11.0.x(*) |
+--------+--------------+------------------+---------------+-------------+
| 2.0.x  | 2.1.x        | 3.0.x            | 1.1.x,        | 1.1.x,      |
|        |              |                  | 1.0.x(*)      | 1.0.x,      |
|        |              |                  |               | 0.11.0.x(*) |
+--------+--------------+------------------+---------------+-------------+
| 1.3.x  | 1.3.x,       | 2.3.x,           | 0.11.0.x(**), | 0.10.x.x    |
|        | 1.2.x,       | 2.2.x,           | 0.10.2.x      | or higher   |
|        | 1.1.x        | 2.1.x            |               |             |
+--------+--------------+------------------+---------------+-------------+
| 1.2.x  | 1.2.x,       | 2.2.x,           | 0.10.1.x      | 0.10.x.x    |
|        | 1.1.x        | 2.1.x            |               | or higher   |
+--------+--------------+------------------+---------------+-------------+

其他来源

Answer 3

现在支持双向客户端兼容性，您无需再担心兼容性矩阵，对于启用 KIP-35 的客户端，任何版本都很好，KIP-35 从 Broker 协议发布 - 0.10.0， Java 客户 - 0.10.2

参考：

https://cwiki.apache.org/confluence/display/KAFKA/Compatibility+Matrix

https://www.confluent.io/blog/upgrading-apache-kafka-clients-just-got-easier/

确定 Kafka-Client 与 kafka-broker 的兼容性

Determine the Kafka-Client compatibility with kafka-broker

java

apache-kafka

spring-kafka

兼容性矩阵

其他来源