如何在同一个 L2 网络上创建多个 Kafka 集群?
How to create multiple Kafka clusters on same L2 network?
我在同一个 L2 网络上有 12 个 Kafka 节点。
我想在同一个 L2 网络上创建 4 个 Kafka 集群(每个集群有 3 个节点)。
是否可以这样做以及如何做到?
如果不是,那么解决方法是什么?
Kafka 集群基本上由其成员使用的 Zookeeper 数据模型定义。所以,如果你想让一些代理属于一个给定的集群,你必须让它们指向同一个 Zookeeper 命名空间。实际上,如果你想拥有 4 个集群,那么你将需要在 Zookeeper 集合中使用 4 个不同的 chroot,并让每个集群使用不同的 chroot。例如,集群A的成员将使用zookeeper.connect=127.0.0.1:2181/cluster/a,集群B的成员将使用zookeeper.connect=127.0.0.1:2181/cluster/b 等等。当然,另一种选择是为每个集群使用完全独立的 Zookeeper 集成。
来自ZooKeeper Programmer's Guide
An optional "chroot" suffix may also be appended to the connection
string. This will run the client commands while interpreting all paths
relative to this root (similar to the unix chroot command). If used
the example would look like: "127.0.0.1:4545/app/a" or
"127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a" where the client
would be rooted at "/app/a" and all paths would be relative to this
root - ie getting/setting/etc... "/foo/bar" would result in operations
being run on "/app/a/foo/bar" (from the server perspective). This
feature is particularly useful in multi-tenant environments where each
user of a particular ZooKeeper service could be rooted differently.
This makes re-use much simpler as each user can code his/her
application as if it were rooted at "/", while actual location (say
/app/a) could be determined at deployment time.
我在同一个 L2 网络上有 12 个 Kafka 节点。
我想在同一个 L2 网络上创建 4 个 Kafka 集群(每个集群有 3 个节点)。
是否可以这样做以及如何做到? 如果不是,那么解决方法是什么?
Kafka 集群基本上由其成员使用的 Zookeeper 数据模型定义。所以,如果你想让一些代理属于一个给定的集群,你必须让它们指向同一个 Zookeeper 命名空间。实际上,如果你想拥有 4 个集群,那么你将需要在 Zookeeper 集合中使用 4 个不同的 chroot,并让每个集群使用不同的 chroot。例如,集群A的成员将使用zookeeper.connect=127.0.0.1:2181/cluster/a,集群B的成员将使用zookeeper.connect=127.0.0.1:2181/cluster/b 等等。当然,另一种选择是为每个集群使用完全独立的 Zookeeper 集成。
来自ZooKeeper Programmer's Guide
An optional "chroot" suffix may also be appended to the connection string. This will run the client commands while interpreting all paths relative to this root (similar to the unix chroot command). If used the example would look like: "127.0.0.1:4545/app/a" or "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a" where the client would be rooted at "/app/a" and all paths would be relative to this root - ie getting/setting/etc... "/foo/bar" would result in operations being run on "/app/a/foo/bar" (from the server perspective). This feature is particularly useful in multi-tenant environments where each user of a particular ZooKeeper service could be rooted differently. This makes re-use much simpler as each user can code his/her application as if it were rooted at "/", while actual location (say /app/a) could be determined at deployment time.