使用 Kubernetes 中的 StatefulSet 对象查找没有静态 IP 的 kafka 代理时出错

Question

我正在尝试使用 confluentinc docker images. I based my solution on this and this post 在 Kubernetes 上部署 Zookeeper 和 Kafka。 Zookeeper 运行日志中没有错误。我想使用 StatefulSet 部署 3 个 Kafka 代理。我的 yaml 文件的问题是我不知道如何在使用 3 个代理时为 Kafka 配置 KAFKA_ADVERTISED_LISTENERS 属性。

这里是 zookeeper 的 yaml 文件：

apiVersion: v1
kind: Service
metadata:
  name: zookeeper
  labels:
    app: zookeeper
spec:
  clusterIP: None
  ports:
  - name: client
    port: 2181
    protocol: TCP
  - name: follower
    port: 2888
    protocol: TCP
  - name: leader
    port: 3888
    protocol: TCP
  selector:
    app: zookeeper
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: zookeeper
spec:
  replicas: 1
  serviceName: zookeeper
  selector:
    matchLabels:
      app: zookeeper # has to match .spec.template.metadata.labels
  template:
    metadata:
      labels:
        app: zookeeper # has to match .spec.selector.matchLabels
    spec:
      hostname: zookeeper
      containers:
      - name: zookeeper
        image: confluentinc/cp-zookeeper:5.5.0
        ports:
        - containerPort: 2181
        env:
        - name: ZOOKEEPER_CLIENT_PORT
          value: "2181"
        - name: ZOOKEEPER_ID
          value: "1"
        - name: ZOOKEEPER_SERVER_1
          value: zookeeper

对于卡夫卡经纪人：

apiVersion: v1
kind: Service
metadata:
  name: kafka-service
  labels:
    app: kafka
spec:
  type: LoadBalancer
  ports:
  - port: 9092
    name: kafka-port
    protocol: TCP
  selector:
    app: kafka
    id: "0"
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: kafka
spec:
  replicas: 3
  serviceName: kafka
  podManagementPolicy: OrderedReady
  selector:
    matchLabels:
      app: kafka # has to match .spec.template.metadata.labels
  template:
    metadata:
      labels:
        app: kafka # has to match .spec.selector.matchLabels
    spec:
      containers:
      - name: kafka
        image: confluentinc/cp-enterprise-kafka:5.5.0
        ports:
        - containerPort: 9092
        env:
        - name: KAFKA_ZOOKEEPER_CONNECT
          value: zookeeper:2181 # zookeeper-2.zookeeper.default.svc.cluster.local
        - name: KAFKA_ADVERTISED_LISTENERS
          value: "LISTENER_0://kafka-0:9092,LISTENER_1://kafka-1:9093,LISTENER_2://kafka-2:9094"
        - name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
          value: "LISTENER_0:PLAINTEXT,LISTENER_1:PLAINTEXT,LISTENER_2:PLAINTEXT"
        - name: KAFKA_INTER_BROKER_LISTENER_NAME
          value: LISTENER_0

我得到 3 kafka pods 运行，kafka-0 正在连接，但 kafka-1 和 kafka-2 没有连接。

$ kubectl get pods -o wide
NAME          READY   STATUS             RESTARTS   AGE     IP           NODE       NOMINATED NODE   READINESS GATES
kafka-0       1/1     Running            0          4m12s   172.17.0.4   minikube   <none>           <none>
kafka-1       1/1     Running            5          4m9s    172.17.0.5   minikube   <none>           <none>
kafka-2       0/1     CrashLoopBackOff   4          4m7s    172.17.0.6   minikube   <none>           <none>
zookeeper-0   1/1     Running            0          21m     172.17.0.3   minikube   <none>           <none>

错误是说我已经在第一个广告连播 kafka-0 中发布了 kafka-0:9092,kafka-1:9093,kafka-2:9094。所以，我想它必须是动态的。如何配置？

[2020-09-30 14:56:40,519] ERROR [KafkaServer id=1017] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.IllegalArgumentException: requirement failed: Configured end points kafka-0:9092,kafka-1:9093,kafka-2:9094 in advertised listeners are already registered by broker 1012
    at kafka.server.KafkaServer.$anonfun$createBrokerInfo(KafkaServer.scala:436)
    at kafka.server.KafkaServer.$anonfun$createBrokerInfo$adapted(KafkaServer.scala:434)
    at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
    at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
    at kafka.server.KafkaServer.createBrokerInfo(KafkaServer.scala:434)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:293)
    at io.confluent.support.metrics.SupportedServerStartable.startup(SupportedServerStartable.java:114)
    at io.confluent.support.metrics.SupportedKafka.main(SupportedKafka.java:66)

Answer 1

我一直在阅读此博客 post "Kafka Listeners - Explained" 并且我能够使用以下配置配置 3 个 Kafka 代理。

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: kafka
spec:
  replicas: 3
  serviceName: kafka
  podManagementPolicy: OrderedReady
  selector:
    matchLabels:
      app: kafka # has to match .spec.template.metadata.labels
  template:
    metadata:
      labels:
        app: kafka # has to match .spec.selector.matchLabels
    spec:
      restartPolicy: Always
      containers:
      - name: kafka
        image: confluentinc/cp-enterprise-kafka:5.5.0
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits: # limit of 0.5 cpu and 512MiB of memory
            memory: "512Mi"
            cpu: "500m"
        # imagePullPolicy: Always
        ports:
        - containerPort: 9092
          name: kafka-0
        - containerPort: 9093
          name: kafka-1
        - containerPort: 9094
          name: kafka-2
        env:
        - name: MY_METADATA_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: MY_POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: STAS_DELAY
          value: "120"
        - name: KAFKA_ZOOKEEPER_CONNECT
          value: zookeeper:2181 # zookeeper-2.zookeeper.default.svc.cluster.local
        - name: KAFKA_ADVERTISED_LISTENERS
          value: "INSIDE://$(MY_POD_IP):9092"
        - name: KAFKA_LISTENERS
          value: "INSIDE://$(MY_POD_IP):9092"
        - name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
          value: "INSIDE:PLAINTEXT"
        - name: KAFKA_INTER_BROKER_LISTENER_NAME
          value: "INSIDE"

使用 Kubernetes 中的 StatefulSet 对象查找没有静态 IP 的 kafka 代理时出错

Error when using StatefulSet object from Kubernetes to find the kafka broker without a static IP

apache-kafka

kubernetes

confluent-platform