kafka connection error: Failed to resolve, Connection refused

Question

Zookeeper 和 kafka 运行正在使用

docker compose up -d

---
version: '2'

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:6.1.0
    hostname: zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  broker:
    image: confluentinc/cp-kafka:6.1.0
    hostname: broker
    container_name: broker
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_ADVERTISED_HOST_NAME: broker
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:9092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0

我在本地有一个生产者文件运行ning p = Producer({'bootstrap.servers': 'localhost:9092'})

消费者将通过 docker

运行宁

FROM python:3.9.5-alpine3.14

RUN apk update \
    && /sbin/apk add --no-cache --virtual .deps gcc musl-dev \
    && /usr/local/bin/pip install --no-cache-dir black==19.10b0 \
    && /sbin/apk del --no-cache .deps \
    && apk add --no-cache git \
    && apk add --update py-pip \
    && apk add postgresql-dev gcc python3-dev musl-dev \
    && apk add linux-headers \
    && apk add --no-cache cargo python3-dev openssl-dev libffi-dev g++ make \
    && apk add --no-cache librdkafka-dev

RUN apk add --no-cache jpeg-dev zlib-dev
RUN apk add --no-cache --virtual .build-deps build-base linux-headers \
    && pip install Pillow
    
COPY /requirements.txt /app/requirements.txt

WORKDIR /app

RUN pip3 install -r requirements.txt

COPY . /app

CMD [ "python3", "consumer.py" ]

consumer.py 有代码：

c = Consumer({
    'bootstrap.servers': 'broker:9092',
    'group.id': 'consumer-group-a',
    'auto.offset.reset': 'earliest'
})

我尽量不编辑主机文件。 Failed to resolve 'broker:9092': Name or service not known 和 Connection refused。我在尝试运行时主要遇到了这两个错误。任何帮助将不胜感激。

谢谢。

Answer 1

您需要将 Python 服务放在同一个组合文件中，这样它就属于可以解析 broker DNS 名称的同一个 Docker 网络。

例如

services:
  app:
    build: 
      context: ./app/  # folder with Dockerfile
    environment:
      BOOTSTRAP_SERVERS: broker:9092  # don't hardcode this in your code

  zookeeper:
     ...
  broker:
     ...

注意：您的消费者可能会在 Kafka 实际运行之前尝试连接，因此您应该增加一些睡眠时间。

否则，使用docker network ls，找到托管Kafka和Zookeper的网络，然后使用docker run --network <name> your-python-app

kafka connection error: Failed to resolve, Connection refused

kafka connection error: Failed to resolve, Connection refused

python

apache-kafka

docker-compose