ZeroMQ 在 python 多处理 class/object 解决方案中挂起

Question

我正在尝试在 Python (pyzmq) 中将 ZeroMQ 与多处理一起使用。作为一个最小的（不是）工作示例，我有一个服务器-和一个客户端-class，它们都继承自 multiprocessing.Process。作为子进程的客户端应向服务器子进程发送消息，服务器子进程应打印消息：

#mpzmq_class.py

from multiprocessing import Process
import zmq


class Server(Process):
    def __init__(self):
        super(Server, self).__init__()
        self.ctx = zmq.Context()
        self.socket = self.ctx.socket(zmq.PULL)
        self.socket.connect("tcp://localhost:6068")

    def run(self):
        msg = self.socket.recv_string()
        print(msg)


class Client(Process):
    def __init__(self):
        super(Client, self).__init__()
        self.ctx = zmq.Context()
        self.socket = self.ctx.socket(zmq.PUSH)
        self.socket.bind("tcp://*:6068")

    def run(self):
        msg = "Hello World!"
        self.socket.send_string(msg)

if __name__ == "__main__":
    s = Server()
    c = Client()
    s.start()
    c.start()
    s.join()
    c.join()

现在，如果我运行服务器进程似乎在接收呼叫时挂起 msg = socket.receive_string()。在另一个（更复杂的）案例中，它甚至挂在 socket.connect("...")-statement.

如果我重写脚本以使用函数而不是 classes/objects，它运行就好了：

# mpzmq_function.py

from multiprocessing import Process
import zmq


def server():
    ctx = zmq.Context()
    socket = ctx.socket(zmq.PULL)
    socket.connect("tcp://localhost:6068")
    msg = socket.recv_string()
    print(msg)


def client():
    ctx = zmq.Context()
    socket = ctx.socket(zmq.PUSH)
    socket.bind("tcp://*:6068")
    msg = "Hello World!"
    socket.send_string(msg)

if __name__ == "__main__":
    s = Process(target=server)
    c = Process(target=client)
    s.start()
    c.start()
    s.join()
    c.join()

输出：

paul@AP-X:~$ python3 mpzmq_function.py 
Hello World!

有人可以帮我解决这个问题吗？我想这是我对多处理的使用不了解的地方。

谢谢！

Answer 1

我运行遇到了同样的问题。我想问题是运行方法无法访问上下文对象。也许它与 C 实现有关，事实上，进程没有共享内存。如果在运行方法中实例化上下文，它就可以工作。

这是一个工作示例：

#mpzmq_class.py

from multiprocessing import Process
import zmq


class Base(Process):
    """
    Inherit from Process and
    holds the zmq address.
    """
    def __init__(self, address):
        super().__init__()
        self.address = address


class Server(Base):
    def run(self):
        ctx = zmq.Context()
        socket = ctx.socket(zmq.PULL)
        socket.connect(self.address)
        msg = socket.recv_string()
        print(msg)


class Client(Base):
    def run(self):
        ctx = zmq.Context()
        socket = ctx.socket(zmq.PUSH)
        socket.bind(self.address)
        msg = "Hello World!"
        socket.send_string(msg)


if __name__ == "__main__":
    server_addr = "tcp://127.0.1:6068"
    client_addr = "tcp://*:6068"
    s = Server(server_addr)
    c = Client(client_addr)
    s.start()
    c.start()
    s.join()
    c.join()

我添加了一个基础 class 来证明您仍然可以从运行方法访问普通的 Python 对象。如果将上下文对象放入 init 方法中，它将不起作用。

ZeroMQ 在 python 多处理 class/object 解决方案中挂起

ZeroMQ hangs in a python multiprocessing class/object solution

multiprocessing

zeromq

python-3.x

pyzmq