异步 + 多处理 + unix

Question

我有一个具有以下逻辑的宠物项目：

import asyncio, multiprocessing

async def sub_main():
    print('Hello from subprocess')

def sub_loop():
    asyncio.get_event_loop().run_until_complete(sub_main())

def start():
    multiprocessing.Process(target=sub_loop).start()

start()

如果你运行它，你会看到：

Hello from subprocess

很好。但我要做的是改为 start() 协程：

async def start():
    multiprocessing.Process(target=sub_loop).start()

为了运行它，我必须做这样的事情：

asyncio.get_event_loop().run_until_complete(start())

这里是问题所在：创建子进程时，它会克隆整个 Python 环境，因此事件循环已经运行在那里：

Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "test.py", line 7, in sub_loop
    asyncio.get_event_loop().run_until_complete(sub_main())
  File "/usr/lib/python3.5/asyncio/base_events.py", line 361, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.5/asyncio/base_events.py", line 326, in run_forever
    raise RuntimeError('Event loop is running.')
RuntimeError: Event loop is running.

我试图在子进程端销毁它，但没有成功，但我认为正确的方法是防止它与子进程共享。有可能吗？

更新：这是完整的失败代码：

import asyncio, multiprocessing

import asyncio.unix_events

async def sub_main():
    print('Hello from subprocess')

def sub_loop():
    asyncio.get_event_loop().run_until_complete(sub_main())


async def start():
    multiprocessing.Process(target=sub_loop).start()

asyncio.get_event_loop().run_until_complete(start())

Answer 1

您应该始终添加一个检查以查看您如何运行代码（if __name__ == '__main__': 部分。您的子进程第二次是运行模块中的所有内容，给出你悲伤（无法抗拒）。

import asyncio, multiprocessing

async def sub_main():
    print('Hello from subprocess')

def sub_loop():
    asyncio.get_event_loop().run_until_complete(sub_main())


async def start():
    multiprocessing.Process(target=sub_loop).start()

if __name__ == '__main__':
    asyncio.get_event_loop().run_until_complete(start())

Answer 2

首先，你应该考虑使用loop.run_in_executor with a ProcessPoolExecutor if you plan to run python subprocesses from within the loop. As for your problem, you can use the event loop policy函数来设置一个新的循环：

import asyncio
from concurrent.futures import ProcessPoolExecutor

async def sub_main():
    print('Hello from subprocess')

def sub_loop():
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    loop.run_until_complete(sub_main())

async def start(executor):
    await asyncio.get_event_loop().run_in_executor(executor, sub_loop)

if __name__ == '__main__':
    executor = ProcessPoolExecutor()
    asyncio.get_event_loop().run_until_complete(start(executor))

异步 + 多处理 + unix

asyncio + multiprocessing + unix

python

unix

python-asyncio

python-multiprocessing

python-3.5