方法内的多处理回调不起作用

Question

我正在尝试运行在一个方法中进行多处理。但是回调似乎没有正确运行。

这是我需要实现的示例。

from multiprocessing import Pool
from time import sleep
import random

def sum(task, a, b): 
    sleepTime = random.randint(1, 4)
    print(task, " requires ", sleepTime, " seconds to finish")
    sleep(sleepTime)
    return a+b 

def printResult(result):
    print(result)

myPool = Pool(5)

result1 = myPool.apply_async(sum, args=("task1", 10, 20,), callback = printResult)
result2 = myPool.apply_async(sum, args=("task2", 20, 30,), callback = printResult)
result3 = myPool.apply_async(sum, args=("task3", 30, 40,), callback = printResult)
result4 = myPool.apply_async(sum, args=("task4", 40, 50,), callback = printResult)
result5 = myPool.apply_async(sum, args=("task5", 50, 60,), callback = printResult)

print("Submitted tasks to pool")

myPool.close()
myPool.join()

返回时效果很好

Submitted tasks to pool
task1  requires  3  seconds to finish
task2  requires  4  seconds to finish
task3  requires  1  seconds to finish
task4  requires  4  seconds to finish
task5  requires  1  seconds to finish
110
70
30
50
90

然而，当我将多处理包装在一个方法中并调用该方法时，它似乎并没有像预期的那样运行。

例如

def run_mp():
    from multiprocessing import Pool
    from time import sleep
    import random

    def sum(task, a, b): 
        sleepTime = random.randint(1, 4)
        print(task, " requires ", sleepTime, " seconds to finish")
        sleep(sleepTime)
        return a+b 

    def printResult(result):
        print(result)

    myPool = Pool(5)

    result1 = myPool.apply_async(sum, args=("task1", 10, 20,), callback = printResult)
    result2 = myPool.apply_async(sum, args=("task2", 20, 30,), callback = printResult)
    result3 = myPool.apply_async(sum, args=("task3", 30, 40,), callback = printResult)
    result4 = myPool.apply_async(sum, args=("task4", 40, 50,), callback = printResult)
    result5 = myPool.apply_async(sum, args=("task5", 50, 60,), callback = printResult)

    print("Submitted tasks to pool")

    myPool.close()
    myPool.join()

当调用run_mp()时，它只是returns下面的内容。

Submitted tasks to pool

这不仅仅是在屏幕上打印标准输出的问题。在我处理的实际问题中，似乎没有正确调用回调函数。

Answer 1

简单的回答。用 multiprocessing 来腌制 class 方法是不可能的，或者至少是不容易的。有一个 multiprocessing 的分支，称为 multiprocess（我是作者），它使用 dill 序列化程序...并实现您正在寻找的行为类型。

只需将 from multiprocessing 替换为 from multiprocess，您的代码就会按预期运行。

方法内的多处理回调不起作用

multiprocessing callback inside a method does not work

python

multiprocessing

multiprocess

python-3.x

python-multiprocessing