在 Python 中并行处理不同参数的函数

Question

这是我想要并行运行 printRange() 的简单代码：

def printRange(lrange):
    print ("First is " + str(lrange[0]) + " and last is " + str(lrange[1]))


def runInParallel():
    ranges = [[0, 10], [10, 20], [20, 30]]
    // Call printRange in parallel with each sublist of ranges given as argument

我的问题与这里的不同，每个进程都是硬编码的，启动并最终加入。我想运行 printRange() 与其他 100 个 printRange() 工作函数并行。每次都硬编码是不可行的。这怎么可能？

Answer 1

是这样的吗？

import threading

def printRange(lrange):
    print ("First is " + str(lrange[0]) + " and last is " + str(lrange[1]))

def runInParallel():
    ranges = [[0, 10], [10, 20], [20, 30]]
    for i in ranges:
        t = threading.Thread(target=printRange, args = [i])
        t.start()

runInParallel()

Answer 2

使用多处理

from multiprocessing import Pool


def print_range(lrange):
    print('First is {} and last is {}'.format(lrange[0], lrange[1]))


def run_in_parallel():
    ranges = [[0, 10], [10, 20], [20, 30]]
    pool = Pool(processes=len(ranges))
    pool.map(print_range, ranges)


if __name__ == '__main__':
    run_in_parallel()

输出：

First is 0 and last is 10
First is 10 and last is 20
First is 20 and last is 30

Answer 3

使用async/await进行比较

from asyncio import BaseEventLoop, get_event_loop, gather
from typing import List

async def print_range(lrange: List[int] = []):
  print('First is {} and last is {}'.format(lrange[0], lrange[1]))

def main():
  loop: BaseEventLoop = get_event_loop()
  lranges: list   = [[0, 10], [10, 20], [20, 30]]
  loop.run_until_complete(gather(*(print_range(_) for _ in lranges)))

if __name__ == '__main__':
  main()

在 Python 中并行处理不同参数的函数

Process a function on different arguments in parallel in Python

python

parallel-processing

python-multiprocessing