Python 运行同时有 2 个类然后同时有 2 个类

Question

我有一个包含许多 class 的脚本，例如我有 100 个 class

当我尝试运行同时（并行）所有这些时，我的内存不足并且我的脚本被杀死了。

这是我的脚本结构的示例：

from threading import *

class Test1(Thread):
def run(self):
    print("done1")

Test1().start()

class Test2(Thread):
    def run(self):
        print("done2")


Test2().start()

class Test3(Thread):
def run(self):
    print("done3")

Test3().start()

class Test4(Thread):
    def run(self):
        print("done4")


Test4().start()

使用此代码，脚本运行 Test1() Test2() Test3() 和 Test4() 同时（并行）。

我正在寻找的是运行 Test1() 和 Test2() 同时（并行）并等到它们完成然后运行 Test3() 和 Test4()并行。

请问我该怎么做

Answer 1

您可以在这里采取一些方法。您可以使用 queue.Queue 并安排一定数量的线程：

import os
import random
import time

from queue import Queue
from threading import Thread

THREADS = os.cpu_count()


class WorkerThread(Thread):
    def __init__(self, q, idx):
        super().__init__()

        self.q = q
        self.idx = idx

    def run(self):
        for num in iter(self.q.get, "STOP"):
            t = random.uniform(1, 5)
            print(f"START (THREAD: {self.idx})")
            time.sleep(t)
            print(f"END   (THREAD: {self.idx}), RESULT: {num * num}")


def main():
    q = Queue()

    for i in range(10):
        q.put(i)

    threads = []

    for i in range(THREADS):
        t = WorkerThread(q, i)
        t.start()
        threads.append(t)

    for _ in range(THREADS):
        q.put("STOP")


if __name__ == "__main__":
    main()

测试：

$ python test1.py 
START (THREAD: 0)
START (THREAD: 1)
START (THREAD: 2)
START (THREAD: 3)
START (THREAD: 4)
START (THREAD: 5)
START (THREAD: 7)
START (THREAD: 6)
END   (THREAD: 6), RESULT: 36
START (THREAD: 6)
END   (THREAD: 0), RESULT: 0
START (THREAD: 0)
END   (THREAD: 5), RESULT: 25
END   (THREAD: 4), RESULT: 16
END   (THREAD: 1), RESULT: 1
END   (THREAD: 3), RESULT: 9
END   (THREAD: 7), RESULT: 49
END   (THREAD: 6), RESULT: 64
END   (THREAD: 2), RESULT: 4
END   (THREAD: 0), RESULT: 81

或者您可以使用 concurrent.futures module that provides a high-level interface. The ThreadPoolExecutor 对象将创建一个工作池：

import random
import time

from concurrent.futures import ThreadPoolExecutor


def worker(num):
    t = random.uniform(1, 5)
    print(f"START ({num})")
    time.sleep(t)

    return num * num


def main():
    with ThreadPoolExecutor() as executor:
        for result in executor.map(worker, range(10)):
            print(f"RESULT: {result}")


if __name__ == "__main__":
    main()

Python 运行同时有 2 个类然后同时有 2 个类

Python Run 2 classes at the same time then 2 other classes at the same time

python

parallel-processing

multithreading

Python 运行 同时有 2 个 类 然后同时有 2 个 类

Python Run 2 classes at the same time then 2 other classes at the same time

python

parallel-processing

multithreading

Python 运行同时有 2 个类然后同时有 2 个类