取消挂起的 ProcessPoolExecutor 未来

Question

我有 python 函数正在调用我无法控制或更新的 C 库。不幸的是，C 库存在间歇性错误，有时会挂起。为了防止我的应用程序也挂起，我试图隔离 ThreadPoolExecutor 或 ProcessPoolExecutor 中的函数调用，以便只有该线程或进程崩溃。

但是下面的代码挂了，因为executor无法关闭，因为进程还在运行!

是否可以取消一个 future 挂起的执行者？

import time
from concurrent.futures import ThreadPoolExecutor, wait

if __name__ == "__main__":
    def hang_forever(*args):
        print("Starting hang_forever")
        time.sleep(10.0)
        print("Finishing hang_forever")

    print("Starting executor")
    with ThreadPoolExecutor() as executor:
        future = executor.submit(hang_forever)
        print("Submitted future")
        done, not_done = wait([future], timeout=1.0)
        print("Done", done, "Not done", not_done)
        # with never exits because future has hung!
        if len(not_done) > 0:
            raise IOError("Timeout")

Answer 1

docs 说在所有未决的期货完成执行之前不可能关闭执行器：

Regardless of the value of wait, the entire Python program will not exit until all pending futures are done executing.

调用 future.cancel() 无济于事，因为它也会挂起。幸运的是，您可以直接使用 multiprocessing.Process 而不是使用 ProcessPoolExecutor:

来解决您的问题

import time
from multiprocessing import Process


def hang_forever():

    while True:
        print('hang forever...')
        time.sleep(1)


def main():
    proc = Process(target=hang_forever)

    print('start the process')
    proc.start()

    time.sleep(1)

    timeout = 3
    print(f'trying to join the process in {timeout} sec...')
    proc.join(timeout)

    if proc.is_alive():
        print('timeout is exceeded, terminate the process!')
        proc.terminate()
        proc.join()

    print('done')


if __name__ == '__main__':
    main()

输出：

start the process
hang forever...
trying to join the process in 3 sec...
hang forever...
hang forever...
hang forever...
hang forever...
timeout is exceeded, terminate the process!
done

取消挂起的 ProcessPoolExecutor 未来

Cancel an ProcessPoolExecutor future that has hung

python

python-multithreading

python-multiprocessing