Celery 的异常 'TimeLimitExceeded' 与任务组

Question

我有这个芹菜的设置：

WORKER_MAX_TASKS_PER_CHILD = 1
TASK_TIME_LIMIT = 30

当我运行组任务：

from celery import group, shared_task
from time import sleep

@shared_task
def do_something(arg):
    sleep(60)
    return arg*2

group([do_something.s(i) for i in range(3)]).apply_async()

我正在 TimeLimitExceeded 进入组，然后工人立即被芹菜杀死。我该如何处理？

Answer 1

根据 documentation:

The soft time limit allows the task to catch an exception to clean up before it is killed: the hard timeout isn’t catch-able and force terminates the task.

答案很简单：如果您想捕获异常，请不要对任务使用硬时间限制。

Celery 的异常 'TimeLimitExceeded' 与任务组

Celery's exception 'TimeLimitExceeded' with group of tasks

python

celery