芹菜 - 工人有时只会接任务

Question

我正在构建一个可以在线访问的潜在客户生成门户。请不要介意代码的冗长，我现在正在进行大量调试。

我的 Celery 工作器不一致地接收分配给它的任务，我不确定为什么。

奇怪的是，有时它 100% 完美地工作：终端中从来没有任何明显的错误。

我目前在 DEBUG = TRUE 和 REDIS 作为经纪人！

celery 启动 worker 终端命令和响应

celery -A mysite worker -l info --pool=solo
 -------------- celery@DESKTOP-OG8ENRQ v5.0.2 (singularity)
--- ***** -----
-- ******* ---- Windows-10-10.0.19041-SP0 2020-11-09 00:36:13
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app:         mysite:0x41ba490
- ** ---------- .> transport:   redis://localhost:6379//
- ** ---------- .> results:     redis://localhost:6379/
- *** --- * --- .> concurrency: 12 (solo)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery
[tasks]
  . mysite.celery.debug_task
  . submit

[2020-11-09 00:36:13,899: INFO/MainProcess] Connected to redis://localhost:6379//
[2020-11-09 00:36:14,939: WARNING/MainProcess] c:\users\coole\pycharmprojects\lead_django_retry\venv\lib\site-packages\celery\app\control.py:48: DuplicateNodenameWarning: Received multiple replies from node name: celery@DESKTOP-OG8ENRQ.
Please make sure you give each node a unique nodename using
the celery worker `-n` option.
  warnings.warn(DuplicateNodenameWarning(

[2020-11-09 00:36:14,939: INFO/MainProcess] mingle: all alone
[2020-11-09 00:36:14,947: INFO/MainProcess] celery@DESKTOP-OG8ENRQ ready.

views.py

class LeadInputView(FormView):
template_name = 'lead_main.html'
form_class = LeadInput

def form_valid(self, form):
    print("I'm at views")
    form.submit()
    print(form.submit)
    return HttpResponseRedirect('./success/')

tasks.py

@task(name="submit")
def start_task(city, category, email):
    print("I'm at tasks!")
    print(city, category, email)
    """sends an email when feedback form is filled successfully"""
    logger.info("Submitted")
    return start(city, category, email)

forms.py

class LeadInput(forms.Form):
    city = forms.CharField(max_length=50)
    category = forms.CharField(max_length=50)
    email = forms.EmailField()

    def submit(self):
        print("I'm at forms!")
        x = (start_task.delay(self.cleaned_data['city'], self.cleaned_data['category'], self.cleaned_data['email']))
        return x

celery.py

from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.settings')
app = Celery('mysite')

app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)


@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

settings.py

BROKER_URL = 'redis://localhost:6379'
CELERY_RESULT_BACKEND = 'redis://localhost:6379'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'UTC'

runserver 终端看起来像这样：

I'm at views
I'm at forms!
<bound method LeadInput.submit of <LeadInput bound=True, valid=True, fields=(city;category;email)>>

但是工人并没有说它拿起了任何东西，只是说“celery@DESKTOP-OG8ENRQ 准备好了”。除了，当它确实起作用时……出于某种原因？我很茫然！

Answer 1

默认情况下，如果您愿意在同一主机上使用多个工作人员，则 celery 将使用主机名作为工作人员名称，然后指定 -n 选项。

celery -A mysite worker -l info --pool=solo -n worker2@%h

您的代码工作正常，但任务已传递给第一个工作人员，请参阅 DuplicateNodenameWarning with no obvious reason #2938

Answer 2

看到这个的人你好。事实证明，这是 celery（或者也许是 redis？）的错误......显然有很多 windows 用户运行参与其中。 https://github.com/celery/celery/issues/3759

事实证明，答案是在启动 worker 时让 -P solo。我不确定为什么会这样......但是解决了它！

感谢 Naqib 的帮助！你让我找到了解决方案。

芹菜 - 工人有时只会接任务

Celery - worker only sometimes picks up tasks

python

django

asynchronous

redis

celery