Python 创建工作服务器时出现 Celery 错误

Question

我开始使用 python 中的 Celery 模块，并按照 Celery 网站上的说明创建了一个新脚本，但在使用

创建 Celery 工作服务器时收到错误

celery -A addTask worker --loglevel=info

addTask 是一个返回总和的简单 python 脚本：

from celery import Celery 
app = Celery('addTask', broker = 'amqp://guest@localhost//')
@app.task
def add(x,y):
    return x+y

调用 addTask.py 的另一个脚本是 runAddTask.py：

import addTask
if __name__=='__main__':
    result = addTask.add.delay(5,5)

我用的Celery的版本是4.0.2，Python的版本是3.5.2。我在 Windows 10 上运行。我安装了 rabbitmq 作为代理，所以脚本没问题。这与 celeryproject 中列出的完全相同，但是，我收到如下错误消息：

C:\Users\user\desktop\parallel_python\celery> celery -A addTask worker --loglevel=info
celery : [2017-01-04 14:23:46,058: CRITICAL/MainProcess] Unrecoverable error: AttributeError("Can't pickle local object 
'Pool.__init__.<locals>.Process'",)
At line:1 char:1
+ celery -A addTask worker --loglevel=info
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: ([2017-01-04 14:...ls>.Process'",):String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

Traceback (most recent call last):
  File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\celery\worker\worker.py", line 203, in start
    self.blueprint.start(self)
  File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\celery\bootsteps.py", line 119, in start
    step.start(parent)
  File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\celery\bootsteps.py", line 370, in start
    return self.obj.start()
  File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\celery\concurrency\base.py", line 131, in 
start
    self.on_start()
  File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\celery\concurrency\prefork.py", line 112, in 
on_start
    **self.options)
  File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\pool.py", line 1008, in __init__
    self._create_worker_process(i)
  File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\pool.py", line 1117, in 
_create_worker_process
    w.start()
  File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\process.py", line 122, in start
    self._popen = self._Popen(self)
  File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\context.py", line 383, in _Popen
    return Popen(process_obj)
  File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\popen_spawn_win32.py", line 79, in 
__init__
    reduction.dump(process_obj, to_child)
  File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\reduction.py", line 99, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'Pool.__init__.<locals>.Process'

 -------------- celery@userv4.0.2 (latentcall)
---- **** ----- 
--- * ***  * -- Windows-10-10.0.14393-SP0 2017-01-04 14:23:46
-- * - **** --- 
- ** ---------- [config]
- ** ---------- .> app:         addTask:0x1d17467e0b8
- ** ---------- .> transport:   amqp://guest:**@localhost:5672//
- ** ---------- .> results:     disabled://
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** ----- 
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . addTask.add

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\spawn.py", line 159, in spawn_main
    new_handle = steal_handle(parent_pid, pipe_handle)
  File "c:\users\user\appdata\local\programs\python\python35\lib\site-packages\billiard\reduction.py", line 126, in 
steal_handle
    _winapi.DUPLICATE_SAME_ACCESS | _winapi.DUPLICATE_CLOSE_SOURCE)
PermissionError: [WinError 5] Access is denied

我真的卡在这里了。任何帮助是极大的赞赏！谢谢！

Answer 1

https://github.com/celery/celery/issues/3551

windows 未被 celery 4 正式支持。

您可以降级到 Celery 3.x.x。

Python 创建工作服务器时出现 Celery 错误

Python Celery error when creating worker server

python

distributed-computing