重试 Deferred 中未处理的错误：在 scrapy 中

Question

我在将图像上传到 AWS S3 时在 Scrapy 中出现 BotoServerError 因为它恰好是服务器停机。我该如何处理这些错误，以便将项目管道设置为休眠一段时间，然后再次尝试上传？

2016-11-17 05:59:55 [twisted] CRITICAL: 
Traceback (most recent call last):
  File "D:\Kerja\HIT\PYTHON~1\<project_name>\<project_name>\lib\site-packages\twisted\python\threadpool.py", line 246, in inContext
    result = inContext.theWork()
  File "D:\Kerja\HIT\PYTHON~1\<project_name>\<project_name>\lib\site-packages\twisted\python\threadpool.py", line 262, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "D:\Kerja\HIT\PYTHON~1\<project_name>\<project_name>\lib\site-packages\twisted\python\context.py", line 118, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "D:\Kerja\HIT\PYTHON~1\<project_name>\<project_name>\lib\site-packages\twisted\python\context.py", line 81, in callWithContext
    return func(*args,**kw)
  File "D:\Kerja\HIT\PYTHON~1\<project_name>\<project_name>\lib\site-packages\boto\s3\key.py", line 1426, in set_contents_from_string
    encrypt_key=encrypt_key)
  File "D:\Kerja\HIT\PYTHON~1\<project_name>\<project_name>\lib\site-packages\boto\s3\key.py", line 1293, in set_contents_from_file
    chunked_transfer=chunked_transfer, size=size)
  File "D:\Kerja\HIT\PYTHON~1\<project_name>\<project_name>\lib\site-packages\boto\s3\key.py", line 750, in send_file
    chunked_transfer=chunked_transfer, size=size)
  File "D:\Kerja\HIT\PYTHON~1\<project_name>\<project_name>\lib\site-packages\boto\s3\key.py", line 951, in _send_file_internal
    query_args=query_args
  File "D:\Kerja\HIT\PYTHON~1\<project_name>\<project_name>\lib\site-packages\boto\s3\connection.py", line 668, in make_request
    retry_handler=retry_handler
  File "D:\Kerja\HIT\PYTHON~1\<project_name>\<project_name>\lib\site-packages\boto\connection.py", line 1071, in make_request
    retry_handler=retry_handler)
  File "D:\Kerja\HIT\PYTHON~1\<project_name>\<project_name>\lib\site-packages\boto\connection.py", line 1028, in _mexe
    raise BotoServerError(response.status, response.reason, body)
BotoServerError: BotoServerError: 503 Service Unavailable

Answer 1

感谢@paul！根据他的回答，我通过卸载包 boto 并安装包 boto3 来解决它。

重试 Deferred 中未处理的错误：在 scrapy 中

Retry Unhandled error in Deferred: in scrapy

python

amazon-s3

boto

amazon-web-services

scrapy