使用 api 客户端库从 google 云存储下载失败，出现 HTTP 416

Question

不幸的是，我不想依赖除 googleapiclient 之外的其他软件包，并且在从存储桶下载对象时遇到一些问题。

from googleapiclient.http import MediaIoBaseDownload
import googleapiclient.discovery

storage_service = googleapiclient.discovery.build(
    serviceName='storage', version='v1', credentials=creds)

f = storage_service.objects()
results = f.list(bucket='MYBUCKET').execute()

for d in results['items']:
    with open(d['name']), 'wb') as fh:
        req = MediaIoBaseDownload(
            fh,
            f.get_media(bucket=d['bucket'], object=d['name'], generation=d['generation']),
            chunksize=1024*1024
        )
        done = False
        while done is False:
            status, done = req.next_chunk()

现在这会产生以下错误：

---------------------------------------------------------------------------
HttpError                                 Traceback (most recent call last)
<ipython-input-210-d66ce751dec5> in <module>()
      4         done = False
      5         while done is False:
----> 6             status, done = req.next_chunk()

path_to_my_env/lib/python2.7/site-packages/googleapiclient/_helpers.pyc in positional_wrapper(*args, **kwargs)
    128                 elif positional_parameters_enforcement == POSITIONAL_WARNING:
    129                     logger.warning(message)
--> 130             return wrapped(*args, **kwargs)
    131         return positional_wrapper
    132 

path_to_my_env/lib/python2.7/site-packages/googleapiclient/http.pyc in next_chunk(self, num_retries)
    703       return MediaDownloadProgress(self._progress, self._total_size), self._done
    704     else:
--> 705       raise HttpError(resp, content, uri=self._uri)
    706 
    707 

HttpError: <HttpError 416 when requesting https://www.googleapis.com/storage/v1/b/MYBUCKET/o/MYOBJECT?generation=1234&alt=media returned "Requested range not satisfiable">

有人知道我在某处遗漏了什么或从存储中下载文件的最佳做法是什么吗？我发现的一切都依赖于特定于存储的库。

Answer 1

这似乎与 this issue 有关。

一些文件（大多数）是 0 字节文件。

问题已通过以下代码解决：

for d in results['items']:
    request = f.get_media(bucket=d['bucket'], object=d['name'], generation=d['generation'])
    response = request.execute()
    if response != '':
        with open(d['name']), 'wb') as fh:
            fh.write(response)

使用 api 客户端库从 google 云存储下载失败，出现 HTTP 416

Downloading from google cloud storage with api client library fails with HTTP 416

python

download

google-api-python-client