boto set_contents_from_filename 内存泄漏

Question

我在使用 boto 上传文件时发现内存泄漏。我在这里做错了什么吗？如果我取消睡眠或者如果我不在两个不同的桶之间交替，内存使用量似乎不会持续增加。

import time, resource, os

import boto

conn = boto.connect_s3()
for i in range(20):
  print resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
  path = 'test.png'
  bucket = conn.lookup('jca-screenshots-' + ('thumbs' if i % 2 == 0 else 'normal'))
  k = boto.s3.key.Key(bucket)
  k.key = os.path.basename(path)
  k.set_contents_from_filename(path)
  time.sleep(5)

示例输出：

Answer 1

通过切换库解决：https://github.com/tax/python-requests-aws

import time, resource, os

import requests
from awsauth import S3Auth

with open(os.path.expanduser("~/.boto")) as f:
  lines = f.read().splitlines()
ACCESS_KEY = lines[1].split(' = ')[1]
SECRET_KEY = lines[2].split(' = ')[1]

for i in range(20):
  print resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
  url = 'http://{}.s3.amazonaws.com/{}'.format(
    'jca-screenshots-' + ('thumbs' if i % 2 == 0 else 'normal'), 'test.png')
  with open('test.png', 'rb') as f:
    resp = requests.put(url, data=f, auth=S3Auth(ACCESS_KEY, SECRET_KEY))
  print 'resp:', resp
  time.sleep(5)

boto set_contents_from_filename 内存泄漏

boto set_contents_from_filename memory leak

python

boto