如何从端点下载文件并将该文件上传到 S3？（GITLAB）

Question

我实际上是在尝试从 Gitlab REST 端点下载一个 zip 文件，该端点应该 return 给定项目 ID 的存储库。

我使用 axios 调用端点并尝试直接将响应数据上传到 S3，但它似乎 returning 了一个损坏的文件，正如它 returns 所说的那样无法打开。

我正在使用无服务器功能下载此文件并尝试 return 一个 S3 URL 到客户端。

Headers 回复

  res.headers {
  date: 'Wed, 19 Jan 2022 13:44:42 GMT',
  'content-type': 'application/zip',
  'transfer-encoding': 'chunked',
  connection: 'close',
  'cache-control': 'max-age=0, private, must-revalidate',
  'content-disposition': 'attachment; filename="third-project-eac3ea41c782df4bee4fe07ecc3bf356f7f74f47-eac3ea41c782df4bee4fe07ecc3bf356f7f74f47.zip"',
  'content-transfer-encoding': 'binary',
  etag: 'W/"12ae32cb1ec02d01eda3581b127c1fee"',
  vary: 'Origin',
  'x-content-type-options': 'nosniff',
  'x-frame-options': 'SAMEORIGIN',
  'x-request-id': '01FSS9A6W1RM77TYMFM6G7HKZ8',
  'x-runtime': '0.063985',
  'strict-transport-security': 'max-age=31536000',
  'referrer-policy': 'strict-origin-when-cross-origin',
  'ratelimit-observed': '2',
  'ratelimit-remaining': '1998',
  'ratelimit-reset': '1642599941',
  'ratelimit-resettime': 'Wed, 19 Jan 2022 13:45:41 GMT',
  'ratelimit-limit': '2000',
  'gitlab-lb': 'fe-17-lb-gprd',
  'gitlab-sv': 'localhost',
  'cf-cache-status': 'MISS',
  'expect-ct': 'max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"',
  'report-to': '{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=r6XcPGaiU7JhlicrSC9iBZgXXCOMoBMXjU8kvxjZGb5UkUQBIjemmAOOX39m1ijVCnQROVhNNxc6B%2B4x%2FNf5ZG9cc8GLY%2BfMYUE29gJkHN624QKJRSX8HBrMqEQ%3D"}],"group":"cf-nel","max_age":604800}',
  nel: '{"success_fraction":0.01,"report_to":"cf-nel","max_age":604800}',
  server: 'cloudflare',
  'cf-ray': '6d007fcb7a8a8e63-PDX'
}

这是我正在使用的代码

const requestURL = `https://gitlab.com/api/v4/projects/${repoID}/repository/archive.zip?sha=${commitSHA}`;
 let url = await Axios({
    method: "GET",
    url: requestURL,
    headers: {
      Accept: "*/*",
      Authorization: "Bearer " + authToken,
    },
  })
    .then(async (res) => {
      
      const Key = "path/to/file";
      const params = {
        Bucket: BUCKET_URL,
        Key,
        Body: res.data,
        ContentType: "application/zip",
      };
      await s3.upload(params).promise();

      const URL = await s3.getSignedUrl("getObject", {
        Bucket: BUCKET_URL,
        Key,
        ResponseContentDisposition:'attachment; filename = test.zip"',
      });
      return URL ;
    })
    .catch((error) => {
      console.log(error);
    });
   return url;

但是当我尝试从前端在我的浏览器上访问这个 URL 时，它 return 是一个更大且损坏的文件 test.zip。

这是我在postman上收到的数据截图，如果我单击保存响应并将文件命名为 filename.zip，它会显示文件的实际内容。

如有任何帮助，我们将不胜感激！

Answer 1

这成功了！

const requestURL = `https://gitlab.com/api/v4/projects/${repoID}/repository/archive.zip?sha=${commitSHA}`;
 let url = await Axios.get(requestURL,{
    headers: {
      Accept: "*/*",
      Authorization: "Bearer " + authToken,
    },
    responseType:'arraybuffer'
  })
    .then(async (res) => {
      
      const bufferData = Buffer.from(res.data)
      const Key = "path/to/file";
      const params = {
        Bucket: BUCKET_URL,
        Key,
        Body: bufferData,
        ContentType: "buffer",
      };
      await s3.upload(params).promise();

      const URL = await s3.getSignedUrl("getObject", {
        Bucket: BUCKET_URL,
        Key,
        ResponseContentDisposition:'attachment; filename = test.zip"',
      });
      return URL ;
    })
    .catch((error) => {
      console.log(error);
    });
   return url;

如何从端点下载文件并将该文件上传到 S3？（GITLAB）

How to download a file from an endpoint and upload that file to S3 ?(GITLAB)

amazon-s3

gitlab

gitlab-api