如何使用 NodeJS (NestJS) 向 Google 云存储 (GCS) 进行可续传上传？

Question

我在 NestJS 中创建可恢复的上传服务到 GCS 时遇到问题。

场景是，客户端从前端上传一个文件，在后端直接发送给GCS，而不是暂时存储在BE服务器上。

这是我正在使用的代码片段。

try {
  const filePath = path.join(directory, nameWithExtension);
  const file = this.bucket.file(filePath);
  const passthroughStream = new stream.PassThrough();
  passthroughStream.write(image.buffer);
  passthroughStream.end();

  const streamFileUpload = async () => {
    passthroughStream
      .pipe(file.createWriteStream({ resumable: true, gzip: true, public: true }))
      .on('finish', () => console.log(`resumable upload succeed`));
    return filePath;
  };

  const res = await streamFileUpload().catch((error) => {
    throw new Error(`${logPrefix} Error uploading ${filePath} ${error.message}`);
  });
  return `${process.env.GOOGLE_STORAGE_ENDPOINT}/${this.bucket.name}/${res}`;
} catch (error) {
  throw new Error(`${logPrefix} Error uploading ${error.message}`);
}

关于 createWriteStream 我包含了选项 resumable: true 但它似乎没有按预期工作。

我很好奇这个https://cloud.google.com/storage/docs/performing-resumable-uploads，但还是不太明白。

非常感谢任何建议，谢谢！

更新 2021/10/06

我将代码更改为如下所示：

async resumableUpload(directory: string, image: MultipartFile, nameWithExtension: string): Promise<string> {
  const logPrefix = 'GoogleStorageService.resumableUpload:';
  const filePath = path.join(directory, nameWithExtension);
  const { buffer } = image;
  const blob = this.bucket.file(filePath);
  const promiseUpload = new Promise((resolve, reject) => {
    const blobStream = blob.createWriteStream({
      resumable: true,
      gzip: true,
      public: true,
    });
    blobStream
      .on('error', () => {
        reject(`${logPrefix} Unable to upload image, something went wrong`);
      })
      .on('finish', async () => {
        const publicUrl = new URL(process.env.GOOGLE_STORAGE_ENDPOINT || '');
        publicUrl.pathname = path.join(this.bucket.name, filePath);
        resolve(publicUrl.toString());
      })
      .end(buffer);
  });
  const response = promiseUpload
    .then((res: string) => res)
    .catch((err: Error) => {
      throw new Error(`${logPrefix} Error uploading ${err.message}`);
    });
  return response;
}

结果很好。

但是如果有更好的方法请不要犹豫，为这个问题提供最好的建议。谢谢

Answer 1

将@Wisnu 解决方案发布为社区 wiki 以获得更好的可见性。 Wisnu 编辑了下面的代码后，断点续传服务开始工作了。


async resumableUpload(directory: string, image: MultipartFile, nameWithExtension: string): Promise<string> {
  const logPrefix = 'GoogleStorageService.resumableUpload:';
  const filePath = path.join(directory, nameWithExtension);
  const { buffer } = image;
  const blob = this.bucket.file(filePath);
  const promiseUpload = new Promise((resolve, reject) => {
    const blobStream = blob.createWriteStream({
      resumable: true,
      gzip: true,
      public: true,
    });
    blobStream
      .on('error', () => {
        reject(`${logPrefix} Unable to upload image, something went wrong`);
      })
      .on('finish', async () => {
        const publicUrl = new URL(process.env.GOOGLE_STORAGE_ENDPOINT || '');
        publicUrl.pathname = path.join(this.bucket.name, filePath);
        resolve(publicUrl.toString());
      })
      .end(buffer);
  });
  const response = promiseUpload
    .then((res: string) => res)
    .catch((err: Error) => {
      throw new Error(`${logPrefix} Error uploading ${err.message}`);
    });
  return response;
}

如何使用 NodeJS (NestJS) 向 Google 云存储 (GCS) 进行可续传上传？

How to make a resumable upload using NodeJS (NestJS) to Google Cloud Storage (GCS)?

google-cloud-platform

resumable

nestjs