使用 Lambda / S3 逐行读取文件

Question

我想逐行读取位于 S3 上的文件。我尝试了以下我在网上搜索到的代码，但 Lambda 函数在没有调用任何 readline 回调的情况下退出。我做错了什么？

const aws = require('aws-sdk');
const s3 = new aws.S3({ apiVersion: '2006-03-01' });
const readline = require('readline');

exports.handler = async (event, context, callback) => {
    const bucket = event.Records[0].s3.bucket.name;
    const key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '));
    const params = {
        Bucket: bucket,
        Key: key,
    };

    const s3ReadStream = s3.getObject(params).createReadStream();

    const rl = readline.createInterface({
      input: s3ReadStream,
      terminal: false
    });

    rl.on('line', (line) => {
      console.log(`Line from file: ${line}`);
    });
    rl.on('error', () => {
        console.log('error');
    });
    rl.on('close', function () {
        console.log('closed');
        context.succeed();
    });
    console.log('done');
};

Answer 1

getObject 不只是 return 存储在 S3 中的对象。它 return 一个 JSON 对象，其 Body 字段包含存储到 S3 的对象的 blob。另请参阅文档 here.

的 Response 部分

Answer 2

我找到问题了。我已经有一段时间没有在 Lambda 上编写代码了，我认为它只会在调用上下文时退出。我现在正在等待承诺得到解决（或拒绝，我稍后会实施）。

const aws = require('aws-sdk');
const s3 = new aws.S3({ apiVersion: '2006-03-01' });
const readline = require('readline');

exports.handler = async (event, context, callback) => {
    const bucket = event.Records[0].s3.bucket.name;
    const key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '));
    const params = {
        Bucket: bucket,
        Key: key,
    };
    const s3ReadStream = s3.getObject(params).createReadStream();

    const rl = readline.createInterface({
      input: s3ReadStream,
      terminal: false
    });

    let myReadPromise = new Promise((resolve, reject) => {

        rl.on('line', (line) => {
          console.log(`Line from file: ${line}`);
        });
        rl.on('error', () => {
            console.log('error');
        });
        rl.on('close', function () {
            console.log('closed');
            resolve();
        });
    });

    try { await myReadPromise; }
    catch(err) {
        console.log('an error has occurred');
    }

    console.log('done reading!');
};

使用 Lambda / S3 逐行读取文件

Read a file line by line using Lambda / S3

amazon-s3

node.js

aws-sdk

aws-lambda