DynamoDB batchWrite 在继续之前没有等待,并且没有错误?
DynamoDB batchWrite not awaiting before moving on, and no errors?
我有一个 lambda 预计会遍历扫描的项目,在 25 个项目批次中对每个项目进行批处理,从这些项目创建 DeleteRequest 并使用 Dynamo 的 .batchWrite()
方法删除它们。
我在使用 DynamoDB 的 dynamoDb.batchWrite({ RequestItems: requestItems }).promise()
方法时遇到了问题,它似乎永远不会在移动到我的 25 循环批次的下一次迭代之前完成。我认为这是我的 async/await 的问题,但是,我似乎无法发现它!
此 lambda 运行ning 的输出如下所示:
我有一个扫描项目的方法:
async function getAllItemsFromTable (TableName) {
const res = await dynamoDb.scan({ TableName }).promise()
return res.Items
}
一个批处理函数(所以每次请求都是针对25个items,满足需求):
function batch(array, size) {
const chunked_arr = []
let index = 0
while (index < array.length) {
chunked_arr.push(array.slice(index, size + index))
index += size
}
return chunked_arr
}
删除函数(通过写入 Dynamo 来处理这些批次):
async function deleteAllItemsFromTable (items) {
let numItemsDeleted = 0
const batchedArr = batch(items, 25)
batchedArr.forEach(async (chunk) => {
const requestItems = {
[tableName]: chunk.map((item) => {
numItemsDeleted++
const Key = { id: item.id }
return { DeleteRequest: { Key } }
}),
}
if (requestItems[tableName].length > 0) {
console.log("requestItems", requestItems)
try {
console.log("Starting Batch Write..") // this prints.
// this line below NEVER finishes. It also doesn't spit any errors out. I'm awaiting, what else?
await dynamoDb.batchWrite({ RequestItems: requestItems }).promise()
console.log("Finished Batch Write..") // this doesn't ever print.
} catch (error) {
console.log("Error: ", error) // this doesn't ever print.
}
}
})
console.log("numItemsDeleted", numItemsDeleted) // this prints.
return { numItemsDeleted }
}
最后,我 运行 我的 lambda 是这样的:
const items = await getAllItemsFromTable(tableName)
const { numItemsDeleted } = await deleteAllItemsFromTable(items)
console.log(`--- ${numItemsDeleted} items deleted`)
问题是您永远不会等待批处理列表循环执行。你开始所有的执行,然后移动到下一行。所以你最终永远不会等待处决。
变化:
batchedArr.forEach(async (chunk) => {...})
至
await Promise.all(batchedArr.map(async (chunk) => {...})))
我有一个 lambda 预计会遍历扫描的项目,在 25 个项目批次中对每个项目进行批处理,从这些项目创建 DeleteRequest 并使用 Dynamo 的 .batchWrite()
方法删除它们。
我在使用 DynamoDB 的 dynamoDb.batchWrite({ RequestItems: requestItems }).promise()
方法时遇到了问题,它似乎永远不会在移动到我的 25 循环批次的下一次迭代之前完成。我认为这是我的 async/await 的问题,但是,我似乎无法发现它!
此 lambda 运行ning 的输出如下所示:
我有一个扫描项目的方法:
async function getAllItemsFromTable (TableName) {
const res = await dynamoDb.scan({ TableName }).promise()
return res.Items
}
一个批处理函数(所以每次请求都是针对25个items,满足需求):
function batch(array, size) {
const chunked_arr = []
let index = 0
while (index < array.length) {
chunked_arr.push(array.slice(index, size + index))
index += size
}
return chunked_arr
}
删除函数(通过写入 Dynamo 来处理这些批次):
async function deleteAllItemsFromTable (items) {
let numItemsDeleted = 0
const batchedArr = batch(items, 25)
batchedArr.forEach(async (chunk) => {
const requestItems = {
[tableName]: chunk.map((item) => {
numItemsDeleted++
const Key = { id: item.id }
return { DeleteRequest: { Key } }
}),
}
if (requestItems[tableName].length > 0) {
console.log("requestItems", requestItems)
try {
console.log("Starting Batch Write..") // this prints.
// this line below NEVER finishes. It also doesn't spit any errors out. I'm awaiting, what else?
await dynamoDb.batchWrite({ RequestItems: requestItems }).promise()
console.log("Finished Batch Write..") // this doesn't ever print.
} catch (error) {
console.log("Error: ", error) // this doesn't ever print.
}
}
})
console.log("numItemsDeleted", numItemsDeleted) // this prints.
return { numItemsDeleted }
}
最后,我 运行 我的 lambda 是这样的:
const items = await getAllItemsFromTable(tableName)
const { numItemsDeleted } = await deleteAllItemsFromTable(items)
console.log(`--- ${numItemsDeleted} items deleted`)
问题是您永远不会等待批处理列表循环执行。你开始所有的执行,然后移动到下一行。所以你最终永远不会等待处决。
变化:
batchedArr.forEach(async (chunk) => {...})
至
await Promise.all(batchedArr.map(async (chunk) => {...})))