MongoDB：在聚合中使用 $lookup 在嵌套文档上填充数组

Question

我有这三个集合，我正在尝试用来自其他两个集合的 'joining in' 数据完全填充 messages_lists 集合的 messages 字段。

1. message_lists:

{ 
    "_id" : ObjectId("5e96207ea4143b3204373534"), 
    "name" : "6bq32q8TcP", 
    "messages" : [
        {
            "inserted" : ISODate("2020-04-14T20:43:42.086+0000"), 
            "srcId" : "DNWLMSNW217480-20200414T194933Z"
        }, 
        {
            "inserted" : ISODate("2020-04-16T17:40:22.585+0000"), 
            "srcId" : "DNWLMSNW217737-20200416T163914Z"
        }
    ]
}

2。留言：

{ 
    "_id" : ObjectId("5e9615205bbcbe360ad40fb7"),
    "srcId" : "DNWLMSNW217480-20200414T194933Z", 
    "status" : "expired",  
    "versions" : [
        {
            "direction" : NumberInt(-1), 
            "points" : [ 50080.028, 50080.029 ]  
        }, 
        {
            "direction" : NumberInt(1), 
            "points" : [ 50080.028, 50080.029 ] 
        }
    ]
},
{ 
    "_id" : ObjectId("5e988b95ab20413033dbe9fb"), 
    "status" : "expired", 
    "srcId" : "DNWLMSNW217737-20200416T163914Z",  
    "versions" : [
        {
            "direction" : NumberInt(1), 
            "points" : [ 50060.096, 50060.097 ] 
        }, 
        {
            "direction" : NumberInt(-1), 
            "points" : [ 50060.096, 50060.097 ]
        }
    ]
}

3。 loc_points:

{ 
    "locId" : 50060.096, 
    "road" : "A1", 
    "locCity" : NumberInt(445), 
    "locState" : NumberInt(260) 
},
{  
    "locId" : 50060.097, 
    "road" : "A1", 
    "locCity" : NumberInt(445), 
    "locState" : NumberInt(260) 
},
{ 
    "locId" : 50080.028, 
    "road" : "A2", 
    "locCity" : NumberInt(690), 
    "locState" : NumberInt(260)
},
{ 
    "locId" : 50080.029, 
    "road" : "A2", 
    "locCity" : NumberInt(690), 
    "locState" : NumberInt(260), 
}

我的mongoDB版本是3.4.4，这是我解决问题的尝试：

第 1 步： 我能够使用此查询成功完成第一个 $lookup 阶段：

db.getCollection("message_lists").aggregate([
    { $match: { name: '6bq32q8TcP' } },
    { $unwind: { path: '$messages', preserveNullAndEmptyArrays: true }},
    { $lookup: {
        from: 'messages',
        localField: 'messages.srcId',
        foreignField: 'srcId',
        as: 'messages'
    }},
    { $unwind: { path: '$messages', preserveNullAndEmptyArrays: true }},
    { $group: { 
        _id: null, 
        messages: { $push: '$messages' }
    }},
])

我对从上述查询中得到的结果非常满意。它看起来像这样，基本上设法用 messages 集合中包含的完整消息数据替换原始 messages_list 集合中对 messages 集合的引用 messages.srcId：

{ 
    "_id" : null, 
    "messages" : [
        {
            "_id" : ObjectId("5e9615205bbcbe360ad40fb7"), 
            "srcId" : "DNWLMSNW217480-20200414T194933Z",
            "status" : "expired", 
            "versions" : [
                {
                    "direction" : NumberInt(-1), 
                    "points" : [ 50080.028, 50080.029 ] 
                }, 
                {
                    "direction" : NumberInt(1), 
                    "points" : [ 50080.028, 50080.029 ] 
                }
            ] 
        }, 
        {
            "_id" : ObjectId("5e988b95ab20413033dbe9fb"), 
            "srcId" : "DNWLMSNW217737-20200416T163914Z",
            "status" : "expired", 
            "versions" : [
                {
                    "direction" : NumberInt(1), 
                    "points" : [ 50060.096, 50060.097 ]
                }, 
                {
                    "direction" : NumberInt(-1), 
                    "points" : [ 50060.096, 50060.097 ] 

                }
            ] 
        }
    ]
}

第 2 步： 此步骤的目标是使用 loc_points 集合中的数据填充 messages.versions.points 数组，以便获得最终结果结果应该是这样的：

这就是我需要的结果:

{ 
    "_id" : null, 
    "messages" : [
        {
            "_id" : ObjectId("5e9615205bbcbe360ad40fb7"), 
            "srcId" : "DNWLMSNW217480-20200414T194933Z",
            "status" : "expired", 
            "versions" : [
                {
                    "direction" : NumberInt(-1), 
                    "points" : [ 
                      { 
                          "locId" : 50080.028, 
                          "road" : "A2", 
                          "locCity" : NumberInt(690), 
                          "locState" : NumberInt(260)
                      },
                      { 
                          "locId" : 50080.029, 
                          "road" : "A2", 
                          "locCity" : NumberInt(690), 
                          "locState" : NumberInt(260), 
                      }
                    ] 
                }, 
                {
                    "direction" : NumberInt(1), 
                    "points" : [ 
                      { 
                          "locId" : 50080.028, 
                          "road" : "A2", 
                          "locCity" : NumberInt(690), 
                          "locState" : NumberInt(260)
                      },
                      { 
                          "locId" : 50080.029, 
                          "road" : "A2", 
                          "locCity" : NumberInt(690), 
                          "locState" : NumberInt(260), 
                      }
                    ] 
                }
            ] 
        }, 
        {
            "_id" : ObjectId("5e988b95ab20413033dbe9fb"), 
            "srcId" : "DNWLMSNW217737-20200416T163914Z",
            "status" : "expired", 
            "versions" : [
                {
                    "direction" : NumberInt(1), 
                    "points" : [ 
                      { 
                          "locId" : 50060.096, 
                          "road" : "A1", 
                          "locCity" : NumberInt(445), 
                          "locState" : NumberInt(260) 
                      },
                      {  
                          "locId" : 50060.097, 
                          "road" : "A1", 
                          "locCity" : NumberInt(445), 
                          "locState" : NumberInt(260) 
                      }
                    ]
                }, 
                {
                    "direction" : NumberInt(-1), 
                    "points" : [ 
                      { 
                          "locId" : 50060.096, 
                          "road" : "A1", 
                          "locCity" : NumberInt(445), 
                          "locState" : NumberInt(260) 
                      },
                      {  
                          "locId" : 50060.097, 
                          "road" : "A1", 
                          "locCity" : NumberInt(445), 
                          "locState" : NumberInt(260) 
                      }
                    ] 
                }
            ] 
        }
    ]
}

为了到达那里，我的理解是我需要像这样执行额外的 $lookup：

{ $lookup: {
    from: 'loc_points',
    localField: 'messages.versions.points',
    foreignField: 'locId',
    as: 'messages.versions.points',
}},
{ $unwind: { path: '$messages.versions.points', preserveNullAndEmptyArrays: true }}

但是，经过一整天的研究并阅读了至少五个其他 Whosebug 线程后，我无法找到类似于我的深层嵌套文档结构的解决方案。

如果有更熟悉 mongoDB 聚合框架的人能帮助我，我将不胜感激！提前致谢！！

编辑： 在 Joe 的下面建议之后，我在最后的 $group 语句之前添加了第二个 $lookup 。所以现在我的整体查询如下所示：

db.getCollection("message_lists").aggregate([
    { $match: { name: '6bq32q8TcP' } },
    { $unwind: { path: '$messages', preserveNullAndEmptyArrays: true }},
    { $lookup: {
        from: 'messages',
        localField: 'messages.srcId',
        foreignField: 'srcId',
        as: 'messages'
    }},
    { $unwind: { path: '$messages', preserveNullAndEmptyArrays: true }},
    { $lookup: {
        from: 'loc_points',
        localField: 'messages.versions.points',
        foreignField: 'locId',
        as: 'messages.versions.points',
    }},
    { $group: { 
        _id: null, 
        messages: { $push: '$messages' }
    }}
])

但是，结果 messages.versions.points 作为一个空数组返回，所有其他 version 字段都丢失并且 messages.versions 甚至不再是数组：

{ 
    "_id" : null, 
    "messages" : [
        {
            "_id" : ObjectId("5e9615205bbcbe360ad40fb7"), 
            "srcId" : "DNWLMSNW217480-20200414T194933Z",
            "status" : "expired", 
            "versions" : {
                "points" : [
                ]
            },  
        }, 
        {
            "_id" : ObjectId("5e988b95ab20413033dbe9fb"), 
            "srcId" : "DNWLMSNW217737-20200416T163914Z",
            "status" : "expired", 
            "versions" : {
                "points" : [
                ]
            },  
        }
    ]
}

我只是想不通我需要如何修改 $group 语句才能正确填充 messages.versions.points。任何帮助表示赞赏！（我知道，来自 MySQL 世界，我表明我还没有完全掌握聚合的概念，但与此同时，我正在努力寻找适合我的场景的教程）

Answer 1

$lookup 阶段 returns 管道中每个源文档的单个数组。

如果您在现有管道之后使用了您记下的附加 $lookup 阶段，它将添加一个数组，该数组将包含任何消息的任何版本所引用的点集。

为了获得为每条消息的每个版本填充的点数，必须展开这些点数。您已经有了消息，因此您需要展开版本和运行额外查找，然后按消息分组以将版本收集回数组，然后现有组收集消息。

Answer 2

试试这个

db.message_lists.aggregate([
    {
        $unwind: '$messages'
    },
    {
        $lookup: {
            from: 'messages',
            localField: 'messages.srcId',
            foreignField: 'srcId',
            as: 'messages'
        }
    },
    {
        $project: {
            _id: 1,
            name: 1,
            message: { $arrayElemAt: ["$messages", 0] },
        }
    },
    {
        $unwind: '$message.versions'
    },
    {
        $unwind: '$message.versions.points'
    },
    {
        $lookup: {
            from: 'loc_point',
            localField: 'message.versions.points',
            foreignField: 'locId',
            as: 'points'
        }
    },
    {
        $unwind: '$points'
    },
    {
        $project: {
            _id: 1,
            name: 1,
            message: {
                _id: '$message._id',
                srcId: '$message.srcId',
                status: '$message.status',
                versions: {
                    direction: "$message.versions.direction",
                    points: '$points'
                }
            }
        }
    },
    {
        $group: {
            _id: { message_id: '$message._id', direction: '$message.versions.direction' },
            name: { $first: '$name' },
            srcId: { $first: '$message.srcId' },
            status: { $first: '$message.status' },
            points: {
                $push: '$message.versions.points'
            }
        }
    },
    {
        $group: {
            _id: '$_id',
            name: { $first: '$name' },
            srcId: { $first: 'srcId' },
            status: { $first: '$status' },
            versions: {
                $push: {
                    direction: '$_id.direction',
                    points: '$points'
                }
            }
        }
    },
    {
        $project: {
            _id: "$_id.message_id",
            name: 1,
            srcId: 1,
            status: 1,
            versions: 1
        }
    },
    {
        $unwind: '$versions'
    },
    {
        $group: {
            _id: '$_id',
            status: { $first: '$status' },
            srcId: { $first: '$srcId' },
            versions: { $push: '$versions' }

        }
    },
    {
        $group: {
            _id: null,
            messages: { $push: '$$ROOT' }
        }
    }
])

MongoDB：在聚合中使用 $lookup 在嵌套文档上填充数组

MongoDB: populating arrays on nested document with $lookup in aggregation

lookup

grouping

nested

aggregate

mongodb