将结果从 jq 导出到 csv
export results from jq to csv
我有以下数据集，我想将其导出为 CSV：
数据集：
{
"data": {
"activeFindings": {
"findings": [
{
"findingId": "risk#80703",
"accountId": "00000000-000000-0000000-000000",
"products": [
"GWSERVER01"
],
"findingDisplayName": "risk#80703",
"severity": "CRITICAL",
"findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
"findingImpact": "Potential ESXi host crash",
"recommendations": [
"This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
"This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
],
"kbLinkURLs": [
"https://kb.vmware.com/s/article/80703"
],
"recommendationsVCF": [
"This issue is resolved with VMware Cloud Foundation 4.1"
],
"kbLinkURLsVCF": [
"https://kb.vmware.com/s/article/80703"
],
"categoryName": "Storage",
"findingTypes": [
"UPGRADE"
],
"firstObserved": 1629806351877,
"totalAffectedObjectsCount": 12,
"affectedObjects": [
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server01.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server02.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server03.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server04.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server05.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server06.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server07.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
}
]
}
],
"totalRecords": 1,
"timeTaken": 56
}
}
}
{
"data": {
"activeFindings": {
"findings": [
{
"findingId": "risk#80703",
"accountId": "00000000-000000-0000000-000000",
"products": [
"GWSERVER02.corp.contoso.org"
],
"findingDisplayName": "risk#80703",
"severity": "CRITICAL",
"findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
"findingImpact": "Potential ESXi host crash",
"recommendations": [
"This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
"This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
],
"kbLinkURLs": [
"https://kb.vmware.com/s/article/80703"
],
"recommendationsVCF": [
"This issue is resolved with VMware Cloud Foundation 4.1"
],
"kbLinkURLsVCF": [
"https://kb.vmware.com/s/article/80703"
],
"categoryName": "Storage",
"findingTypes": [
"UPGRADE"
],
"firstObserved": 1635968448112,
"totalAffectedObjectsCount": 2,
"affectedObjects": [
{
"sourceName": "GWSERVER02.corp.contoso.org",
"objectName": "server10.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17167734",
"solutionTags": [],
"firstObserved": 1635968448112
},
{
"sourceName": "GWSERVER02.corp.contoso.org",
"objectName": "server11.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17167734",
"solutionTags": [],
"firstObserved": 1635968448112
}
]
}
],
"totalRecords": 1,
"timeTaken": 51
}
}
}
和header如下：
"Finding Id","Issue Description","Risk if no Action Taken","Severity","Recommendations","Source Name","Object Name","Object Type","Host Version","Build","First Observed","Reference"
Header键映射如下：
- 查找 ID = findingId
- 问题描述 = 发现描述
- 不采取行动的风险 = findingImpact
- 严重性=严重性
- 建议=建议
- 来源名称 = 来源名称
- Object 名称 = objectName
- Object类型=对象类型
- 主机版本=版本
- Build = buildNumber
- 首次观察到 = firstObserved
- 参考 = kbLinkURLs
不幸的是，我们必须对每个调查结果和产品执行一次 API 调用（例如：我们无法一次提取所有产品的所有调查结果 - API 不会允许我们执行这样的查询，因此，我们必须进行多次调用才能获得所有结果及其关联的 objects。）
话虽如此，将数据导出到 csv 的首选方法是什么？尽管我们必须循环遍历多个节点，但使用 jq 的 @CSV 是否可行？
任何 help/guidance 将不胜感激。
谢谢！
注1：
chepner
请求的数据集的剥离版本
{
"data": {
"activeFindings": {
"findings": [
{
"findingId": "risk#80703",
"accountId": "00000000-000000-0000000-000000",
"products": [
"GWSERVER01"
],
"findingDisplayName": "risk#80703",
"severity": "CRITICAL",
"findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
"findingImpact": "Potential ESXi host crash",
"recommendations": [
"This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
"This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
],
"kbLinkURLs": [
"https://kb.vmware.com/s/article/80703"
],
"recommendationsVCF": [
"This issue is resolved with VMware Cloud Foundation 4.1"
],
"kbLinkURLsVCF": [
"https://kb.vmware.com/s/article/80703"
],
"categoryName": "Storage",
"findingTypes": [
"UPGRADE"
],
"firstObserved": 1629806351877,
"totalAffectedObjectsCount": 12,
"affectedObjects": [
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server01.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server02.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
]
}
],
"totalRecords": 1,
"timeTaken": 56
}
}
}
{
"data": {
"activeFindings": {
"findings": [
{
"findingId": "risk#80703",
"accountId": "00000000-000000-0000000-000000",
"products": [
"GWSERVER02.corp.contoso.org"
],
"findingDisplayName": "risk#80703",
"severity": "CRITICAL",
"findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
"findingImpact": "Potential ESXi host crash",
"recommendations": [
"This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
"This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
],
"kbLinkURLs": [
"https://kb.vmware.com/s/article/80703"
],
"recommendationsVCF": [
"This issue is resolved with VMware Cloud Foundation 4.1"
],
"kbLinkURLsVCF": [
"https://kb.vmware.com/s/article/80703"
],
"categoryName": "Storage",
"findingTypes": [
"UPGRADE"
],
"firstObserved": 1635968448112,
"totalAffectedObjectsCount": 2,
"affectedObjects": [
{
"sourceName": "GWSERVER02.corp.contoso.org",
"objectName": "server10.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17167734",
"solutionTags": [],
"firstObserved": 1635968448112
},
{
"sourceName": "GWSERVER02.corp.contoso.org",
"objectName": "server11.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17167734",
"solutionTags": [],
"firstObserved": 1635968448112
}
]
}
],
"totalRecords": 1,
"timeTaken": 51
}
}
}
生成的 CSV 文件：
"Finding Id","Issue Description","Risk if no Action Taken","Severity","Recommendations","Source Name","Object Name","Object Type","Host Version","Build","First Observed","Reference"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER01.corp.contoso.org","server01.corp.contoso.org","HostSystem","6.7.0","17499825","1629806351877","https://kb.vmware.com/s/article/80703"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER01.corp.contoso.org","server02.corp.contoso.org","HostSystem","6.7.0","17499825","1629806351877","https://kb.vmware.com/s/article/80703"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER02.corp.contoso.org","server10.corp.contoso.org","HostSystem","6.7.0","17167734","1635968448112","https://kb.vmware.com/s/article/80703"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER02.corp.contoso.org","server11.corp.contoso.org","HostSystem","6.7.0","17167734","1635968448112","https://kb.vmware.com/s/article/80703"
您可以在删除生成的 null 值后的最后一部分应用 @csv通过使用 delpaths([path(.[] | select(.==null))]) 以防止生成连续的冗余逗号，例如
jq -r '.data.activeFindings.findings[]
| [.findingId , .findingDescription, .findingImpact, .severity, (.recommendations | join(",")) , .affectedObjects[].sourceName, .affectedObjects[].objectName, .affectedObjects[].objectType, .affectedObjects[].version, .affectedObjects[].buildNumber, .firstObserved, (.kbLinkURLs | join(",")) ]
| delpaths([path(.[] | select(.==null))])
| @csv'
我会结合 jq 和 spyql，方法如下：
jq -c '.data.activeFindings.findings[]' full_sample.json | spyql "SELECT json->findingId AS 'Finding Id', json->findingDescription AS 'Issue Description', json->findingImpact AS 'Risk if no Action Taken', json->severity AS Severity, ';'.join(json->recommendations) AS Recommendations, json->affectedObjects->sourceName AS 'Source Name', json->affectedObjects->objectName AS 'Object Name', json->affectedObjects->objectType AS 'Object Type', json->affectedObjects->version AS 'Host Version', json->affectedObjects->buildNumber AS Build, json->affectedObjects->firstObserved AS 'First Observed', ';'.join(json->kbLinkURLsVCF) AS Reference FROM json EXPLODE json->affectedObjects TO csv"
Finding Id,Issue Description,Risk if no Action Taken,Severity,Recommendations,Source Name,Object Name,Object Type,Host Version,Build,First Observed,Reference
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server01.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server02.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server03.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server04.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server05.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server06.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server07.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER02.corp.contoso.org,server10.corp.contoso.org,ESX,6.7.0,17167734,1635968448112,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER02.corp.contoso.org,server11.corp.contoso.org,ESX,6.7.0,17167734,1635968448112,https://kb.vmware.com/s/article/80703
我正在使用 jq 提取我们需要的 JSON 部分，同时将输出压缩为 JSON 行（spyql 要求）。然后，spyql 负责剩下的工作，即连接数组（表达式是 python 和一些可选的语法糖）、重命名列并生成 CSV。
如果你想将 firstObserved 时间戳转换为日期时间，你可以这样做（假设 UTC 时间戳）：
$ jq -c '.data.activeFindings.findings[]' full_sample.json | spyql "SELECT json->findingId AS 'Finding Id', json->findingDescription AS 'Issue Description', json->findingImpact AS 'Risk if no Action Taken', json->severity AS Severity, ';'.join(json->recommendations) AS Recommendations, json->affectedObjects->sourceName AS 'Source Name', json->affectedObjects->objectName AS 'Object Name', json->affectedObjects->objectType AS 'Object Type', json->affectedObjects->version AS 'Host Version', json->affectedObjects->buildNumber AS Build, datetime.utcfromtimestamp(json->affectedObjects->firstObserved/1000) AS 'First Observed', ';'.join(json->kbLinkURLsVCF) AS Reference FROM json EXPLODE json->affectedObjects TO csv"
Finding Id,Issue Description,Risk if no Action Taken,Severity,Recommendations,Source Name,Object Name,Object Type,Host Version,Build,First Observed,Reference
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server01.corp.contoso.org,ESX,6.7.0,17499825,2021-08-24 11:59:11.877000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server02.corp.contoso.org,ESX,6.7.0,17499825,2021-08-24 11:59:11.877000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server03.corp.contoso.org,ESX,6.7.0,17499825,2021-08-24 11:59:11.877000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server04.corp.contoso.org,ESX,6.7.0,17499825,2021-08-24 11:59:11.877000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server05.corp.contoso.org,ESX,6.7.0,17499825,2021-08-24 11:59:11.877000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server06.corp.contoso.org,ESX,6.7.0,17499825,2021-08-24 11:59:11.877000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server07.corp.contoso.org,ESX,6.7.0,17499825,2021-08-24 11:59:11.877000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER02.corp.contoso.org,server10.corp.contoso.org,ESX,6.7.0,17167734,2021-11-03 19:40:48.112000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER02.corp.contoso.org,server11.corp.contoso.org,ESX,6.7.0,17167734,2021-11-03 19:40:48.112000,https://kb.vmware.com/s/article/80703
如果您的日期时间不需要毫秒精度，您可以使用整数除法（即 datetime.utcfromtimestamp(json->affectedObjects->firstObserved//1000)）。
免责声明：我是spyql的作者
我有以下数据集，我想将其导出为 CSV：
数据集：
{
"data": {
"activeFindings": {
"findings": [
{
"findingId": "risk#80703",
"accountId": "00000000-000000-0000000-000000",
"products": [
"GWSERVER01"
],
"findingDisplayName": "risk#80703",
"severity": "CRITICAL",
"findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
"findingImpact": "Potential ESXi host crash",
"recommendations": [
"This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
"This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
],
"kbLinkURLs": [
"https://kb.vmware.com/s/article/80703"
],
"recommendationsVCF": [
"This issue is resolved with VMware Cloud Foundation 4.1"
],
"kbLinkURLsVCF": [
"https://kb.vmware.com/s/article/80703"
],
"categoryName": "Storage",
"findingTypes": [
"UPGRADE"
],
"firstObserved": 1629806351877,
"totalAffectedObjectsCount": 12,
"affectedObjects": [
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server01.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server02.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server03.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server04.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server05.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server06.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server07.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
}
]
}
],
"totalRecords": 1,
"timeTaken": 56
}
}
}
{
"data": {
"activeFindings": {
"findings": [
{
"findingId": "risk#80703",
"accountId": "00000000-000000-0000000-000000",
"products": [
"GWSERVER02.corp.contoso.org"
],
"findingDisplayName": "risk#80703",
"severity": "CRITICAL",
"findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
"findingImpact": "Potential ESXi host crash",
"recommendations": [
"This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
"This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
],
"kbLinkURLs": [
"https://kb.vmware.com/s/article/80703"
],
"recommendationsVCF": [
"This issue is resolved with VMware Cloud Foundation 4.1"
],
"kbLinkURLsVCF": [
"https://kb.vmware.com/s/article/80703"
],
"categoryName": "Storage",
"findingTypes": [
"UPGRADE"
],
"firstObserved": 1635968448112,
"totalAffectedObjectsCount": 2,
"affectedObjects": [
{
"sourceName": "GWSERVER02.corp.contoso.org",
"objectName": "server10.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17167734",
"solutionTags": [],
"firstObserved": 1635968448112
},
{
"sourceName": "GWSERVER02.corp.contoso.org",
"objectName": "server11.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17167734",
"solutionTags": [],
"firstObserved": 1635968448112
}
]
}
],
"totalRecords": 1,
"timeTaken": 51
}
}
}
和header如下：
"Finding Id","Issue Description","Risk if no Action Taken","Severity","Recommendations","Source Name","Object Name","Object Type","Host Version","Build","First Observed","Reference"
Header键映射如下：
- 查找 ID = findingId
- 问题描述 = 发现描述
- 不采取行动的风险 = findingImpact
- 严重性=严重性
- 建议=建议
- 来源名称 = 来源名称
- Object 名称 = objectName
- Object类型=对象类型
- 主机版本=版本
- Build = buildNumber
- 首次观察到 = firstObserved
- 参考 = kbLinkURLs
不幸的是，我们必须对每个调查结果和产品执行一次 API 调用（例如：我们无法一次提取所有产品的所有调查结果 - API 不会允许我们执行这样的查询，因此，我们必须进行多次调用才能获得所有结果及其关联的 objects。）
话虽如此，将数据导出到 csv 的首选方法是什么？尽管我们必须循环遍历多个节点，但使用 jq 的 @CSV 是否可行？
任何 help/guidance 将不胜感激。
谢谢！
注1： chepner
请求的数据集的剥离版本{
"data": {
"activeFindings": {
"findings": [
{
"findingId": "risk#80703",
"accountId": "00000000-000000-0000000-000000",
"products": [
"GWSERVER01"
],
"findingDisplayName": "risk#80703",
"severity": "CRITICAL",
"findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
"findingImpact": "Potential ESXi host crash",
"recommendations": [
"This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
"This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
],
"kbLinkURLs": [
"https://kb.vmware.com/s/article/80703"
],
"recommendationsVCF": [
"This issue is resolved with VMware Cloud Foundation 4.1"
],
"kbLinkURLsVCF": [
"https://kb.vmware.com/s/article/80703"
],
"categoryName": "Storage",
"findingTypes": [
"UPGRADE"
],
"firstObserved": 1629806351877,
"totalAffectedObjectsCount": 12,
"affectedObjects": [
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server01.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server02.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
]
}
],
"totalRecords": 1,
"timeTaken": 56
}
}
}
{
"data": {
"activeFindings": {
"findings": [
{
"findingId": "risk#80703",
"accountId": "00000000-000000-0000000-000000",
"products": [
"GWSERVER02.corp.contoso.org"
],
"findingDisplayName": "risk#80703",
"severity": "CRITICAL",
"findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
"findingImpact": "Potential ESXi host crash",
"recommendations": [
"This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
"This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
],
"kbLinkURLs": [
"https://kb.vmware.com/s/article/80703"
],
"recommendationsVCF": [
"This issue is resolved with VMware Cloud Foundation 4.1"
],
"kbLinkURLsVCF": [
"https://kb.vmware.com/s/article/80703"
],
"categoryName": "Storage",
"findingTypes": [
"UPGRADE"
],
"firstObserved": 1635968448112,
"totalAffectedObjectsCount": 2,
"affectedObjects": [
{
"sourceName": "GWSERVER02.corp.contoso.org",
"objectName": "server10.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17167734",
"solutionTags": [],
"firstObserved": 1635968448112
},
{
"sourceName": "GWSERVER02.corp.contoso.org",
"objectName": "server11.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17167734",
"solutionTags": [],
"firstObserved": 1635968448112
}
]
}
],
"totalRecords": 1,
"timeTaken": 51
}
}
}
生成的 CSV 文件：
"Finding Id","Issue Description","Risk if no Action Taken","Severity","Recommendations","Source Name","Object Name","Object Type","Host Version","Build","First Observed","Reference"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER01.corp.contoso.org","server01.corp.contoso.org","HostSystem","6.7.0","17499825","1629806351877","https://kb.vmware.com/s/article/80703"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER01.corp.contoso.org","server02.corp.contoso.org","HostSystem","6.7.0","17499825","1629806351877","https://kb.vmware.com/s/article/80703"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER02.corp.contoso.org","server10.corp.contoso.org","HostSystem","6.7.0","17167734","1635968448112","https://kb.vmware.com/s/article/80703"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER02.corp.contoso.org","server11.corp.contoso.org","HostSystem","6.7.0","17167734","1635968448112","https://kb.vmware.com/s/article/80703"
您可以在删除生成的 null 值后的最后一部分应用 @csv通过使用 delpaths([path(.[] | select(.==null))]) 以防止生成连续的冗余逗号，例如
jq -r '.data.activeFindings.findings[]
| [.findingId , .findingDescription, .findingImpact, .severity, (.recommendations | join(",")) , .affectedObjects[].sourceName, .affectedObjects[].objectName, .affectedObjects[].objectType, .affectedObjects[].version, .affectedObjects[].buildNumber, .firstObserved, (.kbLinkURLs | join(",")) ]
| delpaths([path(.[] | select(.==null))])
| @csv'
我会结合 jq 和 spyql，方法如下：
jq -c '.data.activeFindings.findings[]' full_sample.json | spyql "SELECT json->findingId AS 'Finding Id', json->findingDescription AS 'Issue Description', json->findingImpact AS 'Risk if no Action Taken', json->severity AS Severity, ';'.join(json->recommendations) AS Recommendations, json->affectedObjects->sourceName AS 'Source Name', json->affectedObjects->objectName AS 'Object Name', json->affectedObjects->objectType AS 'Object Type', json->affectedObjects->version AS 'Host Version', json->affectedObjects->buildNumber AS Build, json->affectedObjects->firstObserved AS 'First Observed', ';'.join(json->kbLinkURLsVCF) AS Reference FROM json EXPLODE json->affectedObjects TO csv"
Finding Id,Issue Description,Risk if no Action Taken,Severity,Recommendations,Source Name,Object Name,Object Type,Host Version,Build,First Observed,Reference
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server01.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server02.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server03.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server04.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server05.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server06.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server07.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER02.corp.contoso.org,server10.corp.contoso.org,ESX,6.7.0,17167734,1635968448112,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER02.corp.contoso.org,server11.corp.contoso.org,ESX,6.7.0,17167734,1635968448112,https://kb.vmware.com/s/article/80703
我正在使用 jq 提取我们需要的 JSON 部分，同时将输出压缩为 JSON 行（spyql 要求）。然后，spyql 负责剩下的工作，即连接数组（表达式是 python 和一些可选的语法糖）、重命名列并生成 CSV。
如果你想将 firstObserved 时间戳转换为日期时间，你可以这样做（假设 UTC 时间戳）：
$ jq -c '.data.activeFindings.findings[]' full_sample.json | spyql "SELECT json->findingId AS 'Finding Id', json->findingDescription AS 'Issue Description', json->findingImpact AS 'Risk if no Action Taken', json->severity AS Severity, ';'.join(json->recommendations) AS Recommendations, json->affectedObjects->sourceName AS 'Source Name', json->affectedObjects->objectName AS 'Object Name', json->affectedObjects->objectType AS 'Object Type', json->affectedObjects->version AS 'Host Version', json->affectedObjects->buildNumber AS Build, datetime.utcfromtimestamp(json->affectedObjects->firstObserved/1000) AS 'First Observed', ';'.join(json->kbLinkURLsVCF) AS Reference FROM json EXPLODE json->affectedObjects TO csv"
Finding Id,Issue Description,Risk if no Action Taken,Severity,Recommendations,Source Name,Object Name,Object Type,Host Version,Build,First Observed,Reference
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server01.corp.contoso.org,ESX,6.7.0,17499825,2021-08-24 11:59:11.877000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server02.corp.contoso.org,ESX,6.7.0,17499825,2021-08-24 11:59:11.877000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server03.corp.contoso.org,ESX,6.7.0,17499825,2021-08-24 11:59:11.877000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server04.corp.contoso.org,ESX,6.7.0,17499825,2021-08-24 11:59:11.877000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server05.corp.contoso.org,ESX,6.7.0,17499825,2021-08-24 11:59:11.877000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server06.corp.contoso.org,ESX,6.7.0,17499825,2021-08-24 11:59:11.877000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server07.corp.contoso.org,ESX,6.7.0,17499825,2021-08-24 11:59:11.877000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER02.corp.contoso.org,server10.corp.contoso.org,ESX,6.7.0,17167734,2021-11-03 19:40:48.112000,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER02.corp.contoso.org,server11.corp.contoso.org,ESX,6.7.0,17167734,2021-11-03 19:40:48.112000,https://kb.vmware.com/s/article/80703
如果您的日期时间不需要毫秒精度，您可以使用整数除法（即 datetime.utcfromtimestamp(json->affectedObjects->firstObserved//1000)）。
免责声明：我是spyql的作者