从 json 对象中过滤掉所有出现的给定属性
Filter out all occurrences of given properties from json object
我正在使用 PowerShell 从 API 调用中提取数据，对其进行更新，然后将其传递回 API。
我想知道是否有一种简单的方法可以修改JSON对象，过滤掉所有不需要的属性在 JSON 结构中的任何位置？
我尝试了以下方法，但是结果 JSON 只删除了 最低级别 属性（即“p2”）
$example = ConvertFrom-Json '{"a":{"p1": "value1"},"p2": "value2", "b":"valueb"}'
$exclude = "p1", "p2"
$clean = $example | Select-Object -Property * -ExcludeProperty $exclude
ConvertTo-Json $clean -Compress
结果 => {"a":{"p1":"value1"},"b":"valueb"}
我想删除所有 $exlude 个条目，无论它们位于 JSON 中的哪个位置。有没有简单的解决办法？
更新
这是另一个（更复杂的）JSON例子：
{
"a": {
"p1": "value 1",
"c": "value c",
"d": {
"e": "value e",
"p2": "value 3"
},
"f": [
{
"g": "value ga",
"p1": "value 4a"
},
{
"g": "value gb",
"p1": "value 4b"
}
]
},
"p2": "value 2",
"b": "value b"
}
预期结果（删除所有 p1 和 p2 键）：
{
"a": {
"c": "value c",
"d": {
"e": "value e"
},
"f": [
{
"g": "value ga"
},
{
"g": "value gb"
}
]
},
"b": "value b"
}
不幸的是，似乎没有容易的方法。事实证明，正确处理数组非常具有挑战性。我的方法是递归展开输入 (JSON) 对象，包括任何数组，这样我们就可以轻松应用过滤，然后从过滤后的属性构建一个新对象。
第一步和第三步包含在以下可重用的辅助函数中，一个用于展开 (ConvertTo-FlatObjectValues)，一个用于重建对象 (ConvertFrom-FlatObjectValues)。还有第三个函数（ConvertFrom-TreeHashTablesToArrays），但它仅供 ConvertFrom-FlatObjectValues.
内部使用
Function ConvertTo-FlatObjectValues {
<#
.SYNOPSIS
Unrolls a nested PSObject/PSCustomObject "property bag".
.DESCRIPTION
Unrolls a nested PSObject/PSCustomObject "property bag" such as created by ConvertFrom-Json into flat objects consisting of path, name and value.
Fully supports arrays at the root as well as for properties and nested arrays.
#>
[CmdletBinding()]
param (
[Parameter(Mandatory, ValueFromPipeline)] $InputObject,
[string] $Separator = '.',
[switch] $KeepEmptyObjects,
[switch] $KeepEmptyArrays,
[string] $Path, # Internal parameter for recursion.
[string] $Name # Internal parameter for recursion.
)
process {
if( $InputObject -is [System.Collections.IList] ) {
if( $KeepEmptyArrays ) {
# Output a special item to keep empty array.
[PSCustomObject]@{
Path = ($Path, "#").Where{ $_ } -join $Separator
Name = $Name
Value = $null
}
}
$i = 0
$InputObject.ForEach{
# Recursively unroll array elements.
$childPath = ($Path, "#$i").Where{ $_ } -join $Separator
ConvertTo-FlatObjectValues -InputObject $_ -Path $childPath -Name $Name `
-Separator $Separator -KeepEmptyObjects:$KeepEmptyObjects -KeepEmptyArrays:$KeepEmptyArrays
$i++
}
}
elseif( $InputObject -is [PSObject] ) {
if( $KeepEmptyObjects ) {
# Output a special item to keep empty object.
[PSCustomObject]@{
Path = $Path
Name = $Name
Value = [ordered] @{}
}
}
$InputObject.PSObject.Properties.ForEach{
# Recursively unroll object properties.
$childPath = ($Path, $_.Name).Where{ $_ } -join $Separator
ConvertTo-FlatObjectValues -InputObject $_.Value -Path $childPath -Name $_.Name `
-Separator $Separator -KeepEmptyObjects:$KeepEmptyObjects -KeepEmptyArrays:$KeepEmptyArrays
}
}
else {
# Output scalar
[PSCustomObject]@{
Path = $Path
Name = $Name
Value = $InputObject
}
}
}
}
function ConvertFrom-FlatObjectValues {
<#
.SYNOPSIS
Convert a flat list consisting of path and value into tree(s) of PSCustomObject.
.DESCRIPTION
Convert a flat list consisting of path and value, such as generated by ConvertTo-FlatObjectValues, into tree(s) of PSCustomObject.
The output can either be an array (not unrolled) or a PSCustomObject, depending on the structure of the input data.
#>
[CmdletBinding()]
param (
[Parameter(Mandatory, ValueFromPipelineByPropertyName)] [string] $Path,
[Parameter(Mandatory, ValueFromPipelineByPropertyName)] [AllowNull()] $Value,
[Parameter()] [string] $Separator = '.'
)
begin {
$tree = [ordered]@{}
}
process {
# At first store everything (including array elements) into hashtables.
$branch = $Tree
do {
# Split path into root key and path remainder.
$key, $path = $path.Split( $Separator, 2 )
if( $path ) {
# We have multiple path components, so we may have to create nested hash table.
if( -not $branch.Contains( $key ) ) {
$branch[ $key ] = [ordered] @{}
}
# Enter sub tree.
$branch = $branch[ $key ]
}
else {
# We have arrived at the leaf -> set its value
$branch[ $key ] = $value
}
}
while( $path )
}
end {
# So far we have stored the original arrays as hashtables with keys like '#0', '#1', ... (possibly non-consecutive).
# Now convert these hashtables back into actual arrays and generate PSCustomObject's from the remaining hashtables.
ConvertFrom-TreeHashTablesToArrays $tree
}
}
Function ConvertFrom-TreeHashTablesToArrays {
<#
.SYNOPSIS
Internal function called by ConvertFrom-FlatObjectValues.
.DESCRIPTION
- Converts arrays stored as hashtables into actual arrays.
- Converts any remaining hashtables into PSCustomObject's.
#>
[CmdletBinding()]
param (
[Parameter(Mandatory, ValueFromPipeline)] [Collections.IDictionary] $InputObject
)
process {
# Check if $InputObject has been generated from an array.
$isArray = foreach( $key in $InputObject.Keys ) { $key.StartsWith('#'); break }
if( $isArray ) {
# Sort array indices as they might be unordered. A single '#' as key will be skipped, because it denotes an empty array.
$sortedByKeyNumeric = $InputObject.GetEnumerator().Where{ $_.Key -ne '#' } |
Sort-Object { [int]::Parse( $_.Key.SubString( 1 ) ) }
$outArray = $sortedByKeyNumeric.ForEach{
if( $_.Value -is [Collections.IDictionary] ) {
# Recursion. Output array element will either be an object or a nested array.
ConvertFrom-TreeHashTablesToArrays $_.Value
}
else {
# Output array element is a scalar value.
$_.Value
}
}
, $outArray # Comma-operator prevents unrolling of the array, to support nested arrays.
}
else {
# $InputObject has been generated from an object. Copy it to $outProps recursively and output as PSCustomObject.
$outProps = [ordered] @{}
$InputObject.GetEnumerator().ForEach{
$outProps[ $_.Key ] = if( $_.Value -is [Collections.IDictionary] ) {
# Recursion. Output property will either be an object or an array.
ConvertFrom-TreeHashTablesToArrays $_.Value
}
else {
# Output property is a scalar value.
$_.Value
}
}
[PSCustomObject] $outProps
}
}
}
用法示例：
$example = ConvertFrom-Json @'
{
"a": {
"p1": "value 1",
"c": "value c",
"d": {
"e": "value e",
"p2": "value 3"
},
"f": [
{
"g": "value ga",
"p1": "value 4a"
},
{
"g": "value gb",
"p1": "value 4b"
}
]
},
"p2": "value 2",
"b": "value b"
}
'@
$exclude = "p1", "p2"
$clean = ConvertTo-FlatObjectValues $example | # Step 1: unroll properties
Where-Object Name -notin $exclude | # Step 2: filter
ConvertFrom-FlatObjectValues # Step 3: rebuild object
$clean | ConvertTo-Json -Depth 9
输出：
{
"a": {
"c": "value c",
"d": {
"e": "value e"
},
"f": [
{
"g": "value ga"
},
{
"g": "value gb"
}
]
},
"b": "value b"
}
使用注意事项：
- 如果子对象在过滤后不包含任何属性，则会被删除。空数组也被删除。您可以通过将
-KeepEmptyObjects and/or -KeepEmptyArrays 传递给函数 ConvertTo-FlatObjectValues. 来防止这种情况
- 如果输入 JSON 是根级别的数组，请确保将其作为参数传递给
ConvertTo-FlatObjectValues，而不是通过管道传递它（这会展开它并且函数不会不再知道它是一个数组）。
- 也可以对属性的整个路径进行过滤。例如要仅在
a 对象中删除 P1 属性，您可以编写 Where-Object Path -ne a.p1。要查看路径的外观，只需调用 ConvertTo-FlatObjectValues $example 即可输出属性和数组元素的平面列表：
Path Name Value
---- ---- -----
a.p1 p1 value 1
a.c c value c
a.d.e e value e
a.d.p2 p2 value 3
a.f.#0.g g value ga
a.f.#0.p1 p1 value 4a
a.f.#1.g g value gb
a.f.#1.p1 p1 value 4b
p2 p2 value 2
b b value b
执行说明：
在展开期间 ConvertTo-FlatObjectValues 为数组元素创建单独的路径段（键），看起来像“#n”，其中 n 是数组索引。这允许我们在 ConvertFrom-FlatObjectValues.
中重建对象时更统一地对待数组和对象
ConvertFrom-FlatObjectValues 首先在其 process 部分为所有对象和数组创建嵌套哈希表。这使得将属性重新收集到它们各自的对象中变得容易。在这部分代码中仍然没有对数组进行特殊处理。中间结果现在看起来像这样：
{
"a": {
"c": "value c",
"d": {
"e": "value e"
},
"f": {
"#0": {
"g": "value ga"
},
"#1": {
"g": "value gb"
}
}
},
"b": "value b"
}
只有在ConvertFrom-FlatObjectValues的end部分，数组是从哈希表重建的，这是由函数ConvertFrom-TreeHashTablesToArrays完成的。它将具有以“#”开头的键的哈希表变回实际数组。由于过滤，索引可能是 non-consecutive，所以我们可以只收集值而忽略索引。虽然对于给定的用例不是必需的，但数组索引将被排序以使函数更健壮并支持以任何顺序接收的索引。
PowerShell 函数中的递归相对较慢，因为 parameter-binding 开销。如果性能最重要，代码应该用内联 C# 重写或使用像 Collections.Queue 这样的数据结构来避免递归（以牺牲代码可读性为代价）。
我正在使用 PowerShell 从 API 调用中提取数据，对其进行更新，然后将其传递回 API。
我想知道是否有一种简单的方法可以修改JSON对象，过滤掉所有不需要的属性在 JSON 结构中的任何位置？
我尝试了以下方法，但是结果 JSON 只删除了 最低级别 属性（即“p2”）
$example = ConvertFrom-Json '{"a":{"p1": "value1"},"p2": "value2", "b":"valueb"}'
$exclude = "p1", "p2"
$clean = $example | Select-Object -Property * -ExcludeProperty $exclude
ConvertTo-Json $clean -Compress
结果 => {"a":{"p1":"value1"},"b":"valueb"}
我想删除所有 $exlude 个条目，无论它们位于 JSON 中的哪个位置。有没有简单的解决办法？
更新
这是另一个（更复杂的）JSON例子：
{
"a": {
"p1": "value 1",
"c": "value c",
"d": {
"e": "value e",
"p2": "value 3"
},
"f": [
{
"g": "value ga",
"p1": "value 4a"
},
{
"g": "value gb",
"p1": "value 4b"
}
]
},
"p2": "value 2",
"b": "value b"
}
预期结果（删除所有 p1 和 p2 键）：
{
"a": {
"c": "value c",
"d": {
"e": "value e"
},
"f": [
{
"g": "value ga"
},
{
"g": "value gb"
}
]
},
"b": "value b"
}
不幸的是，似乎没有容易的方法。事实证明，正确处理数组非常具有挑战性。我的方法是递归展开输入 (JSON) 对象，包括任何数组，这样我们就可以轻松应用过滤，然后从过滤后的属性构建一个新对象。
第一步和第三步包含在以下可重用的辅助函数中，一个用于展开 (ConvertTo-FlatObjectValues)，一个用于重建对象 (ConvertFrom-FlatObjectValues)。还有第三个函数（ConvertFrom-TreeHashTablesToArrays），但它仅供 ConvertFrom-FlatObjectValues.
Function ConvertTo-FlatObjectValues {
<#
.SYNOPSIS
Unrolls a nested PSObject/PSCustomObject "property bag".
.DESCRIPTION
Unrolls a nested PSObject/PSCustomObject "property bag" such as created by ConvertFrom-Json into flat objects consisting of path, name and value.
Fully supports arrays at the root as well as for properties and nested arrays.
#>
[CmdletBinding()]
param (
[Parameter(Mandatory, ValueFromPipeline)] $InputObject,
[string] $Separator = '.',
[switch] $KeepEmptyObjects,
[switch] $KeepEmptyArrays,
[string] $Path, # Internal parameter for recursion.
[string] $Name # Internal parameter for recursion.
)
process {
if( $InputObject -is [System.Collections.IList] ) {
if( $KeepEmptyArrays ) {
# Output a special item to keep empty array.
[PSCustomObject]@{
Path = ($Path, "#").Where{ $_ } -join $Separator
Name = $Name
Value = $null
}
}
$i = 0
$InputObject.ForEach{
# Recursively unroll array elements.
$childPath = ($Path, "#$i").Where{ $_ } -join $Separator
ConvertTo-FlatObjectValues -InputObject $_ -Path $childPath -Name $Name `
-Separator $Separator -KeepEmptyObjects:$KeepEmptyObjects -KeepEmptyArrays:$KeepEmptyArrays
$i++
}
}
elseif( $InputObject -is [PSObject] ) {
if( $KeepEmptyObjects ) {
# Output a special item to keep empty object.
[PSCustomObject]@{
Path = $Path
Name = $Name
Value = [ordered] @{}
}
}
$InputObject.PSObject.Properties.ForEach{
# Recursively unroll object properties.
$childPath = ($Path, $_.Name).Where{ $_ } -join $Separator
ConvertTo-FlatObjectValues -InputObject $_.Value -Path $childPath -Name $_.Name `
-Separator $Separator -KeepEmptyObjects:$KeepEmptyObjects -KeepEmptyArrays:$KeepEmptyArrays
}
}
else {
# Output scalar
[PSCustomObject]@{
Path = $Path
Name = $Name
Value = $InputObject
}
}
}
}
function ConvertFrom-FlatObjectValues {
<#
.SYNOPSIS
Convert a flat list consisting of path and value into tree(s) of PSCustomObject.
.DESCRIPTION
Convert a flat list consisting of path and value, such as generated by ConvertTo-FlatObjectValues, into tree(s) of PSCustomObject.
The output can either be an array (not unrolled) or a PSCustomObject, depending on the structure of the input data.
#>
[CmdletBinding()]
param (
[Parameter(Mandatory, ValueFromPipelineByPropertyName)] [string] $Path,
[Parameter(Mandatory, ValueFromPipelineByPropertyName)] [AllowNull()] $Value,
[Parameter()] [string] $Separator = '.'
)
begin {
$tree = [ordered]@{}
}
process {
# At first store everything (including array elements) into hashtables.
$branch = $Tree
do {
# Split path into root key and path remainder.
$key, $path = $path.Split( $Separator, 2 )
if( $path ) {
# We have multiple path components, so we may have to create nested hash table.
if( -not $branch.Contains( $key ) ) {
$branch[ $key ] = [ordered] @{}
}
# Enter sub tree.
$branch = $branch[ $key ]
}
else {
# We have arrived at the leaf -> set its value
$branch[ $key ] = $value
}
}
while( $path )
}
end {
# So far we have stored the original arrays as hashtables with keys like '#0', '#1', ... (possibly non-consecutive).
# Now convert these hashtables back into actual arrays and generate PSCustomObject's from the remaining hashtables.
ConvertFrom-TreeHashTablesToArrays $tree
}
}
Function ConvertFrom-TreeHashTablesToArrays {
<#
.SYNOPSIS
Internal function called by ConvertFrom-FlatObjectValues.
.DESCRIPTION
- Converts arrays stored as hashtables into actual arrays.
- Converts any remaining hashtables into PSCustomObject's.
#>
[CmdletBinding()]
param (
[Parameter(Mandatory, ValueFromPipeline)] [Collections.IDictionary] $InputObject
)
process {
# Check if $InputObject has been generated from an array.
$isArray = foreach( $key in $InputObject.Keys ) { $key.StartsWith('#'); break }
if( $isArray ) {
# Sort array indices as they might be unordered. A single '#' as key will be skipped, because it denotes an empty array.
$sortedByKeyNumeric = $InputObject.GetEnumerator().Where{ $_.Key -ne '#' } |
Sort-Object { [int]::Parse( $_.Key.SubString( 1 ) ) }
$outArray = $sortedByKeyNumeric.ForEach{
if( $_.Value -is [Collections.IDictionary] ) {
# Recursion. Output array element will either be an object or a nested array.
ConvertFrom-TreeHashTablesToArrays $_.Value
}
else {
# Output array element is a scalar value.
$_.Value
}
}
, $outArray # Comma-operator prevents unrolling of the array, to support nested arrays.
}
else {
# $InputObject has been generated from an object. Copy it to $outProps recursively and output as PSCustomObject.
$outProps = [ordered] @{}
$InputObject.GetEnumerator().ForEach{
$outProps[ $_.Key ] = if( $_.Value -is [Collections.IDictionary] ) {
# Recursion. Output property will either be an object or an array.
ConvertFrom-TreeHashTablesToArrays $_.Value
}
else {
# Output property is a scalar value.
$_.Value
}
}
[PSCustomObject] $outProps
}
}
}
用法示例：
$example = ConvertFrom-Json @'
{
"a": {
"p1": "value 1",
"c": "value c",
"d": {
"e": "value e",
"p2": "value 3"
},
"f": [
{
"g": "value ga",
"p1": "value 4a"
},
{
"g": "value gb",
"p1": "value 4b"
}
]
},
"p2": "value 2",
"b": "value b"
}
'@
$exclude = "p1", "p2"
$clean = ConvertTo-FlatObjectValues $example | # Step 1: unroll properties
Where-Object Name -notin $exclude | # Step 2: filter
ConvertFrom-FlatObjectValues # Step 3: rebuild object
$clean | ConvertTo-Json -Depth 9
输出：
{
"a": {
"c": "value c",
"d": {
"e": "value e"
},
"f": [
{
"g": "value ga"
},
{
"g": "value gb"
}
]
},
"b": "value b"
}
使用注意事项：
- 如果子对象在过滤后不包含任何属性，则会被删除。空数组也被删除。您可以通过将
-KeepEmptyObjectsand/or-KeepEmptyArrays传递给函数ConvertTo-FlatObjectValues. 来防止这种情况
- 如果输入 JSON 是根级别的数组，请确保将其作为参数传递给
ConvertTo-FlatObjectValues，而不是通过管道传递它（这会展开它并且函数不会不再知道它是一个数组）。 - 也可以对属性的整个路径进行过滤。例如要仅在
a对象中删除P1属性，您可以编写Where-Object Path -ne a.p1。要查看路径的外观，只需调用ConvertTo-FlatObjectValues $example即可输出属性和数组元素的平面列表：Path Name Value ---- ---- ----- a.p1 p1 value 1 a.c c value c a.d.e e value e a.d.p2 p2 value 3 a.f.#0.g g value ga a.f.#0.p1 p1 value 4a a.f.#1.g g value gb a.f.#1.p1 p1 value 4b p2 p2 value 2 b b value b
执行说明：
在展开期间
中重建对象时更统一地对待数组和对象ConvertTo-FlatObjectValues为数组元素创建单独的路径段（键），看起来像“#n”，其中 n 是数组索引。这允许我们在ConvertFrom-FlatObjectValues.ConvertFrom-FlatObjectValues首先在其process部分为所有对象和数组创建嵌套哈希表。这使得将属性重新收集到它们各自的对象中变得容易。在这部分代码中仍然没有对数组进行特殊处理。中间结果现在看起来像这样：{ "a": { "c": "value c", "d": { "e": "value e" }, "f": { "#0": { "g": "value ga" }, "#1": { "g": "value gb" } } }, "b": "value b" }只有在
ConvertFrom-FlatObjectValues的end部分，数组是从哈希表重建的，这是由函数ConvertFrom-TreeHashTablesToArrays完成的。它将具有以“#”开头的键的哈希表变回实际数组。由于过滤，索引可能是 non-consecutive，所以我们可以只收集值而忽略索引。虽然对于给定的用例不是必需的，但数组索引将被排序以使函数更健壮并支持以任何顺序接收的索引。PowerShell 函数中的递归相对较慢，因为 parameter-binding 开销。如果性能最重要，代码应该用内联 C# 重写或使用像
Collections.Queue这样的数据结构来避免递归（以牺牲代码可读性为代价）。