在 JSON 上应用掩码以仅保留强制性数据
Apply a mask on a JSON to keep only mandatory data
我有一个 API 需要一些潜在的强制数据来创建一个临时会话：
e.g.: at first, my POST /signup endpoint user need to send me the following data:
{
"customer": {
"age": 21,
"ssn": "000 00 0000",
"address": {
"street": "Customer St.",
"phone": "+66 444 333 222"
}
}
}
Let's call it JSON a.
另一方面，我有一些法律合作伙伴需要一些这些数据，但不是全部:
e.g.:
{
"customer": {
"ssn": "The SSN is mandatory to register against XXX Company",
"address": {
"phone": "XXX Company will send a text message to validate your registration"
}
}
}
Let's call it JSON b.
由于最近的法律限制，在我的信息系统中，我必须仅保留用户他选择的工作流程[=51]必须携带的强制性数据=].
因此我的问题：是否有一个函数（内置于 Jackson 或其他一些 JSON 处理库或您推荐的算法）我可以这样应用给定 JSON b 和 JSON a，它将输出以下 JSON:
{
"customer": {
"ssn": "000 00 0000",
"address": {
"phone": "+66 444 333 222"
}
}
}
思考了一下，我发现了一些可能的解决方案：
- 合并
JSON a 与 JSON b，如果发生冲突则取 JSON b 值，将结果命名为 JSON c
- 在
JSON c 和 JSON a 之间进行比较（丢弃等于数据），如果发生冲突，取 JSON a 个值
我知道使用 com.fasterxml.jackson.databind.ObjectMapper#readerForUpdating 可以使用 Jackson 合并两个 JSON，所以我的问题可以简化为：有没有办法在两个 JSON并给出冲突解决函数?
感谢 https://github.com/algesten/jsondiff which gave me the keywords to find a more maintained https://github.com/java-json-tools/json-patch（能够使用您自己的 ObjectMapper），我找到了答案（参见 mask() 方法）：
import static org.assertj.core.api.AssertionsForClassTypes.assertThat;
import com.fasterxml.jackson.annotation.JsonMerge;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.github.fge.jsonpatch.diff.JsonDiff;
import java.math.BigInteger;
import lombok.AccessLevel;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.EqualsAndHashCode;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.SneakyThrows;
import lombok.ToString;
import org.junit.jupiter.api.Test;
class JsonPatchTest {
private static final ObjectMapper mapper = new ObjectMapper();
@Getter
@Builder
@ToString
@EqualsAndHashCode
@NoArgsConstructor(access = AccessLevel.PRIVATE)
@AllArgsConstructor(access = AccessLevel.PRIVATE)
public static class Data {
@JsonMerge Customer customer;
@Getter
@Builder
@ToString
@EqualsAndHashCode
@NoArgsConstructor(access = AccessLevel.PRIVATE)
@AllArgsConstructor(access = AccessLevel.PRIVATE)
public static class Customer {
@JsonMerge BigInteger age;
@JsonMerge String ssn;
@JsonMerge Address address;
@Getter
@Builder
@ToString
@EqualsAndHashCode
@NoArgsConstructor(access = AccessLevel.PRIVATE)
@AllArgsConstructor(access = AccessLevel.PRIVATE)
public static class Address {
@JsonMerge String street;
@JsonMerge String phone;
}
}
@SneakyThrows
Data merge(Data parent) {
var originCopyAsString = mapper.writerFor(this.getClass()).writeValueAsString(this);
var parentAsString = mapper.writerFor(this.getClass()).writeValueAsString(parent);
var parentCopy = mapper.readerFor(this.getClass()).readValue(parentAsString);
var clone = mapper.readerForUpdating(parentCopy).readValue(originCopyAsString);
return (Data) clone;
}
}
@SneakyThrows
@Test
void mask() {
final var diff = JsonDiff.asJsonPatch(mapper.readTree(jsonC()), mapper.readTree(jsonB()));
final var masked = diff.apply(mapper.readTree(jsonA())).toPrettyString();
assertThat(masked).isEqualToIgnoringWhitespace(masked());
}
private String jsonA() {
return "{\n"
+ " \"customer\": {\n"
+ " \"age\": 21,\n"
+ " \"ssn\": \"000 00 0000\",\n"
+ " \"address\": {\n"
+ " \"street\": \"Customer St.\",\n"
+ " \"phone\": \"+66 444 333 222\"\n"
+ " }\n"
+ " }\n"
+ "}";
}
private String jsonB() {
return "{\n"
+ " \"customer\": {\n"
+ " \"ssn\": \"The SSN is mandatory to register against XXX Company\",\n"
+ " \"address\": {\n"
+ " \"phone\": \"XXX Company will send a text message to validate your registration\"\n"
+ " }\n"
+ " }\n"
+ "}";
}
@SneakyThrows
private String jsonC() {
final Data dataA = mapper.readerFor(Data.class).readValue(jsonA());
final Data dataB = mapper.readerFor(Data.class).readValue(jsonB());
final Data merged = dataB.merge(dataA);
return mapper.writerFor(Data.class).writeValueAsString(merged);
}
@SneakyThrows
private String masked() {
return "{\n"
+ " \"customer\": {\n"
+ " \"ssn\": \"000 00 0000\",\n"
+ " \"address\": {\n"
+ " \"phone\": \"+66 444 333 222\"\n"
+ " }\n"
+ " }\n"
+ "}";
}
}
我建议使用 token/event/stream-based 解决方案。下面只是使用tinyparser/generator libhttps://github.com/anatolygudkov/green-jelly（Gson和Jackson都提供了面向流的API）的说明：
import org.green.jelly.AppendableWriter;
import org.green.jelly.JsonEventPump;
import org.green.jelly.JsonNumber;
import org.green.jelly.JsonParser;
import java.io.StringWriter;
import java.io.Writer;
import java.util.Arrays;
import java.util.HashSet;
import java.util.Set;
public class FilterMyJson {
private static final String jsonToFilter = "{\n" +
" \"customer\": {\n" +
" \"age\": 21,\n" +
" \"ssn\": \"000 00 0000\",\n" +
" \"address\": {\n" +
" \"street\": \"Customer St.\",\n" +
" \"phone\": \"+66 444 333 222\"\n" +
" }\n" +
" }\n" +
"}";
public static void main(String[] args) {
final StringWriter result = new StringWriter();
final JsonParser parser = new JsonParser();
parser.setListener(new MyJsonFilter(result, "age", "street"));
parser.parse(jsonToFilter); // if you read a file with a buffer,
// call parse() several times part by part in a loop until EOF
parser.eoj(); // and then call .eoj()
System.out.println(result);
}
static class MyJsonFilter extends JsonEventPump {
private final Set<String> objectMembersToFilter;
private boolean currentObjectMemberIsAllowed;
MyJsonFilter(final Writer output, final String... objectMembersToFilter) {
super(new AppendableWriter<>(output));
this.objectMembersToFilter = new HashSet<>(Arrays.asList(objectMembersToFilter));
}
@Override
public boolean onObjectMember(final CharSequence name) {
currentObjectMemberIsAllowed =
!objectMembersToFilter.contains(name.toString());
return super.onObjectMember(name);
}
@Override
public boolean onStringValue(final CharSequence data) {
if (!currentObjectMemberIsAllowed) {
return true;
}
return super.onStringValue(data);
}
@Override
public boolean onNumberValue(final JsonNumber number) {
if (!currentObjectMemberIsAllowed) {
return true;
}
return super.onNumberValue(number);
}
}
}
打印：
{
"customer":
{
"ssn": "000 00 0000",
"address":
{
"phone": "+66 444 333 222"
}
}
}
代码非常简单。现在它只过滤掉字符串和数字标量。不支持对象层次结构。因此，您可能需要针对某些情况改进代码。
此类解决方案的道具：
- file/data 不需要完全加载到内存中，你
可以毫无问题地处理 megs/gigs
- 它工作得更快，尤其是对于大文件
- 使用此模式很容易实现任何自定义 type/rule 转换。例如，您的过滤器很容易被参数化，您不必每次更改数据结构时都重新编译代码
我有一个 API 需要一些潜在的强制数据来创建一个临时会话：
e.g.: at first, my
POST /signupendpoint user need to send me the following data:{ "customer": { "age": 21, "ssn": "000 00 0000", "address": { "street": "Customer St.", "phone": "+66 444 333 222" } } }Let's call it
JSON a.
另一方面，我有一些法律合作伙伴需要一些这些数据，但不是全部:
e.g.:
{ "customer": { "ssn": "The SSN is mandatory to register against XXX Company", "address": { "phone": "XXX Company will send a text message to validate your registration" } } }Let's call it
JSON b.
由于最近的法律限制，在我的信息系统中，我必须仅保留用户他选择的工作流程[=51]必须携带的强制性数据=].
因此我的问题：是否有一个函数（内置于 Jackson 或其他一些 JSON 处理库或您推荐的算法）我可以这样应用给定 JSON b 和 JSON a，它将输出以下 JSON:
{
"customer": {
"ssn": "000 00 0000",
"address": {
"phone": "+66 444 333 222"
}
}
}
思考了一下，我发现了一些可能的解决方案：
- 合并
JSON a与JSON b，如果发生冲突则取JSON b值，将结果命名为JSON c - 在
JSON c和JSON a之间进行比较（丢弃等于数据），如果发生冲突，取JSON a个值
我知道使用 com.fasterxml.jackson.databind.ObjectMapper#readerForUpdating 可以使用 Jackson 合并两个 JSON，所以我的问题可以简化为：有没有办法在两个 JSON并给出冲突解决函数?
感谢 https://github.com/algesten/jsondiff which gave me the keywords to find a more maintained https://github.com/java-json-tools/json-patch（能够使用您自己的 ObjectMapper），我找到了答案（参见 mask() 方法）：
import static org.assertj.core.api.AssertionsForClassTypes.assertThat;
import com.fasterxml.jackson.annotation.JsonMerge;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.github.fge.jsonpatch.diff.JsonDiff;
import java.math.BigInteger;
import lombok.AccessLevel;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.EqualsAndHashCode;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.SneakyThrows;
import lombok.ToString;
import org.junit.jupiter.api.Test;
class JsonPatchTest {
private static final ObjectMapper mapper = new ObjectMapper();
@Getter
@Builder
@ToString
@EqualsAndHashCode
@NoArgsConstructor(access = AccessLevel.PRIVATE)
@AllArgsConstructor(access = AccessLevel.PRIVATE)
public static class Data {
@JsonMerge Customer customer;
@Getter
@Builder
@ToString
@EqualsAndHashCode
@NoArgsConstructor(access = AccessLevel.PRIVATE)
@AllArgsConstructor(access = AccessLevel.PRIVATE)
public static class Customer {
@JsonMerge BigInteger age;
@JsonMerge String ssn;
@JsonMerge Address address;
@Getter
@Builder
@ToString
@EqualsAndHashCode
@NoArgsConstructor(access = AccessLevel.PRIVATE)
@AllArgsConstructor(access = AccessLevel.PRIVATE)
public static class Address {
@JsonMerge String street;
@JsonMerge String phone;
}
}
@SneakyThrows
Data merge(Data parent) {
var originCopyAsString = mapper.writerFor(this.getClass()).writeValueAsString(this);
var parentAsString = mapper.writerFor(this.getClass()).writeValueAsString(parent);
var parentCopy = mapper.readerFor(this.getClass()).readValue(parentAsString);
var clone = mapper.readerForUpdating(parentCopy).readValue(originCopyAsString);
return (Data) clone;
}
}
@SneakyThrows
@Test
void mask() {
final var diff = JsonDiff.asJsonPatch(mapper.readTree(jsonC()), mapper.readTree(jsonB()));
final var masked = diff.apply(mapper.readTree(jsonA())).toPrettyString();
assertThat(masked).isEqualToIgnoringWhitespace(masked());
}
private String jsonA() {
return "{\n"
+ " \"customer\": {\n"
+ " \"age\": 21,\n"
+ " \"ssn\": \"000 00 0000\",\n"
+ " \"address\": {\n"
+ " \"street\": \"Customer St.\",\n"
+ " \"phone\": \"+66 444 333 222\"\n"
+ " }\n"
+ " }\n"
+ "}";
}
private String jsonB() {
return "{\n"
+ " \"customer\": {\n"
+ " \"ssn\": \"The SSN is mandatory to register against XXX Company\",\n"
+ " \"address\": {\n"
+ " \"phone\": \"XXX Company will send a text message to validate your registration\"\n"
+ " }\n"
+ " }\n"
+ "}";
}
@SneakyThrows
private String jsonC() {
final Data dataA = mapper.readerFor(Data.class).readValue(jsonA());
final Data dataB = mapper.readerFor(Data.class).readValue(jsonB());
final Data merged = dataB.merge(dataA);
return mapper.writerFor(Data.class).writeValueAsString(merged);
}
@SneakyThrows
private String masked() {
return "{\n"
+ " \"customer\": {\n"
+ " \"ssn\": \"000 00 0000\",\n"
+ " \"address\": {\n"
+ " \"phone\": \"+66 444 333 222\"\n"
+ " }\n"
+ " }\n"
+ "}";
}
}
我建议使用 token/event/stream-based 解决方案。下面只是使用tinyparser/generator libhttps://github.com/anatolygudkov/green-jelly（Gson和Jackson都提供了面向流的API）的说明：
import org.green.jelly.AppendableWriter;
import org.green.jelly.JsonEventPump;
import org.green.jelly.JsonNumber;
import org.green.jelly.JsonParser;
import java.io.StringWriter;
import java.io.Writer;
import java.util.Arrays;
import java.util.HashSet;
import java.util.Set;
public class FilterMyJson {
private static final String jsonToFilter = "{\n" +
" \"customer\": {\n" +
" \"age\": 21,\n" +
" \"ssn\": \"000 00 0000\",\n" +
" \"address\": {\n" +
" \"street\": \"Customer St.\",\n" +
" \"phone\": \"+66 444 333 222\"\n" +
" }\n" +
" }\n" +
"}";
public static void main(String[] args) {
final StringWriter result = new StringWriter();
final JsonParser parser = new JsonParser();
parser.setListener(new MyJsonFilter(result, "age", "street"));
parser.parse(jsonToFilter); // if you read a file with a buffer,
// call parse() several times part by part in a loop until EOF
parser.eoj(); // and then call .eoj()
System.out.println(result);
}
static class MyJsonFilter extends JsonEventPump {
private final Set<String> objectMembersToFilter;
private boolean currentObjectMemberIsAllowed;
MyJsonFilter(final Writer output, final String... objectMembersToFilter) {
super(new AppendableWriter<>(output));
this.objectMembersToFilter = new HashSet<>(Arrays.asList(objectMembersToFilter));
}
@Override
public boolean onObjectMember(final CharSequence name) {
currentObjectMemberIsAllowed =
!objectMembersToFilter.contains(name.toString());
return super.onObjectMember(name);
}
@Override
public boolean onStringValue(final CharSequence data) {
if (!currentObjectMemberIsAllowed) {
return true;
}
return super.onStringValue(data);
}
@Override
public boolean onNumberValue(final JsonNumber number) {
if (!currentObjectMemberIsAllowed) {
return true;
}
return super.onNumberValue(number);
}
}
}
打印：
{
"customer":
{
"ssn": "000 00 0000",
"address":
{
"phone": "+66 444 333 222"
}
}
}
代码非常简单。现在它只过滤掉字符串和数字标量。不支持对象层次结构。因此，您可能需要针对某些情况改进代码。
此类解决方案的道具：
- file/data 不需要完全加载到内存中，你 可以毫无问题地处理 megs/gigs
- 它工作得更快，尤其是对于大文件
- 使用此模式很容易实现任何自定义 type/rule 转换。例如，您的过滤器很容易被参数化，您不必每次更改数据结构时都重新编译代码