在 Azure 数据工厂中将 JSON 文件从平面数组转换为嵌套数组

Question

我正在尝试将数据从我的 Oracle 数据库复制到搜索索引中，并且我正在使用 Azure 数据工厂将数据从 oracle 复制到 Azure Blob 存储。我如何使用它，将数据导入为嵌套 JSON 文件。现在，在查询 Oracle 之后，我得到这样的数据：

[{"BOOKING_ID":1.0,"REFERENCES":"ABC00001","ROUTES":{"ROUTE":1.0,"DESTINATION":"Atlanta, USA","ORIGIN":"New York, USA"}}
,{"BOOKING_ID":2.0,"REFERENCES":"ABC00322","ROUTES":{"ROUTE":2.0,"DESTINATION":"Las Vegas, USA","ORIGIN":"Los Angeles, USA"}}
,{"BOOKING_ID":3.0,"REFERENCES":"ABC32322","ROUTES":{"ROUTE":3.0,"DESTINATION":"Berlin, GER","ORIGIN":"Moscow, RUS"}}
,{"BOOKING_ID":4.0,"REFERENCES":"ABC543345","ROUTES":{"ROUTE":4.0,"DESTINATION":"Rome, ITA","ORIGIN":"Bejin, CHN"}}
,{"BOOKING_ID":5.0,"REFERENCES":"ABC51145","ROUTES":{"ROUTE":5.0,"DESTINATION":"Warsaw, POL","ORIGIN":"Copenhagen, DEN"}}
,{"BOOKING_ID":5.0,"REFERENCES":"ABC51145","ROUTES":{"ROUTE":6.0,"DESTINATION":"Copenhaged, DEN","ORIGIN":"Paris, FRA"}}
,{"BOOKING_ID":5.0,"REFERENCES":"ABC51145","ROUTES":{"ROUTE":7.0,"DESTINATION":"Paris, FRA","ORIGIN":"Madrid, ESP"}}
]

但我需要这样的数据：

[
  {
    "BOOKING_ID": 1.0,
    "REFERENCES": "ABC00001",
    "ROUTES": [
      {
        "ROUTE": 1.0,
        "DESTINATION": "Atlanta, USA",
        "ORIGIN": "New York, USA"
      }
    ]
  },
  {
    "BOOKING_ID": 2.0,
    "REFERENCES": "ABC00322",
    "ROUTES": [
      {
        "ROUTE": 2.0,
        "DESTINATION": "Las Vegas, USA",
        "ORIGIN": "Los Angeles, USA"
      }
    ]
  },
  {
    "BOOKING_ID": 3.0,
    "REFERENCES": "ABC32322",
    "ROUTES": [
      {
        "ROUTE": 3.0,
        "DESTINATION": "Berlin, GER",
        "ORIGIN": "Moscow, RUS"
      }
    ]
  },
  {
    "BOOKING_ID": 4.0,
    "REFERENCES": "ABC543345",
    "ROUTES": [
      {
        "ROUTE": 4.0,
        "DESTINATION": "Rome, ITA",
        "ORIGIN": "Bejin, CHN"
      }
    ]
  },
  {
    "BOOKING_ID": 5.0,
    "REFERENCES": "ABC51145",
    "ROUTES": [
      {
        "ROUTE": 5.0,
        "DESTINATION": "Warsaw, POL",
        "ORIGIN": "Copenhagen, DEN"
      },
      {
        "ROUTE": 6.0,
        "DESTINATION": "Copenhaged, DEN",
        "ORIGIN": "Paris, FRA"
      },
      {
        "ROUTE": 7.0,
        "DESTINATION": "Paris, FRA",
        "ORIGIN": "Madrid, ESP"
      }
    ]
  }
]

更新我将 Azure Functions 与 lodash 结合使用，但现在我正尝试从 Azure Blob 存储接收 JSON。问题是，当我尝试阅读 JSON 时，我得到的结果是这样的：

"type": "Buffer",
    "data": [
        239,
        187,
        191,
        91,
        123,
...

并且所有数据都是字节类型。

Answer 1

你的要求是按BOOKING_ID分组，将ROUTES对象合并成一个数组。不能直接在文案中实现activity

两个想法：

1.Using Web Activity + Azure Function Activity.

在WebActivity中，将查询方法封装成RESTAPI和return平面jsondata.PassWeb的输出Activity 到 Azure 函数 Activity。在Azure Function方法中，根据需要将数组json数据循环到嵌套数组中，然后将Azure Function的输出配置为Azure Blob Storage。（请参考此link）

2.Using Custom Activity.

您可以在 Azure Batch Job 中依赖 VM 执行脚本。比如你可以使用cx-Oracle包通过BOOKING_ID查询json数据顺序，然后使用python代码循环结果并根据需要转换它。

在 Azure 数据工厂中将 JSON 文件从平面数组转换为嵌套数组

Convert JSON file from flat to nested arrays in Azure Data Factory

arrays

json

azure

azure-data-factory