Azure 数据资源管理器自动摄取失败但手动工作

Question

我正在使用 Azure 数据资源管理器和事件中心从 blob 存储中提取 JSON。

我遵循的步骤是：

逻辑应用创建 blob JSON，格式为 {"id":"12345","message":"base64String"}
在 Azure 数据资源管理器中，我创建了一个 table，然后使用“从 blob 容器获取数据”。这成功导入了我已经在容器中的数据。
然后我单击 link 设置自动摄取。此自动摄取失败，出现以下异常：

.show ingestion failures

"OperationId": f533e94f-58f5-4c88-837e-e1b9117850fa,
"Database": redacteddb,
"Table": redactedtbl,
"FailedOn": 2021-04-28T14:19:25.2781929Z,
"IngestionSourcePath": https://redactedstore.blob.core.windows.net/attachments/12345.txt.json,
"Details": Stream_NoDataToIngest: The input stream produced 0 bytes. This usually means that the input JSON stream was ill formed.,
"FailureKind": Permanent,
"RootActivityId": guid...,
"OperationKind": DataIngestPull,
"OriginatesFromUpdatePolicy": 0,
"ErrorCode": Stream_NoDataToIngest,
"Principal": dstsapp=KustoIngestionProd,
"ShouldRetry": 0,
"User": ,
"IngestionProperties": [Format=Json/mandatory, IngestionMapping=[{"column":"id","datatype":"","Path":"$['id']"},{"column":"message","datatype":"","Path":"$['message']"}], ValidationPolicy=[Options=ValidateCsvInputConstantColumns, Implications=BestEffort, IsDetailedErrorReportingEnabled=False], Tags=[ToStringEmpty], IngestIfNotExists=[ToStringEmpty], ZipPattern=[null]],

如果我运行通过 https://dataexplorer.azure.com/ 再次手动摄取它就可以了。

谁能帮我知道接下来如何解决问题？我在设置摄取时做错了什么？

Answer 1

尝试重新配置您的数据连接以使用 multijson / multilinejson 格式，而不是 json.

参见：https://docs.microsoft.com/en-us/azure/data-explorer/ingest-json-formats?tabs=kusto-query-language#the-json-format

The JSON format Azure Data Explorer supports two JSON file formats:

json: Line separated JSON. Each line in the input data has exactly one JSON record.

multijson: Multi-lined JSON. The parser ignores the line separators and reads a record from the previous position to the end of a valid JSON.

For more information, see JSON Lines.

Azure 数据资源管理器自动摄取失败但手动工作

Azure data explorer automatic ingestion fails but manual works

azure

azure-data-explorer