如何将 BIG JSON 数据反序列化为 DataGridView

Question

我想将一个大 JSON 文件 (~600MB) 反序列化到 DataGridView 中，而不会遇到 "maxJsonLength" 错误。

我的代码适用于小型 JSON 文件。但是对于更大的 JSON 文件，我得到 "maxJsonLength" 错误。由于我不是最有经验的编码员，是否有解决此问题的解决方法或简单方法？

Private Sub BtnOpenFile_Click(sender As Object, e As EventArgs) Handles BtnOpenFileOld.Click
        OpenFileDialog1.InitialDirectory = "C:\"

        If OpenFileDialog1.ShowDialog = DialogResult.Cancel Then
        End If

        Dim JSonFilePath As String = File.ReadAllText(OpenFileDialog1.FileName)
        LblFilePath.Text = OpenFileDialog1.FileName

        DataGridView1.Rows.Clear()
        DataGridView1.Refresh()

        Dim dict As Object = New JavaScriptSerializer().Deserialize(Of List(Of Object))(JSonFilePath)

        For Each item As Object In dict
            DataGridView1.Rows.Add(item("EMail").ToString, item("Timestamp").ToString, item("Number").ToString)
        Next

    End Sub

我的 JSON 文件看起来像

[
  {
    "EMail": "one@mail.com",
    "Timestamp": "2019-05-25T21:24:06.799381+02:00",
    "Number": 206074,
    "randomtrash1": "notneeded",
    "randomtrash2": "notneeded",
    "randomtrash3": "notneeded",
    "randomtrash4": "notneeded",
    "randomtrash5": "notneeded",
    "randomtrash6": "notneeded",
    "randomtrash7": "notneeded",
    "randomtrash8": "notneeded",
    "randomtrash9": "notneeded"
  },
  {
    "EMail": "two@mail.com",
    "Timestamp": "2019-05-25T21:24:06.8273826+02:00",
    "Number": 7397,
    "randomtrash1": "notneeded",
    "randomtrash2": "notneeded",
    "randomtrash3": "notneeded",
    "randomtrash4": "notneeded",
    "randomtrash5": "notneeded",
    "randomtrash6": "notneeded",
    "randomtrash7": "notneeded",
    "randomtrash8": "notneeded",
    "randomtrash9": "notneeded",
    "randomtrash10": "notneeded",
    "randomtrash11": "notneeded",
    "randomtrash12": "notneeded",
    "randomtrash13": "notneeded",
    "randomtrash14": "notneeded",
    "randomtrash15": "notneeded",
    "randomtrash16": "notneeded",
    "randomtrash17": "notneeded",
    "randomtrash18": "notneeded",
    "randomtrash19": "notneeded",
    "randomtrash20": "notneeded",
    "randomtrash21": "notneeded"
  }
]

Answer 1

避免 maxJsonLength 错误的最小修复是设置 JavaScriptSerializer.MaxJsonLength = int.MaxValue as shown in this answer to The length of the string exceeds the value set on the maxJsonLength property by Taha Rehman Siddiqui:

Dim serializer As JavaScriptSerializer = New JavaScriptSerializer()
serializer.MaxJsonLength = Int32.MaxValue
Dim dict As Object = serializer.Deserialize(Of List(Of Object))(JSonFilePath)

但是，您当前正在将一个 600MB 的文件加载到一个 600-1,200MB 的字符串中，该字符串正在接近 .Net maximum string length. If you file gets somewhat larger, you will start to get out-of-memory exceptions. A better approach would be to deserialize directly from the file using a streaming solution, however JavaScriptSerializer does not support streaming. Thus I'd recommend switching to json.net。

首先，如图所示安装Json.NET here。接下来，由于您似乎不想为 JSON 定义显式数据模型，因此创建以下静态方法：

Public Module JsonExtensions
    Public Function LoadAnonymousType(Of T)(ByVal path as String, ByVal anonymousTypeObject as T, Optional ByVal settings as JsonSerializerSettings = Nothing) as T
        Using streamReader As TextReader = File.OpenText(path)
            Dim serializer = JsonSerializer.CreateDefault(settings)
            return CType(serializer.Deserialize(streamreader, GetType(T)), T)
        End Using
    End Function
End Module

这样做是将 path 参数指定位置的文件的内容反序列化为 anonymousTypeObject 参数指定类型的数据模型。

现在您可以将 JSON 直接从文件反序列化为 anonymous type objects 的数组，如下所示：

Dim array = JsonExtensions.LoadAnonymousType( _
    FileName, _
    { New With {.Email = CType(Nothing, String), .Timestamp = CType(Nothing, String), .Number = CType(Nothing, Long) }} _
)
For Each item In array
    ' DataGridView1.Rows.Add(item("EMail").ToString, item("Timestamp").ToString, item("Number").ToString)
    Console.WriteLine("Email = {0}, Timestamp = {1}, Number = {2}", item.Email, item.Timestamp, item.Number)
Next

或者，您可以创建一个显式数据模型，如下所示：

Public Class RootObject
    Public Property Email As String
    Public Property Timestamp As String
    Public Property Number As Long
End Class

并反序列化如Deserialize JSON from a file:

Dim list As List(Of RootObject) = Nothing
Using streamReader As TextReader = File.OpenText(FileName)
    list = CType(JsonSerializer.CreateDefault().Deserialize(streamReader, GetType(List(Of RootObject))), List(Of RootObject))
End Using
For Each item In list
    ' DataGridView1.Rows.Add(item("EMail").ToString, item("Timestamp").ToString, item("Number").ToString)
    Console.WriteLine("Email = {0}, Timestamp = {1}, Number = {2}", item.Email, item.Timestamp, item.Number)
Next

演示 fiddle here.

Answer 2

Imports Newtonsoft.Json

Public Class Form1

Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load

    Dim str As String = _
            "[" + _
            "  {" + _
            "    ""EMail"": ""one@mail.com""," + _
            "    ""Timestamp"": ""2019-05-25T21:24:06.799381+02:00""," + _
            "    ""Number"": 206074," + _
            "  }," + _
            "  {" + _
            "    ""EMail"": ""two@mail.com""," + _
            "    ""Timestamp"": ""2019-05-25T21:24:06.8273826+02:00""," + _
            "    ""Number"": 7397," + _
            "  }," + _
            "]"

    Try
        Dim list As List(Of jsnn) = JsonConvert.DeserializeObject(Of List(Of jsnn))(str)

        For Each item As jsnn In list
            Console.WriteLine(item.EMail & ": " & item.Timestamp & ": " & item.Number)
        Next

    Catch ex As Exception
        MsgBox(ex.Message)
    End Try

End Sub

End Class

Public Class jsnn
    Public Property EMail As String
    Public Property Timestamp As DateTime
    Public Property Number As Integer
End Class

这是截图

如何将 BIG JSON 数据反序列化为 DataGridView

How to deserialize BIG JSON data into DataGridView

vb.net

json

datagridview

json.net

json-deserialization