C# 内存不足写入 CSV

Question

我一直在为我的一个朋友开发一个小程序，他有一个非常大的文件，我将其读入数据网格视图 > 修改数据 > 导出到 csv。直到最近他要求我对数据导出的方式进行一些更改时，我才设法使一切都相对良好地工作。出于某种原因，我在运行这个函数时遇到内存不足异常。

private void ExportData(int fileNum = 1, int rowCount = 0)
    {
        int lastRow = rowCount;
        if (!Directory.Exists(ExportPath + dataFilePath.Name))
            Directory.CreateDirectory(ExportPath + dataFilePath.Name);
        StreamWriter sw = new StreamWriter(ExportPath + dataFilePath.Name + @"\" + dataFilePath.Name + "_" + fileNum + ".csv");

        //var headers = dataGridView1.Columns.Cast<DataGridViewColumn>();
        //sw.WriteLine(string.Join(",", headers.Select(column => "\"" + column.HeaderText + "\"").ToArray()));
        sw.WriteLine("Unit,UPC,Brand,Vendor,List Cost,QTY,Price,Description,Attribute 1,Attribute 2," +
            "Descriptor 1,Descriptor 2,Descriptor 3,Descriptor 4,Descriptor 5,Descriptor 6,Descriptor 7,Descriptor 8");

        for (int i = 0; i < 50000; i++)
        {
            rowCount = lastRow + i;

            if (rowCount >= dataGridView1.RowCount)
                break;
            var cells = dataGridView1.Rows[rowCount].Cells.Cast<DataGridViewCell>();
            sw.WriteLine(string.Join(",", cells.Select(cell => "\"" + cell.Value + "\"").ToArray()));
        }

        sw.Close();
        sw.Dispose();
        lastRow = rowCount + 1;
        if (lastRow < dataGridView1.RowCount - 1)
            ExportData(fileNum + 1, lastRow);
        else
        {
            progressBar1.BeginInvoke(new MethodInvoker(delegate {
                progressBar1.Style = ProgressBarStyle.Blocks;
                button_OpenDataFile.Enabled = true;
                button_ConvertFromRaw.Enabled = true;
                button_exportLS.Enabled = true;

                Console.WriteLine("[Main] Export complete.");
            }));
        }
    }

var cells = dataGridView1.Rows[rowCount].Cells.Cast<DataGridViewCell>(); 似乎是发生错误的行。

任何人都可以深入了解我做错了什么吗？

谢谢！

Answer 1

做这个实验：将代码转换为循环，而不是使用递归：

private void ExportData()
{
    //You only need to do this once, take it out of the loop.
    if (!Directory.Exists(ExportPath + dataFilePath.Name))
        Directory.CreateDirectory(ExportPath + dataFilePath.Name);

    var fileNum = 0;
    var rowCount = 0;

    while (rowCount < dataGridView1.RowCount)
    {
        fileNum = fileNum + 1;

        using (StreamWriter sw = new StreamWriter(ExportPath + dataFilePath.Name + @"\" + dataFilePath.Name + "_" + fileNum + ".csv")
        {
            sw.WriteLine("Unit,UPC,Brand,Vendor,List Cost,QTY,Price,Description,Attribute 1,Attribute 2," +
                "Descriptor 1,Descriptor 2,Descriptor 3,Descriptor 4,Descriptor 5,Descriptor 6,Descriptor 7,Descriptor 8");

            for (int i = 0; i < 50000; i++)
            {
                rowCount = rowCount + 1;

                if (rowCount >= dataGridView1.RowCount)
                    break;
                var cells = dataGridView1.Rows[rowCount].Cells.Cast<DataGridViewCell>();
                sw.WriteLine(string.Join(",", cells.Select(cell => "\"" + cell.Value + "\"").ToArray()));
            }
        }   //sw.Close() and sw.Dispose() not needed because of the 'using'. You may want to do sw.Flush().
    }

    //The 'else' part of your original recursive method
    progressBar1.BeginInvoke(new MethodInvoker(delegate {
        progressBar1.Style = ProgressBarStyle.Blocks;
        button_OpenDataFile.Enabled = true;
        button_ConvertFromRaw.Enabled = true;
        button_exportLS.Enabled = true;

        Console.WriteLine("[Main] Export complete.");
    }));
}

错误消失了吗？可能是。递归使用堆栈中的大量内存，并且直到递归结束时才释放它，当再次进入堆栈时。您收到错误的行恰好尝试将 csv 文件中整行的内容添加到内存中。如果递归堆栈的内存几乎已满，那可能是导致内存不足异常的最后一次丢弃。

我删除了一些看起来多余的累加器变量，希望我没有弄乱循环的范围。

我从循环中删除了 CreateDirectory，并为 StreamWriter 添加了一个 using 语句。我不认为这些是你出错的原因，因为目录只创建了一次，并且你在递归调用之前处理了 StreamWriter ，但无论如何如果你想确认它你可以尝试撤消这些更改在non-recursive一一代码中，看看是否再次出现错误。

Answer 2

所以我想通了。
我想遍历 datagridview 并不是可行的方法。
相反，我只是使用我的数据源导出了数据。它要快得多……从 2 分钟减少到大约 2 秒。

谢谢大家的帮助！

private void ExportData()
    {
        //You only need to do this once, take it out of the loop.
        if (!Directory.Exists(ExportPath + dataFilePath.Name))
            Directory.CreateDirectory(ExportPath + dataFilePath.Name);

        var fileNum = 0;
        var rowCount = 0;

        while (rowCount < dataGridView1.RowCount)
        {
            fileNum = fileNum + 1;

            using (StreamWriter sw = new StreamWriter(ExportPath + dataFilePath.Name + @"\" + dataFilePath.Name + "_" + fileNum + ".csv"))
            {
                sw.WriteLine("Unit,UPC,Brand,Vendor,List Cost,QTY,Price,Description,Attribute 1,Attribute 2" +
                "Descriptor 1,Descriptor 2,Descriptor 3,Descriptor 4,Descriptor 5,Descriptor 6,Descriptor 7,Descriptor 8");

                for (int i = 0; i < 50000; i++)
                {
                    rowCount = rowCount + 1;

                    if (rowCount >= dataGridView1.RowCount)
                        break;
                    var s = new string[]
                    {
                        "\"" + DATA[rowCount].Unit + "\"",
                        "\"" + DATA[rowCount].UPC + "\"",
                        "\"" + DATA[rowCount].Brand + "\"",
                        "\"" + DATA[rowCount].Vendor + "\"",
                        "\"" + DATA[rowCount].List_Cost + "\"",
                        "\"" + DATA[rowCount].Quantity.ToString() + "\"",
                        "\"" + DATA[rowCount].Price + "\"",
                        "\"" + DATA[rowCount].Description + "\"",
                        "\"" + DATA[rowCount].Attribute_1 + "\"",
                        "\"" + DATA[rowCount].Attribute_2 + "\"",
                        "\"" + DATA[rowCount].Descriptor_1 + "\"",
                        "\"" + DATA[rowCount].Descriptor_2 + "\"",
                        "\"" + DATA[rowCount].Descriptor_3 + "\"",
                        "\"" + DATA[rowCount].Descriptor_4 + "\"",
                        "\"" + DATA[rowCount].Descriptor_5 + "\"",
                        "\"" + DATA[rowCount].Descriptor_6 + "\"",
                        "\"" + DATA[rowCount].Descriptor_7 + "\"",
                        "\"" + DATA[rowCount].Descriptor_8 + "\""
                    };
                    sw.WriteLine(string.Join(",", s));
                    sw.Flush();
                }
            }   //sw.Close() and sw.Dispose() not needed because of the 'using'. You may want to do sw.Flush().
        }

        //The 'else' part of your original recursive method
        progressBar1.BeginInvoke(new MethodInvoker(delegate
        {
            progressBar1.Style = ProgressBarStyle.Blocks;
            button_OpenDataFile.Enabled = true;
            button_ConvertFromRaw.Enabled = true;
            button_exportLS.Enabled = true;

            Console.WriteLine("[Main] Export complete.");
        }));
    }'

C# 内存不足写入 CSV

C# Out of Memory writing to CSV

c#

csv

out-of-memory