批量导出到 CSV

Question

我正在处理大量的 csv 约会，向用户发送通知等。我想做的是设置一个 isProcessed 标志，表示当前行已经处理我不确定如何处理在我当前的循环中执行此操作。

public void DumpTableToFile(SqlConnection connection, string tableName, string destinationFile)
    {
        using (var command = new SqlCommand("select LineType,CustomerFirstName AS 'Forename' ,CustomerLastName,Age,dob as 'Date of Birth',maritalStatus AS 'Marital Status',homePhone AS 'Home', mobileNumber AS Mobile,emailAddress AS Email,Address1 + Address2  + PostCode AS 'Address' ,employmentStatus AS Employment,occupation AS Occupation,propertyValue AS 'Property Value',mortgageBalance AS 'Mortgage Balance',balanceOnSecuredDebt AS 'Balance on secured Debt',mortgageType as 'Mortgage Type' from " + tableName, connection))
        using (var reader = command.ExecuteReader())
        using (var outFile = File.CreateText(destinationFile))
        {
            string[] columnNames = GetColumnNames(reader).ToArray();
            int numFields = columnNames.Length;
            outFile.WriteLine(string.Join(",", columnNames));
            if (reader.HasRows)
            {
                while (reader.Read())
                {
                    string[] columnValues =
                        Enumerable.Range(0, numFields)
                                  .Select(i => reader.GetValue(i).ToString())
                                  .Select(field => string.Concat("\"", field.Replace("\"", "\"\""), "\""))
                                  .ToArray();

                    outFile.WriteLine(string.Join(",", columnValues));
                }
            }
        }

该标志称为 isProcessed，一旦它通过 csv 导出，我想将其设置为 true。这样我就可以批量导出了。它存在于相同的 table 约会

编辑 1

抱歉，我没有说明我希望将此标志写回约会 table，因为它在 csv 导出中吐出的当前记录 csv 导出有效，我只需要一种方法来识别它它已被导出，因此不会进行第二次处理。

Answer 1

执行以下步骤：

为您的查询添加条件：WHERE NOT isProcessed，因此接下来您进行导出时，只会处理未处理的记录。
完成导出后，将此命令发送到数据库："UPDATE " + tableName + " SET isProcessed=true WHERE <exact same criteria as the select statement>"。这样所有记录现在都标记为已处理。
围绕导出机制包装一个 TransactionScope，包括。更新。这样，当出现故障时，整个操作将被回滚。
将 try-catch 包裹在 TransactionScope 周围，因此当导出失败时，CSV 文件将被删除，因此您永远不会有一半的导出批次。

你的代码会变成这样（我没测试）：

public void DumpTableToFile(SqlConnection connection, string tableName, string destinationFile)
{
    try
    {
        using(var transaction = new TransactionScope())
        {
            // Select all non-processed records.
            using (var command = new SqlCommand("select LineType,CustomerFirstName AS 'Forename' ,CustomerLastName,Age,dob as 'Date of Birth',maritalStatus AS 'Marital Status',homePhone AS 'Home', mobileNumber AS Mobile,emailAddress AS Email,Address1 + Address2  + PostCode AS 'Address' ,employmentStatus AS Employment,occupation AS Occupation,propertyValue AS 'Property Value',mortgageBalance AS 'Mortgage Balance',balanceOnSecuredDebt AS 'Balance on secured Debt',mortgageType as 'Mortgage Type' from " + tableName 
                + " WHERE NOT isProcessed", connection))
            using(var reader = command.ExecuteReader())
            using(var outFile = File.CreateText(destinationFile))
            {
                string[] columnNames = GetColumnNames(reader).ToArray();
                int numFields = columnNames.Length;
                outFile.WriteLine(string.Join(",", columnNames));
                if (reader.HasRows)
                {
                    while(reader.Read())
                    {
                        string[] columnValues =
                            Enumerable.Range(0, numFields)
                                .Select(i => reader.GetValue(i).ToString())
                                .Select(field => string.Concat("\"", field.Replace("\"", "\"\""), "\""))
                                .ToArray();

                        outFile.WriteLine(string.Join(",", columnValues));
                    }
                }
            }

            // Update the same records that were just exported.
            using (var command = new SqlCommand("UPDATE " + tableName + " SET isProcessed=true WHERE NOT isProcessed", connection))
                command.ExecuteNonQuery();

            transaction.Complete();
        }
    }
    catch
    {
        // If something went wrong, delete the export file.
        File.Delete(destinationFile);
        throw;
    }
}

批量导出到 CSV

Batch Exporting to CSV

c#

sql-server

csv

ado.net