如何在 ASP.NET 核心中提供仍在写入的文件
How to serve a file in ASP.NET Core while it's still being written
我有一个后台服务不断写入的日志文件。到目前为止，用户需要能够下载该文件。当我 return 一个 MVC FileResult 时，由于 Content-Length 不匹配，我得到了一个 InvalidOperationException，大概是因为一些内容在文件被提供时被写入了文件。提供了一个文件，大部分没问题，但通常最后一行不完整。
后台服务基本上是这样做的：
var stream = new FileStream(evidenceFilePath, FileMode.Append, FileAccess.Write, FileShare.Read);
while (true) // obviously it isn't actually this, but it does happen a lot!
{
var content = "log content\r\n";
stream.Write(Encoding.UTF8.GetBytes(content);
}
以下是控制器操作的一些变体（都有相同的结果）：
public IActionResult DownloadLog1()
{
return PhysicalFile("C:\path\to\the\file.txt", "text/plain", enableRangeProcessing: false); // also tried using true
}
public IActionResult DownloadLog2()
{
var stream = new FileStream("C:\path\to\the\file.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
return File(stream, "text/plain", enableRangeProcessing: false); // also tried true
}
这是我在尝试上述任一操作时遇到的异常：
System.InvalidOperationException: Response Content-Length mismatch: too many bytes written (216072192 of 216059904).
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ThrowTooManyBytesWritten(Int32 count)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.VerifyAndUpdateWrite(Int32 count)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.WriteAsync(ReadOnlyMemory`1 data, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpResponseStream.WriteAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Http.Extensions.StreamCopyOperation.CopyToAsync(Stream source, Stream destination, Nullable`1 count, Int32 bufferSize, CancellationToken cancel)
at Microsoft.AspNetCore.Mvc.Infrastructure.FileResultExecutorBase.WriteFileAsync(HttpContext context, Stream fileStream, RangeItemHeaderValue range, Int64 rangeLength)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeResultAsync(IActionResult result)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResultFilterAsync[TFilter,TFilterAsync]()
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResultExecutedContext context)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeResultFilters()
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResourceFilter()
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResourceExecutedContext context)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync()
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync()
at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(HttpContext httpContext)
at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware.Invoke(HttpContext httpContext)
at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
我不太介意异常，但如果它没有发生我会更喜欢它。不过，我确实需要修复不完整的最后一行问题。对我来说最明显的解决方案是跟踪已明确写入文件的字节数，并以某种方式仅提供前 n 个字节。我没有看到任何简单的方法可以使用 FileResult 和构造它的各种辅助方法来做到这一点。该文件可能会变得非常大（最多约 500MB），因此在内存中缓冲似乎不切实际。
文件是非托管资源。
因此，当您访问非托管资源（如文件）时，它是通过句柄打开的。如果是文件，则为 open_file_handle（凭记忆回忆）。
因此，我建议（非常通用）编写日志条目的最佳方式：
打开文件，
写入文件，
关闭文件，
如果适用则处理掉
简而言之，不要让流保持打开状态。
其次，对于控制器，您可以查看 MSDN 示例以通过控制器提供文件。
好吧，一般来说，您可能会遇到文件锁定问题，因此您需要对此进行规划和补偿。但是，您在这里的直接问题更容易解决。问题归结为 returning 流。在 returned 响应时正在写入该流，因此在创建响应主体时计算的内容长度是错误的。
你需要做的是抓取某个时间点的日志，即读入byte[]。然后，您可以 return 而不是流，内容长度将被正确计算，因为 byte[] 在读取后不会改变。
using (var stream = new FileStream("C:\path\to\the\file.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
using (var ms = new MemoryStream())
{
await stream.CopyToAsync(ms);
return File(ms.ToArray(), "text/plain");
}
我最终编写了一个自定义的 ActionResult 和 IActionResultExecutor 来匹配，它们在很大程度上基于 MVC FileStreamResult and FileStreamResultExecutor:
public class PartialFileStreamResult : FileResult
{
Stream stream;
long bytes;
/// <summary>
/// Creates a new <see cref="PartialFileStreamResult"/> instance with
/// the provided <paramref name="fileStream"/> and the
/// provided <paramref name="contentType"/>, which will download the first <paramref name="bytes"/>.
/// </summary>
/// <param name="stream">The stream representing the file</param>
/// <param name="contentType">The Content-Type header for the response</param>
/// <param name="bytes">The number of bytes to send from the start of the file</param>
public PartialFileStreamResult(Stream stream, string contentType, long bytes)
: base(contentType)
{
this.stream = stream ?? throw new ArgumentNullException(nameof(stream));
if (bytes == 0)
{
throw new ArgumentOutOfRangeException(nameof(bytes), "Invalid file length");
}
this.bytes = bytes;
}
/// <summary>
/// Gets or sets the stream representing the file to download.
/// </summary>
public Stream Stream
{
get => stream;
set => stream = value ?? throw new ArgumentNullException(nameof(stream));
}
/// <summary>
/// Gets or sets the number of bytes to send from the start of the file.
/// </summary>
public long Bytes
{
get => bytes;
set
{
if (value == 0)
{
throw new ArgumentOutOfRangeException(nameof(bytes), "Invalid file length");
}
bytes = value;
}
}
/// <inheritdoc />
public override Task ExecuteResultAsync(ActionContext context)
{
if (context == null)
{
throw new ArgumentNullException(nameof(context));
}
var executor = context.HttpContext.RequestServices.GetRequiredService<IActionResultExecutor<PartialFileStreamResult>>();
return executor.ExecuteAsync(context, this);
}
}
public class PartialFileStreamResultExecutor : FileResultExecutorBase, IActionResultExecutor<PartialFileStreamResult>
{
public PartialFileStreamResultExecutor(ILoggerFactory loggerFactory)
: base(CreateLogger<PartialFileStreamResultExecutor>(loggerFactory))
{
}
public async Task ExecuteAsync(ActionContext context, PartialFileStreamResult result)
{
if (context == null)
{
throw new ArgumentNullException(nameof(context));
}
if (result == null)
{
throw new ArgumentNullException(nameof(result));
}
using (result.Stream)
{
long length = result.Bytes;
var (range, rangeLength, serveBody) = SetHeadersAndLog(context, result, length, result.EnableRangeProcessing);
if (!serveBody) return;
try
{
var outputStream = context.HttpContext.Response.Body;
if (range == null)
{
await StreamCopyOperation.CopyToAsync(result.Stream, outputStream, length, bufferSize: BufferSize, cancel: context.HttpContext.RequestAborted);
}
else
{
result.Stream.Seek(range.From.Value, SeekOrigin.Begin);
await StreamCopyOperation.CopyToAsync(result.Stream, outputStream, rangeLength, BufferSize, context.HttpContext.RequestAborted);
}
}
catch (OperationCanceledException)
{
// Don't throw this exception, it's most likely caused by the client disconnecting.
// However, if it was cancelled for any other reason we need to prevent empty responses.
context.HttpContext.Abort();
}
}
}
}
我本可以做更多的工作来添加额外的构造函数重载以设置一些可选参数（例如下载文件名等），但这足以满足我的需要。
您需要在 Startup.ConfigureServices 中添加 IActionResultExecutor:
services.AddTransient<IActionResultExecutor<PartialFileStreamResult>, PartialFileStreamResultExecutor>();
因此我的控制器操作变成了：
[HttpGet]
public IActionResult DownloadLog()
{
var (path, bytes) = GetThePathAndTheNumberOfBytesIKnowHaveBeenFlushed();
var stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite); // this ensures that the file can be read while it's still being written
return new PartialFileStreamResult(stream, "text/plain", bytes);
}
我有一个后台服务不断写入的日志文件。到目前为止，用户需要能够下载该文件。当我 return 一个 MVC FileResult 时，由于 Content-Length 不匹配，我得到了一个 InvalidOperationException，大概是因为一些内容在文件被提供时被写入了文件。提供了一个文件，大部分没问题，但通常最后一行不完整。
后台服务基本上是这样做的：
var stream = new FileStream(evidenceFilePath, FileMode.Append, FileAccess.Write, FileShare.Read);
while (true) // obviously it isn't actually this, but it does happen a lot!
{
var content = "log content\r\n";
stream.Write(Encoding.UTF8.GetBytes(content);
}
以下是控制器操作的一些变体（都有相同的结果）：
public IActionResult DownloadLog1()
{
return PhysicalFile("C:\path\to\the\file.txt", "text/plain", enableRangeProcessing: false); // also tried using true
}
public IActionResult DownloadLog2()
{
var stream = new FileStream("C:\path\to\the\file.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
return File(stream, "text/plain", enableRangeProcessing: false); // also tried true
}
这是我在尝试上述任一操作时遇到的异常：
System.InvalidOperationException: Response Content-Length mismatch: too many bytes written (216072192 of 216059904). at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ThrowTooManyBytesWritten(Int32 count) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.VerifyAndUpdateWrite(Int32 count) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.WriteAsync(ReadOnlyMemory`1 data, CancellationToken cancellationToken) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpResponseStream.WriteAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken) at Microsoft.AspNetCore.Http.Extensions.StreamCopyOperation.CopyToAsync(Stream source, Stream destination, Nullable`1 count, Int32 bufferSize, CancellationToken cancel) at Microsoft.AspNetCore.Mvc.Infrastructure.FileResultExecutorBase.WriteFileAsync(HttpContext context, Stream fileStream, RangeItemHeaderValue range, Int64 rangeLength) at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeResultAsync(IActionResult result) at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResultFilterAsync[TFilter,TFilterAsync]() at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResultExecutedContext context) at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeResultFilters() at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResourceFilter() at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResourceExecutedContext context) at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync() at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync() at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(HttpContext httpContext) at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware.Invoke(HttpContext httpContext) at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext) at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
我不太介意异常，但如果它没有发生我会更喜欢它。不过，我确实需要修复不完整的最后一行问题。对我来说最明显的解决方案是跟踪已明确写入文件的字节数，并以某种方式仅提供前 n 个字节。我没有看到任何简单的方法可以使用 FileResult 和构造它的各种辅助方法来做到这一点。该文件可能会变得非常大（最多约 500MB），因此在内存中缓冲似乎不切实际。
文件是非托管资源。
因此，当您访问非托管资源（如文件）时，它是通过句柄打开的。如果是文件，则为 open_file_handle（凭记忆回忆）。
因此，我建议（非常通用）编写日志条目的最佳方式：
打开文件，
写入文件，
关闭文件，
如果适用则处理掉
简而言之，不要让流保持打开状态。
其次，对于控制器，您可以查看 MSDN 示例以通过控制器提供文件。
好吧，一般来说，您可能会遇到文件锁定问题，因此您需要对此进行规划和补偿。但是，您在这里的直接问题更容易解决。问题归结为 returning 流。在 returned 响应时正在写入该流，因此在创建响应主体时计算的内容长度是错误的。
你需要做的是抓取某个时间点的日志，即读入byte[]。然后，您可以 return 而不是流，内容长度将被正确计算，因为 byte[] 在读取后不会改变。
using (var stream = new FileStream("C:\path\to\the\file.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
using (var ms = new MemoryStream())
{
await stream.CopyToAsync(ms);
return File(ms.ToArray(), "text/plain");
}
我最终编写了一个自定义的 ActionResult 和 IActionResultExecutor 来匹配，它们在很大程度上基于 MVC FileStreamResult and FileStreamResultExecutor:
public class PartialFileStreamResult : FileResult
{
Stream stream;
long bytes;
/// <summary>
/// Creates a new <see cref="PartialFileStreamResult"/> instance with
/// the provided <paramref name="fileStream"/> and the
/// provided <paramref name="contentType"/>, which will download the first <paramref name="bytes"/>.
/// </summary>
/// <param name="stream">The stream representing the file</param>
/// <param name="contentType">The Content-Type header for the response</param>
/// <param name="bytes">The number of bytes to send from the start of the file</param>
public PartialFileStreamResult(Stream stream, string contentType, long bytes)
: base(contentType)
{
this.stream = stream ?? throw new ArgumentNullException(nameof(stream));
if (bytes == 0)
{
throw new ArgumentOutOfRangeException(nameof(bytes), "Invalid file length");
}
this.bytes = bytes;
}
/// <summary>
/// Gets or sets the stream representing the file to download.
/// </summary>
public Stream Stream
{
get => stream;
set => stream = value ?? throw new ArgumentNullException(nameof(stream));
}
/// <summary>
/// Gets or sets the number of bytes to send from the start of the file.
/// </summary>
public long Bytes
{
get => bytes;
set
{
if (value == 0)
{
throw new ArgumentOutOfRangeException(nameof(bytes), "Invalid file length");
}
bytes = value;
}
}
/// <inheritdoc />
public override Task ExecuteResultAsync(ActionContext context)
{
if (context == null)
{
throw new ArgumentNullException(nameof(context));
}
var executor = context.HttpContext.RequestServices.GetRequiredService<IActionResultExecutor<PartialFileStreamResult>>();
return executor.ExecuteAsync(context, this);
}
}
public class PartialFileStreamResultExecutor : FileResultExecutorBase, IActionResultExecutor<PartialFileStreamResult>
{
public PartialFileStreamResultExecutor(ILoggerFactory loggerFactory)
: base(CreateLogger<PartialFileStreamResultExecutor>(loggerFactory))
{
}
public async Task ExecuteAsync(ActionContext context, PartialFileStreamResult result)
{
if (context == null)
{
throw new ArgumentNullException(nameof(context));
}
if (result == null)
{
throw new ArgumentNullException(nameof(result));
}
using (result.Stream)
{
long length = result.Bytes;
var (range, rangeLength, serveBody) = SetHeadersAndLog(context, result, length, result.EnableRangeProcessing);
if (!serveBody) return;
try
{
var outputStream = context.HttpContext.Response.Body;
if (range == null)
{
await StreamCopyOperation.CopyToAsync(result.Stream, outputStream, length, bufferSize: BufferSize, cancel: context.HttpContext.RequestAborted);
}
else
{
result.Stream.Seek(range.From.Value, SeekOrigin.Begin);
await StreamCopyOperation.CopyToAsync(result.Stream, outputStream, rangeLength, BufferSize, context.HttpContext.RequestAborted);
}
}
catch (OperationCanceledException)
{
// Don't throw this exception, it's most likely caused by the client disconnecting.
// However, if it was cancelled for any other reason we need to prevent empty responses.
context.HttpContext.Abort();
}
}
}
}
我本可以做更多的工作来添加额外的构造函数重载以设置一些可选参数（例如下载文件名等），但这足以满足我的需要。
您需要在 Startup.ConfigureServices 中添加 IActionResultExecutor:
services.AddTransient<IActionResultExecutor<PartialFileStreamResult>, PartialFileStreamResultExecutor>();
因此我的控制器操作变成了：
[HttpGet]
public IActionResult DownloadLog()
{
var (path, bytes) = GetThePathAndTheNumberOfBytesIKnowHaveBeenFlushed();
var stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite); // this ensures that the file can be read while it's still being written
return new PartialFileStreamResult(stream, "text/plain", bytes);
}