将 zip 从 Blazor WASM 应用程序上传到 AWS-S3 - 遵循教程并且没有观察到错误,但存储桶中也没有文件

Uploading a zip from from Blazor WASM app to AWS-S3 - tutorials followed and no errors observed, but also no file in the bucket

我有一个 Blazor WASM 应用程序(即客户端),用户可以将一些数据上传到该应用程序。在对数据进行一些完整性和安全性检查后,我想将它们上传到 AWS S3 中的一个 zip 文件。

目前我构建的zip文件如下

using (var outStream = new MemoryStream())
{
    using (var archive = new ZipArchive(outStream, ZipArchiveMode.Create, true))
    {
        foreach (var file in imagelist.Files)
        {
            json_name = file.Name + ".json";
            var file_in_archive = archive.CreateEntry(file.Name, CompressionLevel.Optimal);
            using (var entryStream = file_in_archive.Open())
            {
                file.Data.CopyToAsync(entryStream);
            }
            file_in_archive = archive.CreateEntry(json_name, CompressionLevel.Optimal);
            using (var entryStream = file_in_archive.Open())
            {
                formMems[file_counter].CopyToAsync(entryStream);
            }
            file_counter = file_counter + 1;
        }

    }
    Console.WriteLine("Finished zipping");
    uploader.UploadZipFile(outStream, "test_title.zip");
}

我正在尝试通过这种方法上传 zip 文件 uploader.UploadZipFile - 代码如下

public class UploadZip    
{
    public BasicAWSCredentials credentials = new BasicAWSCredentials("access key", "secret key");
 
    public void UploadZipFile(Stream zip_file, string file_name)
    {
        zip_file.Seek(0, SeekOrigin.Begin);
        var config = new AmazonS3Config
        {
            RegionEndpoint = Amazon.RegionEndpoint.Region
        };
        using var client = new AmazonS3Client(credentials, config);
        var uploadRequest = new TransferUtilityUploadRequest
        {
            InputStream = zip_file,
            Key = file_name,
            BucketName = "Bucket-Name",
            CannedACL = S3CannedACL.Private
        };
        var fileTransferUtility = new TransferUtility(client);
        fileTransferUtility.UploadAsync(uploadRequest);
        Console.WriteLine("finshed uploading");
    }
}

(上传器是 UploadZip 的一个实例)。出于显而易见的原因,我删除了存储桶名称、区域、访问密钥和秘密密钥,但这些都是正确的。

访问密钥和秘密密钥属于我为此目的创建的自定义 IAM 用户。政策摘要如下

{
"Version": "2012-10-17",
"Statement": [
    {
        "Sid": "VisualEditor0",
        "Effect": "Allow",
        "Action": "s3:PutObject",
        "Resource": "arn:aws:s3:::Bucket-Name/*"
    }
]

}

并且 S3 存储桶已 Block *all* public access 关闭 ,下面给出了 CORS

[
{
    "AllowedHeaders": [
        "*"
    ],
    "AllowedMethods": [
        "PUT",
        "POST"
    ],
    "AllowedOrigins": [
        "*"
    ],
    "ExposeHeaders": []
}

]

ACL 授予 Bucket 所有者(您的 AWS 账户)ListWrite for Objects,以及 ReadWrite Bucket ACL,其他 3 个受赠者没有其他。没有桶策略。

我只是不明白为什么这不起作用。当我 运行 程序时,我没有看到任何错误,并且两个 Console.write() 调用 运行 很好。

编辑

为了回应评论中的建议,我尝试 运行 使用 Upload 而不是 UploadAsync 修改代码 - 见下文

public class UploadZip    
{
    public BasicAWSCredentials credentials = new BasicAWSCredentials("AKIAZWFJ7CFNER3WG5EP", "EE9eMD/vz8G3Ui3/MZdGDNBY9xv4+/3y0VQAhNzn");
 
    public void UploadZipFile(Stream zip_file, string file_name)
    {
        zip_file.Seek(0, SeekOrigin.Begin);
        var config = new AmazonS3Config
        {
            RegionEndpoint = Amazon.RegionEndpoint.EUWest2
        };
        using var client = new AmazonS3Client(credentials, config);
        var uploadRequest = new TransferUtilityUploadRequest
        {
            InputStream = zip_file,
            Key = file_name,
            BucketName = "web-pupil-data-store",
            CannedACL = S3CannedACL.Private
        };
        var fileTransferUtility = new TransferUtility(client);
        fileTransferUtility.Upload(uploadRequest);
        Console.WriteLine("finshed uploading");
    }
}

这仍然完成了文件的压缩,但是在打印 finished uploading 之前,它现在抛出一个错误,其中有一个大而难以理解的(对我来说)堆栈跟踪,复制在这里

blazor.webassembly.js:1 crit: Microsoft.AspNetCore.Components.WebAssembly.Rendering.WebAssemblyRenderer[100]
      Unhandled exception rendering component: Cannot start threads on this runtime.
System.NotSupportedException: Cannot start threads on this runtime.
  at (wrapper managed-to-native) System.Threading.Thread.Thread_internal(System.Threading.Thread,System.MulticastDelegate)
  at System.Threading.Thread.StartInternal (System.Object principal, System.Threading.StackCrawlMark& stackMark) <0x47bb958 + 0x00008> in <filename unknown>:0 
  at System.Threading.Thread.Start (System.Threading.StackCrawlMark& stackMark) <0x47bb818 + 0x0004e> in <filename unknown>:0 
  at System.Threading.Thread.Start () <0x47bb6d8 + 0x0000e> in <filename unknown>:0 
  at Amazon.Runtime.Internal.Util.BackgroundDispatcher`1[T]..ctor (System.Action`1[T] action) [0x0005c] in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Internal\Util\Dispatcher.cs:55 
  at Amazon.Runtime.Internal.Util.BackgroundInvoker..ctor () [0x00000] in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Internal\Util\Dispatcher.cs:180 
  at Amazon.Util.AWSSDKUtils.get_Dispatcher () [0x00007] in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Util\AWSSDKUtils.cs:674 
  at Amazon.Util.AWSSDKUtils.InvokeInBackground[T] (System.EventHandler`1[TEventArgs] handler, T args, System.Object sender) [0x0004c] in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Util\AWSSDKUtils.cs:661 
  at Amazon.Runtime.Internal.StreamReadTracker.ReadProgress (System.Int32 bytesRead) [0x00047] in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Internal\StreamReadTracker.cs:55 
  at Amazon.Runtime.Internal.Util.EventStream.ReadAsync (System.Byte[] buffer, System.Int32 offset, System.Int32 count, System.Threading.CancellationToken cancellationToken) [0x0009c] in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Internal\Util\EventStream.cs:138 
  at Amazon.Runtime.Internal.Util.ChunkedUploadWrapperStream.FillInputBufferAsync (System.Threading.CancellationToken cancellationToken) [0x00090] in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Internal\Util\ChunkedUploadWrapperStream.cs:189 
  at Amazon.Runtime.Internal.Util.ChunkedUploadWrapperStream.ReadAsync (System.Byte[] buffer, System.Int32 offset, System.Int32 count, System.Threading.CancellationToken cancellationToken) [0x00053] in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Internal\Util\ChunkedUploadWrapperStream.cs:163 
  at System.Threading.Tasks.ValueTask`1[TResult].get_Result () <0x370ff80 + 0x00034> in <filename unknown>:0 
  at System.IO.Stream.CopyToAsyncInternal (System.IO.Stream destination, System.Int32 bufferSize, System.Threading.CancellationToken cancellationToken) <0x36ee098 + 0x00134> in <filename unknown>:0 
  at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore (System.Threading.Tasks.Task serializeToStreamTask, System.IO.MemoryStream tempBuffer) <0x49ea648 + 0x00110> in <filename unknown>:0 
  at System.Net.Http.HttpContent.WaitAndReturnAsync[TState,TResult] (System.Threading.Tasks.Task waitTask, TState state, System.Func`2[T,TResult] returnFunc) <0x49eba60 + 0x000c2> in <filename unknown>:0 
  at System.Net.Http.WebAssemblyHttpHandler.doFetch (System.Threading.Tasks.TaskCompletionSource`1[TResult] tcs, System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) <0x47cfe90 + 0x0038c> in <filename unknown>:0 
  at System.Net.Http.WebAssemblyHttpHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) <0x47af880 + 0x00174> in <filename unknown>:0 
  at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered (System.Threading.Tasks.Task`1[TResult] sendTask, System.Net.Http.HttpRequestMessage request, System.Threading.CancellationTokenSource cts, System.Boolean disposeCts) <0x49ef528 + 0x00134> in <filename unknown>:0 
  at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync (System.Threading.CancellationToken cancellationToken) [0x0003d] in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\HttpHandler\_netstandard\HttpRequestMessageFactory.cs:520 
  at Amazon.Runtime.Internal.HttpHandler`1[TRequestContent].InvokeAsync[T] (Amazon.Runtime.IExecutionContext executionContext) [0x00201] in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\HttpHandler\HttpHandler.cs:183 
  at Amazon.Runtime.Internal.RedirectHandler.InvokeAsync[T] (Amazon.Runtime.IExecutionContext executionContext) [0x00036] in D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\RedirectHandler.cs:59 
  at Amazon.Runtime.Internal.U
f.printErr  @   blazor.webassembly.js:1
f.preRun.push.window.Blazor._internal.dotNetCriticalError   @   blazor.webassembly.js:1
_mono_wasm_invoke_js_unmarshalled   @   dotnet.3.2.0.js:1
do_icall    @   00755c3a:0x10f924
do_icall_wrapper    @   00755c3a:0x50b6a
interp_exec_method  @   00755c3a:0x2588e
interp_runtime_invoke   @   00755c3a:0xf7391
mono_jit_runtime_invoke @   00755c3a:0xddb3d
do_runtime_invoke   @   00755c3a:0x3ba85
mono_runtime_try_invoke @   00755c3a:0xcfdb
try_invoke_perform_wait_callback    @   00755c3a:0xb62e0
worker_callback @   00755c3a:0x9a7a3
fire_tp_callback    @   00755c3a:0x105481
mono_background_exec    @   00755c3a:0xeb5ee
Module._mono_background_exec    @   dotnet.3.2.0.js:1
pump_message    @   dotnet.3.2.0.js:1
setTimeout (async)      
_schedule_background_exec   @   dotnet.3.2.0.js:1
mono_threads_schedule_background_job    @   00755c3a:0x14dbd
ves_icall_System_Threading_ThreadPool_RequestWorkerThread   @   00755c3a:0xa0025
ves_icall_System_Threading_ThreadPool_RequestWorkerThread_raw   @   00755c3a:0x9fecf
do_icall    @   00755c3a:0x10f790
do_icall_wrapper    @   00755c3a:0x50b6a
interp_exec_method  @   00755c3a:0x2588e
interp_runtime_invoke   @   00755c3a:0xf7391
mono_jit_runtime_invoke @   00755c3a:0xddb3d
do_runtime_invoke   @   00755c3a:0x3ba85
mono_runtime_try_invoke @   00755c3a:0xcfdb
mono_runtime_invoke @   00755c3a:0x44b39
mono_wasm_invoke_method @   00755c3a:0xca6a7
Module._mono_wasm_invoke_method @   dotnet.3.2.0.js:1
call_method @   dotnet.3.2.0.js:1
(anonymous) @   dotnet.3.2.0.js:1
endInvokeJSFromDotNet   @   blazor.webassembly.js:1
(anonymous) @   blazor.webassembly.js:1
Promise.then (async)        
beginInvokeJSFromDotNet @   blazor.webassembly.js:1
_mono_wasm_invoke_js_marshalled @   dotnet.3.2.0.js:1
do_icall    @   00755c3a:0x10f8b1
do_icall_wrapper    @   00755c3a:0x50b6a
interp_exec_method  @   00755c3a:0x2588e
interp_runtime_invoke   @   00755c3a:0xf7391
mono_jit_runtime_invoke @   00755c3a:0xddb3d
do_runtime_invoke   @   00755c3a:0x3ba85
mono_runtime_invoke_checked @   00755c3a:0x96a2
mono_runtime_try_invoke_array   @   00755c3a:0x66c26
ves_icall_InternalInvoke    @   00755c3a:0xb44ff
ves_icall_InternalInvoke_raw    @   00755c3a:0xb3ffd
do_icall    @   00755c3a:0x10f8b1
do_icall_wrapper    @   00755c3a:0x50b6a
interp_exec_method  @   00755c3a:0x2588e
interp_runtime_invoke   @   00755c3a:0xf7391
mono_jit_runtime_invoke @   00755c3a:0xddb3d
do_runtime_invoke   @   00755c3a:0x3ba85
mono_runtime_try_invoke @   00755c3a:0xcfdb
mono_runtime_invoke @   00755c3a:0x44b39
mono_wasm_invoke_method @   00755c3a:0xca6a7
Module._mono_wasm_invoke_method @   dotnet.3.2.0.js:1
call_method @   dotnet.3.2.0.js:1
(anonymous) @   dotnet.3.2.0.js:1
beginInvokeDotNetFromJS @   blazor.webassembly.js:1
s   @   blazor.webassembly.js:1
e.invokeMethodAsync @   blazor.webassembly.js:1
(anonymous) @   blazor.webassembly.js:1
t.dispatchEvent @   blazor.webassembly.js:1
(anonymous) @   blazor.webassembly.js:1
(anonymous) @   blazor.webassembly.js:1
e.onGlobalEvent @   blazor.webassembly.js:1

请告知我需要做哪些进一步的测试以确定问题所在,以及如何解决。

编辑更新

根据以下答案之一,我改用 PutObjectRequest。代码如下。这现在给出了一个新错误,我认为这与 S3 存储桶的权限有关。谁能指导我需要更改什么?

代码

public class UploadData
{
    private readonly BasicAWSCredentials credentials = new BasicAWSCredentials("access key", "secret Key");

    public async Task UploadZipFile(Stream zip_file, string file_name)
    {
        zip_file.Seek(0, SeekOrigin.Begin);
        var config = new AmazonS3Config
        {
            RegionEndpoint = Amazon.RegionEndpoint.EUWest2
        };
        using var client = new AmazonS3Client(credentials, config);

        var putRequest1 = new PutObjectRequest
        {
            BucketName = "Bucket_name",
            Key = file_name,
            InputStream = zip_file
        };
        PutObjectResponse response1 = await client.PutObjectAsync(putRequest1);

        Console.WriteLine("finshed uploading");
    }
}

这是错误代码

Unknown encountered on server. Message:'TypeError: Failed to execute 'append' on 'Headers': Invalid value' when putting an object

我对 blazor 和 webassembly 了解不多,但我可以尝试并给你一个建议:

  1. 上面代码中的问题似乎是 webassembly 中缺少多线程支持,另请参见 [1]。
  2. aws-sdk-net 中的 TransferUtility class 使用线程进行分段上传,参见 [2]:

TransferUtility provides a simple API for uploading content to and downloading content from Amazon S3. It makes extensive use of Amazon S3 multipart uploads to achieve enhanced throughput, performance, and reliability.

When uploading large files by specifying file paths instead of a stream, TransferUtility uses multiple threads to upload multiple parts of a single upload at once. When dealing with large content sizes and high bandwidth, this can increase throughput significantly.

因此,我会尝试使用一些低级构造来使用 PutObjectRequest 完成标准上传,请参阅:https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html

如果要上传的文件不是太大,这应该没有任何问题,因为它是单线程方法。

另一个问题是在 Blazor WASM 中启用多线程,但我不确定这是否已经是一个选项,请参阅 [3]。

[1] https://www.meziantou.net/don-t-freeze-ui-while-executing-cpu-intensive-work-in-blazor-webassembly.htm
[2] https://docs.aws.amazon.com/sdkfornet/v3/apidocs/items/S3/TTransferUtility.html
[3] https://github.com/dotnet/aspnetcore/issues/17730