读取的数据长度与预期不同：dataLength=964481363；预期长度=964481376；

Question

我正在尝试从 S3 读取加密对象，然后使用不同的加密密钥将其上传到 S3 中的新存储桶。当我尝试读取对象时，使用 getObject() 并尝试使用 PutObject 请求放置对象。

当我打印source的content length时，是964481376。那么，这个数据Length是从哪里来的呢？这是 964481363

我的源文件的元数据如下所示：

内容类型

binary/octet-stream

x-amz-meta-x-amz-未加密内容长度

964481363

x-amz-meta-x-amz-wrap-alg

公里数

我无法共享 KMS 密钥。

public boolean copyS3Object(AmazonS3Encryption sourceS3Client, AmazonS3URI sourceS3Uri,
            AmazonS3Encryption destS3Client, AmazonS3URI destS3Uri) throws AmazonServiceException, IOException {
        //String eTag;
        S3Object sourceS3Object = null;

        try {
            //Get the Source object stream
            sourceS3Object = sourceS3Client.getObject(sourceS3Uri.getBucket(), sourceS3Uri.getKey());
            ObjectMetadata objectMetadata = sourceS3Object.getObjectMetadata();
            System.out.println("content length " + objectMetadata.getContentLength());
            PutObjectRequest putRequest = new PutObjectRequest(destS3Uri.getBucket(), destS3Uri.getKey(),
                    sourceS3Object.getObjectContent(), sourceS3Object.getObjectMetadata());

            destS3Client.putObject(putRequest);
//
//            if (eTag.isEmpty()) {
//                System.out.println("Copy failed, New object in " + destS3Uri.toString() + " is empty");
//                return false;
//            }

        } catch (AmazonServiceException e) {
            throw e;
        } finally {
            if (sourceS3Object != null) {
                sourceS3Object.close();
            }
        }
        //System.out.println("Copied successfully to " + destS3Uri.toString() + " Etag:" + eTag);
        return true;
    }

但我收到以下错误：

{ "errorMessage": "Data read has a different length than the expected: dataLength=964481363; expectedLength=964481376; includeSkipped=false; in.getClass()=class com.amazonaws.internal.ReleasableInputStream; markedSupported=false; marked=0; resetSinceLastMarked=false; markCount=0; resetCount=0", "errorType": "com.amazonaws.SdkClientException", "stackTrace"：[ "com.amazonaws.util.LengthCheckInputStream.checkLength(LengthCheckInputStream.java:151)", "com.amazonaws.util.LengthCheckInputStream.read(LengthCheckInputStream.java:109)", "java.io.FilterInputStream.read(FilterInputStream.java:107)", "com.amazonaws.services.s3.internal.crypto.CipherLiteInputStream.nextChunk(CipherLiteInputStream.java:225)", "com.amazonaws.services.s3.internal.crypto.CipherLiteInputStream.read(CipherLiteInputStream.java:118)", "com.amazonaws.services.s3.internal.crypto.RenewableCipherLiteInputStream.read(RenewableCipherLiteInputStream.java:112)", "com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:90)", "com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:90)", "com.amazonaws.util.LengthCheckInputStream.read(LengthCheckInputStream.java:107)", "com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:90)", "com.amazonaws.services.s3.internal.MD5DigestCalculatingInputStream.read(MD5DigestCalculatingInputStream.java:128)", "java.io.BufferedInputStream.fill(BufferedInputStream.java:246)", "java.io.BufferedInputStream.read1(BufferedInputStream.java:286)", "java.io.BufferedInputStream.read(BufferedInputStream.java:345)", "com.amazonaws.internal.SdkBufferedInputStream.read(SdkBufferedInputStream.java:76)", "com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:90)", "com.amazonaws.event.ProgressInputStream.read(ProgressInputStream.java:180)", "com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:90)", "org.apache.http.entity.InputStreamEntity.writeTo(InputStreamEntity.java:140)", "com.amazonaws.http.RepeatableInputStreamRequestEntity.writeTo(RepeatableInputStreamRequestEntity.java:160)", "org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156)", "org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:162)", "org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)", "com.amazonaws.http.protocol.SdkHttpRequestExecutor.doSendRequest(SdkHttpRequestExecutor.java:63)", "org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)", "org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)", "org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)", "org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)", "org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)", "org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)", "com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)", "com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1297)", "com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)", "com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)", "com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)", "com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)", "com.amazonaws.http.AmazonHttpClient$RequestExecutor.access0(AmazonHttpClient.java:686)", "com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)", "com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)", "com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)", "com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4914)", "com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4860)", "com.amazonaws.services.s3.AmazonS3Client.access0(AmazonS3Client.java:389)", "com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:5793)", "com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1786)", "com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1746)", "com.amazonaws.services.s3.AmazonS3EncryptionClient.access1(AmazonS3EncryptionClient.java:81)", "com.amazonaws.services.s3.AmazonS3EncryptionClient$S3DirectImpl.putObject(AmazonS3EncryptionClient.java:690)", "com.amazonaws.services.s3.internal.crypto.S3CryptoModuleBase.putObjectUsingMetadata(S3CryptoModuleBase.java:175)", "com.amazonaws.services.s3.internal.crypto.S3CryptoModuleBase.putObjectSecurely(S3CryptoModuleBase.java:161)", "com.amazonaws.services.s3.internal.crypto.CryptoModuleDispatcher.putObjectSecurely(CryptoModuleDispatcher.java:108)", "com.amazonaws.services.s3.AmazonS3EncryptionClient.putObject(AmazonS3EncryptionClient.java:570)", "com.amazon.testCase.lambda.object.UploadObject.coyS3Object(UploadObject.java:93)", "com.amazon.testCase.lambda.object.UploadObject.handleRequest(UploadObject.java:54)", "sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)", "sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)", "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)", "java.lang.reflect.Method.invoke(Method.java:498)" ] }

Answer 1

我想通了。 S3 对象的内容长度为加密后的数据长度。但是当我放置对象时，我试图放置未加密的数据。所以我需要明确地将对象的内容长度设置为未加密的长度。下面的代码解决了这个问题。

 try {
            //Get the Source object stream
            sourceS3Object = sourceS3Client.getObject(sourceS3Uri.getBucket(), sourceS3Uri.getKey());
            //For a decrypted object, the content length in metadata has the encrypted length
            //Set the content length to the unencrypted-data-length
            ObjectMetadata objectMetadata = sourceS3Object.getObjectMetadata();
            Map<String, String> modelMetaData = objectMetadata.getUserMetadata();
            Long unencryptedDataLength = Long.parseLong(modelMetaData.get(
                    "x-amz-unencrypted-content-length"));
            objectMetadata.setContentLength(unencryptedDataLength);
            PutObjectRequest putRequest = new PutObjectRequest(destS3Uri.getBucket(), destS3Uri.getKey(),
                    sourceS3Object.getObjectContent(), objectMetadata);

            eTag = destS3Client.putObject(putRequest).getETag();

            if (eTag.isEmpty()) {
                System.out.println("Copy failed, New object in " + destS3Uri.toString() + " is empty");
                return false;
            }

读取的数据长度与预期不同：dataLength=964481363；预期长度=964481376；

Data read has a different length than the expected: dataLength=964481363; expectedLength=964481376;

java

amazon-s3

amazon-web-services

aws-sdk