发出 S3 GET 请求时收到 400 错误请求

Question

我正在尝试向 S3 发出 AWS 版本 4 授权签名 GET 请求，并收到此请求的错误请求错误 400 Code:InvalidRequest Message:Missing required header：x-amz-content-sha256

如果我在 header 前加上 "Authorization: "，我会得到错误 Code:InvalidArgument Message:Unsupported Authorization Type <ArgumentName>Authorization</ArgumentName> <ArgumentValue>Authorization: AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXX/20200408/eu-west-3/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=vdchzint97uwyt3g%2fjehszrc8zpkbjsx4tfqacsqfow%3d</ArgumentValue>

我正在使用 Delphi XE5 和 Indy 的 TIdHTTP 组件。谁能告诉我我做错了什么？我在下面包含了我的代码。

begin
  bucket := 'mybucket.ata-test';
  obj := 'test.xml';
  region := 'eu-west-3';
  service := 's3';
  aws := 'amazonaws.com';
  YYYYMMDD := FormatDateTime('yyyymmdd', now);
  amzDate := FormatDateTime('yyyymmdd"T"hhnnss"Z"', TTimeZone.Local.ToUniversalTime(Now), TFormatSettings.Create('en-US'));
  emptyHash := lowercase(SHA256HashAsHex(''));
  host := Format('%s.%s.%s.%s', [bucket, service, region, aws]);
  url := Format('%s://%s.%s.%s.%s/%s', ['https', bucket, service, region, aws, obj]);

// *** 1. Build the Canonical Request for Signature Version 4 ***
  // HTTPRequestMethod
  CanonicalRequest := URLEncodeValue('GET') +#10;
  // CanonicalURI
  CanonicalRequest := CanonicalRequest + '/' + URLEncodeValue(obj) +#10;
  // CanonicalQueryString (empty just a newline)
  CanonicalRequest := CanonicalRequest +#10;
  // CanonicalHeaders
  CanonicalRequest := CanonicalRequest + 'host:' + Trim(host) +#10
                                       + 'x-amz-content-sha256:' + emptyHash +#10
                                       + 'x-amz-date:' + Trim(amzDate) +#10;
  // SignedHeaders
  CanonicalRequest := CanonicalRequest + 'host;x-amz-content-sha256;x-amz-date' +#10;
  // HexEncode(Hash(RequestPayload)) - (hash of an empty string)
  CanonicalRequest := CanonicalRequest + emptyHash;

// *** 2. Create a String to Sign for Signature Version 4 ***
  StringToSign := 'AWS4-HMAC-SHA256' +#10
                  + amzDate +#10
                  + UTF8String(YYYYMMDD) +'/'+ UTF8String(region) +'/'+ UTF8String(service) +UTF8String('/aws4_request') +#10
                  + lowercase(SHA256HashAsHex(CanonicalRequest));

// *** 3. Calculate the Signature for AWS Signature Version 4 ***
  DateKey := CalculateHMACSHA256(YYYYMMDD, 'AWS4' + SecretAccessKey);
  DateRegionKey := CalculateHMACSHA256(region, DateKey);
  DateRegionServiceKey := CalculateHMACSHA256(service, DateRegionKey);
  SigningKey := CalculateHMACSHA256('aws4_request', DateRegionServiceKey);

  Signature := lowercase(UrlEncodeValue(CalculateHMACSHA256(StringToSign, SigningKey)));

// *** 4. Create Authorisation Header and Add the Signature to the HTTP Request ***
  AuthorisationHeader := 'AWS4-HMAC-SHA256 Credential='+AccessIdKey+'/'+YYYYMMDD+'/'+region+'/'+service+'/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature='+signature;  
// (Gives <Code>InvalidRequest</Code> <Message>Missing required header for this request: x-amz-content-sha256</Message>)

// Have also tried
// AuthorisationHeader := 'Authorization: AWS4-HMAC-SHA256 Credential='+AccessIdKey+'/'+YYYYMMDD+'/'+region+'/'+service+'/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature='+signature;  
// (Gives <Code>InvalidArgument</Code> <Message>Unsupported Authorization Type</Message>)

// *** 5. Add Header and Make Request ***
  stm := TMemoryStream.Create;
  try
    try
      Idhttp.Request.CustomHeaders.FoldLines := False;
      Idhttp.Request.CustomHeaders.AddValue('Authorization', AuthorisationHeader);
      Idhttp.Get(URL, stm);
    except
      on PE: EIdHTTPProtocolException do begin
        s := PE.ErrorMessage;
        Raise;
      end;
      on E: Exception do begin
        s := E.Message;
        Raise;
      end;
    end;

    stm.Position := 0;
    Memo1.Lines.LoadFromStream(stm);
  finally
    FreeAndNil(stm);
  end; 
end;

function SHA256HashAsHex(const value: string): String;
/// used for stringtosign
var
  sha: TIdHashSHA256;
begin
  LoadOpenSSLLibrary;
  if not TIdHashSHA256.IsAvailable then
    raise Exception.Create('SHA256 hashing is not available!');
  sha := TIdHashSHA256.Create;
  try
    result := sha.HashStringAsHex(value, nil);
  finally
    sha.Free;
  end;
end;

function CalculateHMACSHA256(const value, salt: String): String;
/// used for signingkey
var
  hmac: TIdHMACSHA256;
  hash: TIdBytes;
begin
  LoadOpenSSLLibrary;
  if not TIdHashSHA256.IsAvailable then
    raise Exception.Create('SHA256 hashing is not available!');
  hmac := TIdHMACSHA256.Create;
  try
    hmac.Key := IndyTextEncoding_UTF8.GetBytes(salt);
    hash := hmac.HashValue(IndyTextEncoding_UTF8.GetBytes(value));
    Result := EncodeBytes64(TArray<Byte>(hash));
  finally
    hmac.Free;
  end;
end;

Answer 1

我在您的代码中注意到一些事情：

在创建 YYYYMMDD 和 amzDate 值时，您调用了 Now() 两次，这会创建具有潜力的竞争条件导致这些变量代表不同的日期。不太可能，但有可能。为避免这种情况，您应该只调用 1 次 Now() 并将结果保存到本地 TDateTime 变量，然后在所有 FormatDateTime() 调用中使用该变量。

dtNow := Now();
YYYYMMDD := FormatDateTime('yyyymmdd', dtNow);
amzDate := FormatDateTime('yyyymmdd"T"hhnnss"Z"', TTimeZone.Local.ToUniversalTime(dtNow), TFormatSettings.Create('en-US'));

当使用 TIdHTTP 的 Request.CustomHeaders 属性设置自定义 Authorization header 时，确保您也设置了 Request.BasicAuthentication 属性也为 False，否则 TIdHTTP 可能会使用其 Request.Username 和 Request.Password 属性创建自己的 Authorization: Basic ... header。您不希望 GET 请求中有两个 Authorization header。

Idhttp.Request.BasicAuthentication := False;

您在授权计算中使用了 x-amz-content-sha256 和 x-amz-date header，但您没有将这些 header 添加到实际的 HTTP 请求中。 TIdHTTP 将为您添加 Host header，但您需要自己添加其他 header。

Idhttp.Request.CustomHeaders.AddValue('x-amz-content-sha256', emptyHash);
Idhttp.Request.CustomHeaders.AddValue('x-amz-date', amzDate);

您的 SHA256HashAsHex() 函数在调用 Indy 的 TIdHashSHA256.HashStringAsHex() 方法时未指定字节编码（事实上，它特意将编码显式设置为 nil） .因此，将使用 Indy 的默认字节编码，即 US-ASCII（除非您在 IdGlobal 单元中将 Indy 的 GIdDefaultTextEncoding 变量设置为其他内容）。但是，您的 CalculateHMACSHA256() 函数明确使用了 UTF-8。您的 SHA256HashAsHex() 函数应该使用 IndyTextEncoding_UTF8 来匹配：

result := sha.HashStringAsHex(value, IndyTextEncoding_UTF8);

CalculateHMACSHA256() 的输入盐和输出值需要是二进制字节，而不是字符串，当然也不是 base64 编码或 hex-encoded 字符串。 Calculate the Signature for AWS Signature Version 4 文档中没有任何内容提及 base64 的使用。

var
  DateKey, RegionKey, ServiceKey, SigningKey: TArray<Byte>;
...

// *** 3. Calculate the Signature for AWS Signature Version 4 ***
DateKey := CalculateHMACSHA256(YYYYMMDD, TEncoding.UTF8.GetBytes('AWS4' + SecretAccessKey));
RegionKey := CalculateHMACSHA256(region, DateKey);
ServiceKey := CalculateHMACSHA256(service, RegionKey);
SigningKey := CalculateHMACSHA256('aws4_request', ServiceKey);

Signature := CalculateHMACSHA256Hex(StringToSign, SigningKey); 

... 

function CalculateHMACSHA256(const value: string; const salt: TArray<Byte>): TArray<Byte>;
/// used for signingkey
var
  hmac: TIdHMACSHA256;
  hash: TIdBytes;
begin
  LoadOpenSSLLibrary;
  if not TIdHashSHA256.IsAvailable then
    raise Exception.Create('SHA256 hashing is not available!');
  hmac := TIdHMACSHA256.Create;
  try
    hmac.Key := TIdBytes(salt);
    hash := hmac.HashValue(IndyTextEncoding_UTF8.GetBytes(value));
    Result := TArray<Byte>(hash);
  finally
    hmac.Free;
  end;
end;

function CalculateHMACSHA256Hex(const value: string; const salt: TArray<Byte>): string;
var
  hash: TArray<Byte>;
begin
  hash := CalculateHMACSHA256(value, salt)
  Result := lowercase(ToHex(TIdBytes(hash)));
end;

Answer 2

我刚刚遇到了类似的问题，我会在这里留下我的贡献，因为这个 post 帮助我找到了解决方案。在我的例子中，我需要生成一个带有特定到期时间的签名 url。

    unit Data.Cloud.AmazonAPI.Utils;
    
    interface
    
      //See https://docs.aws.amazon.com/pt_br/AmazonS3/latest/API/sigv4-query-string-auth.html#query-string-auth-v4-signing-example
      //Use example:
      // MyUlrSigned := GetUrlPreSigned('mybucketname','/MyFolder/MyFile.zip','sa-east-1',3600);
    
      function GetUrlPreSigned(ABucket:string;AObjectName:string;ARegion:string;AExpiresIn:Int64=3600):string;
    
    implementation
    
    uses
      System.Classes, System.SysUtils, System.Hash, System.DateUtils;
    
    const
     
      AWS_ACCOUNTNAME     = '<AWSAccessKeyId>';
      AWS_ACCOUNTKEY      = '<AWSSecretAccessKey>';
    
    function SignString(const Signkey: TBytes; const StringToSign: string): TBytes;
    begin
      Result := THashSHA2.GetHMACAsBytes(StringToSign, Signkey);
    end;
    
    function BuildSignature(const StringToSign, DateISO, Region:string; AService: string; LSecretAccessKey:string): string;
    
      function GetSignatureKey(const datestamp, region, serviceName: string): TBytes;
      begin
        Result := SignString(TEncoding.Default.GetBytes('AWS4'+LSecretAccessKey) ,datestamp);
        Result := SignString(Result, region);
        Result := SignString(Result, serviceName);
        Result := SignString(Result, 'aws4_request');
      end;
    
    var
      Signature:string;
      SigningKey : TBytes;
    begin
      SigningKey := GetSignatureKey(DateISO, Region, AService);
      Result  := THash.DigestAsString(SignString(SigningKey, StringToSign));
    end;
    
    function GetHashSHA256Hex( HashString: string): string;
    var
      LBytes: TArray<Byte>;
    begin
      LBytes := THashSHA2.GetHashBytes(HashString);
      Result := THash.DigestAsString(LBytes);
    end;
    
    function GetUrlPreSigned(ABucket:string;AObjectName:string;ARegion:string;AExpiresIn:Int64=3600):string;
    var
      LNow : TDateTime;
      LData : string;
      LTimeStamp : string;
      LAccessKey : string;
      LSecretAccessKey : string;
      LService : string;
      LAws : string;
      LHost : string;
      LUrl : string;
      LQueryParams : string;
      LCanonicalRequest : string;
      LStringToSign : string;
      LSignature : string;
    begin
      LNow := Now();
      LData := FormatDateTime('yyyymmdd', LNow);
      LTimeStamp :=  FormatDateTime('yyyymmdd"T"hhnnss"Z"', TTimeZone.Local.ToUniversalTime(LNow), TFormatSettings.Create('en-US'));
      LAccessKey       := AWS_ACCOUNTNAME;
      LSecretAccessKey := AWS_ACCOUNTKEY;
    
      if AObjectName.StartsWith('/') then
        Delete(AObjectName,1,1);
    
      LService := 's3';
      LAws := 'amazonaws.com';
      LHost := Format('%s-%s.%s', [LService, ARegion, LAws]);
      LUrl := Format('%s://%s-%s.%s/%s/%s', ['https', LService, ARegion, LAws, ABucket, AObjectName]);
    
      LQueryParams := 'X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential='+LAccessKey
                      +'%2F'+LData
                      +'%2F'+ARegion
                      +'%2F'+LService+'%2F'
                      +'aws4_request'
                      +'&X-Amz-Date='+LTimeStamp
                      +'&X-Amz-Expires='+AExpiresIn.ToString
                      +'&X-Amz-SignedHeaders=host';
    
      //1 - CanonicalRequest
      LCanonicalRequest := 'GET' +#10+
                        '/'+ABucket+'/'+AObjectName +#10
                        +LQueryParams +#10
                        +'host:'+LHost+#10
                        +#10
                        +'host'+#10
                        +'UNSIGNED-PAYLOAD';
    
      //2 - StringToSign
      LStringToSign := 'AWS4-HMAC-SHA256' +#10
                      + LTimeStamp +#10
                      + UTF8String(LData)+'/'+ UTF8String(ARegion) +'/'+ UTF8String(LService)+UTF8String('/aws4_request') +#10
                      + lowercase(GetHashSHA256Hex(LCanonicalRequest));
      //3 - Signature
      LSignature := BuildSignature(LStringToSign,LData,ARegion,LService,LSecretAccessKey);
    
      //4 - Signed URL
      Result := LUrl+'?'+LQueryParams+'&X-Amz-Signature='+LSignature;
    end;
    
    end.

发出 S3 GET 请求时收到 400 错误请求

Receiving 400 Bad request when making S3 GET request

amazon-s3

indy10

delphi-xe