Api 网关 websocket $connect 从 http 集成点获得 400,但它适用于其他路由

Api Gateway websocket $connect gets a 400 from the http integration point but it works for other routes

我对使用 WEBSOCKET

的 AWS API 网关的 HTTP 集成有问题

这些是我配置的主要特点(可以在post末尾的云阵模板中详细查看):

当我做的时候

wscat -c wss://ws.mycompany.io

我明白了

error: Unexpected server response: 400

我可以看到一个没有太多信息的 cloudwatch 日志

{"requestTime":"21/May/2021:15:20:06 +0000","requestId":"fr1iDG7CjoEFWTg=","httpMethod":"-","path":"-","routeKey":"$connect","status":400,"responseLatency":-}

为该 websocket api /aws/apigateway/i4vdq18wg9/production 自动创建了另一个日志组,其中包含以下内容

(fr1iDG7CjoEFWTg=) Extended Request Id: fr1iDG7CjoEFWTg=
(fr1iDG7CjoEFWTg=) Verifying Usage Plan for request: fr1iDG7CjoEFWTg=. API Key:  API Stage: i4vdq18wg9/production
(fr1iDG7CjoEFWTg=) API Key  authorized because route '$connect' does not require API Key. Request will not contribute to throttle or quota limits
(fr1iDG7CjoEFWTg=) Usage Plan check succeeded for API Key  and API Stage i4vdq18wg9/production
(fr1iDG7CjoEFWTg=) Starting execution for request: fr1iDG7CjoEFWTg=
(fr1iDG7CjoEFWTg=) WebSocket Request Route: [$connect]
(fr1iDG7CjoEFWTg=) Client [UserAgent: null, SourceIp: 91.194.63.143] is connecting to WebSocket API [i4vdq18wg9].
(fr1iDG7CjoEFWTg=) Endpoint request URI: https://api.mycompany.io/MyCompany.MyService/ws/connect
(fr1iDG7CjoEFWTg=) Endpoint request headers: {Sec-WebSocket-Key=7kp7mwI9YH+WjoGeiGjMEg==, x-amzn-apigateway-api-id=i4vdq18wg9, stage=production, domainName=ws.mycompany.io, X-Forwarded-Proto=https, User-Agent=AmazonAPIGateway_i4vdq18wg9, connectionId=fr1iDchODoECFIQ=, Sec-WebSocket-Version=13, X-Forwarded-For=91.194.63.143, X-Forwarded-Port=443, X-Amzn-Trace-Id=Root=1-60a7cfa6-45d1a7387a071f9c6547dfad, Sec-WebSocket-Extensions=permessage-deflate; client_max_window_bits}
(fr1iDG7CjoEFWTg=) Endpoint request body after transformations: 
(fr1iDG7CjoEFWTg=) Sending request to https://api.mycompany.io/MyCompany.MyService/ws/connect
(fr1iDG7CjoEFWTg=) Received response. Status: 400, Integration latency: 8 ms
(fr1iDG7CjoEFWTg=) Endpoint response headers: {Server=awselb/2.0, Date=Fri, 21 May 2021 15:20:06 GMT, Content-Type=application/json; charset=utf-8, Content-Length=11, Connection=keep-alive}
(fr1iDG7CjoEFWTg=) Endpoint response body before transformations: Bad Request
(fr1iDG7CjoEFWTg=) Client [UserAgent: null, SourceIp: 91.194.63.143] failed to connect to API [i4vdq18wg9]. 

因此,根据集成,对 http 服务的请求似乎失败了 400 Bad Request。不知道为什么。

我决定只为 $connect 集成修改端点,send 将像以前一样指向我的服务。我将 IntegrationUri 更改为指向蜂鸣器,这样我就可以看到 $connect 请求是如何到达服务的:https://mycompanyinterceptor.free.beeceptor.com/ws/connect

然后我再次尝试连接 websocket。

wscat -c wss://ws.mycompany.io

Connected (press CTRL+C to quit)
>

现在可以了。因此,要么我的服务没有正确处理 $connect(看不出为什么),要么 URL 有一些特殊的东西,因为它调用另一个 API 网关,在自定义域后面..

我可以在 beeceptor 看到,$connect 作为 GET 请求到达 与以下 headers

{
  "stage": "production",
  "domainname": "ws.mycompany.io",
  "connectionid": "fr2_RcLUDoECEWQ=",
  "sec-websocket-extensions": "permessage-deflate; client_max_window_bits",
  "sec-websocket-key": "iOkdWsyZw/P8GjSwXwHrbg==",
  "sec-websocket-version": "13",
  "x-amzn-trace-id": "Root=1-60a7d1fb-4c42b6e717aa2c666ecae2ab",
  "x-forwarded-for": "91.194.63.143",
  "x-forwarded-port": "443",
  "x-forwarded-proto": "https",
  "user-agent": "AmazonAPIGateway_i4vdq18wg9",
  "x-amzn-apigateway-api-id": "i4vdq18wg9"
}

Beeceptor 以 200 OK 响应,随后 headers

{
  "content-type": "text/plain",
  "access-control-allow-origin": "*",
  "vary": "Accept-Encoding"
}

还有这个body

Hey ya! Great to see you here. Btw, nothing is configured for this request path. Create a rule and start building a mock API.

现在,我决定 send 通过我的 websocket 转发一条消息到我的 send 路由和我的 http 集成服务

$ wscat -c wss://ws.rubiko.io
Connected (press CTRL+C to quit)
> {'action': 'send', 'data': 'my service should just reply with a 200 OK and the path invoked'}
< OK: /ws/send
>

如您所见,问题不在于与我的服务的连接,因为其他 websocket 请求已正确到达。

我做错了什么? 据我所知,可能影响 $connect 的“不同”事物是:

运行 这里没有想法:-(


CloudFormation 模板

AWSTemplateFormatVersion: 2010-09-09

Description: API Gateway for HTTP and Websocket with custom domains

Parameters:
  hostedZoneId:
    Type: AWS::Route53::HostedZone::Id
    Description: the public hosted zone
    Default: Z0561635YJ0IZJCMK5TF
  certificateArn:
    Type: String
    Description: The TLS certificate ARN
    Default: arn:aws:acm:eu-west-1:754027052283:certificate/35b3a270-4dc3-4935-b108-d22dff688b1f
  prefix:
    Type: String
    Description: The prefix namespace or company name
    Default: mycompany

Mappings:
  gateways:
    api:
      domainName: api.mycompany.io
      logGroupName: /aws/api-mycompany/mycompany-http-gateway-logs
    ws:
      domainName: ws.mycompany.io
      logGroupName: /aws/api-mycompany/mycompany-ws-gateway-logs

Resources:
  privateLink:
    Type: AWS::ApiGatewayV2::VpcLink
    Properties:
      Name: !Sub ${prefix}-private-link
      SecurityGroupIds:
        - Fn::ImportValue: !Sub ${prefix}-web-sg-id
      SubnetIds:
        - Fn::ImportValue: !Sub ${prefix}-private-a-id
        - Fn::ImportValue: !Sub ${prefix}-private-b-id
        - Fn::ImportValue: !Sub ${prefix}-private-c-id
  
  apiGatewayAccountConfig:
    Type: "AWS::ApiGateway::Account"
    Properties:
      CloudWatchRoleArn: !GetAtt apiGatewayWatchLogsRole.Arn

  apiGatewayWatchLogsRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub ${prefix}-api-gateway-logs
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: apigateway.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs

  # Microservices Api Gateway
  apiGatewayDomain:
    Type: AWS::ApiGatewayV2::DomainName
    Properties:
      DomainName: !FindInMap [gateways, api, domainName]
      DomainNameConfigurations:
        - CertificateArn: !Ref certificateArn
          CertificateName: !Sub ${prefix}-certificate
          SecurityPolicy: TLS_1_2

  apiGatewayLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !FindInMap [gateways, api, logGroupName]

  apiGateway:
    Type: AWS::ApiGatewayV2::Api
    Properties:
      Name: !Sub ${prefix}-http-gateway
      Description: Api Gateway for http
      ProtocolType: HTTP
      DisableExecuteApiEndpoint: true

  apiRoute:
    Type: AWS::ApiGatewayV2::Route
    Properties:
      ApiId: !Ref apiGateway
      RouteKey: ANY /{proxy+}
      Target: !Join
        - /
        - - integrations
          - !Ref apiAlbIntegration

  apiAlbIntegration:
    Type: AWS::ApiGatewayV2::Integration
    Properties:
      ApiId: !Ref apiGateway
      Description: Private ALB integration on 80
      IntegrationType: HTTP_PROXY
      IntegrationMethod: ANY
      ConnectionType: VPC_LINK
      ConnectionId: !Ref privateLink
      IntegrationUri: 
        Fn::ImportValue: !Sub ${prefix}-alb-http-listener-id
      PayloadFormatVersion: 1.0

  apiStage:
    Type: AWS::ApiGatewayV2::Stage
    Properties:
      StageName: $default
      AutoDeploy: true
      ApiId: !Ref apiGateway
      AccessLogSettings:
        DestinationArn: !GetAtt apiGatewayLogGroup.Arn
        Format: '{"requestTime":"$context.requestTime","requestId":"$context.requestId","httpMethod":"$context.httpMethod","path":"$context.path","routeKey":"$context.routeKey","status":$context.status,"responseLatency":$context.responseLatency}'
  
  apiMapping:
    Type: AWS::ApiGatewayV2::ApiMapping
    DependsOn: apiGatewayDomain
    Properties: 
      ApiId: !Ref apiGateway
      DomainName: !FindInMap [gateways, api, domainName]
      Stage: !Ref apiStage
  
  aliasDns:
    Type: AWS::Route53::RecordSet
    Properties:
      Comment: DNS entry for public custom domain to alias microservices API URL
      HostedZoneId: !Ref hostedZoneId
      Name: !FindInMap [gateways, api, domainName]
      Type: A
      AliasTarget:
        DNSName: !GetAtt apiGatewayDomain.RegionalDomainName
        HostedZoneId: !GetAtt apiGatewayDomain.RegionalHostedZoneId
        EvaluateTargetHealth: true
  
  # Websocket Api Gateway
  wsGatewayDomain:
    Type: AWS::ApiGatewayV2::DomainName
    Properties:
      DomainName: !FindInMap [gateways, ws, domainName]
      DomainNameConfigurations:
        - CertificateArn: !Ref certificateArn
          CertificateName: !Sub ${prefix}-certificate
          SecurityPolicy: TLS_1_2
  
  wsApiGatewayLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !FindInMap [gateways, ws, logGroupName]

  wsApiGateway:
    Type: AWS::ApiGatewayV2::Api
    Properties:
      Name: !Sub ${prefix}-ws-gateway
      Description: Api Gateway for Websockets
      ProtocolType: WEBSOCKET
      RouteSelectionExpression: $request.body.action
      DisableExecuteApiEndpoint: true

  wsConnectRoute:
    Type: AWS::ApiGatewayV2::Route
    Properties:
      ApiId: !Ref wsApiGateway
      RouteKey: $connect
      AuthorizationType: NONE
      OperationName: ConnectRoute
      # RouteResponseSelectionExpression: $default
      Target: !Join
        - /
        - - integrations
          - !Ref wsConnectIntegration

  wsConnectIntegration:
    Type: AWS::ApiGatewayV2::Integration
    Properties:
      ApiId: !Ref wsApiGateway
      Description: Websocket $connect integration
      IntegrationType: HTTP_PROXY
      IntegrationMethod: ANY
      IntegrationUri: https://api.mycompany.io/MyCompany.MyService/ws/connect
      RequestParameters:
        "integration.request.header.domainName": "context.domainName"
        "integration.request.header.stage": "context.stage"
        "integration.request.header.connectionId": "context.connectionId"
      PayloadFormatVersion: 1.0

  # wsConnectRouteResponse:
  #   Type: AWS::ApiGatewayV2::RouteResponse
  #   Properties:
  #     ApiId: !Ref wsApiGateway
  #     RouteId: !Ref wsConnectRoute
  #     RouteResponseKey: $default

  wsSendRoute:
    Type: AWS::ApiGatewayV2::Route
    Properties:
      ApiId: !Ref wsApiGateway
      RouteKey: send
      AuthorizationType: NONE
      OperationName: SendRoute
      RouteResponseSelectionExpression: $default
      Target: !Join
        - /
        - - integrations
          - !Ref wsSendIntegration

  wsSendIntegration:
    Type: AWS::ApiGatewayV2::Integration
    Properties:
      ApiId: !Ref wsApiGateway
      Description: Websocket send integration
      IntegrationType: HTTP_PROXY
      IntegrationMethod: ANY
      IntegrationUri: https://api.mycompany.io/MyCompany.MyService/ws/send
      RequestParameters:
        "integration.request.header.domainName": "context.domainName"
        "integration.request.header.stage": "context.stage"
        "integration.request.header.connectionId": "context.connectionId"
      PayloadFormatVersion: 1.0

  wsSendRouteResponse:
    Type: AWS::ApiGatewayV2::RouteResponse
    Properties:
      ApiId: !Ref wsApiGateway
      RouteId: !Ref wsSendRoute
      RouteResponseKey: $default

  wsDisconnectRoute:
    Type: AWS::ApiGatewayV2::Route
    Properties:
      ApiId: !Ref wsApiGateway
      RouteKey: $disconnect
      AuthorizationType: NONE
      OperationName: DisconnectRoute
      RouteResponseSelectionExpression: $default
      Target: !Join
        - /
        - - integrations
          - !Ref wsDisconnectIntegration

  wsDisconnectIntegration:
    Type: AWS::ApiGatewayV2::Integration
    Properties:
      ApiId: !Ref wsApiGateway
      Description: Websocket $disconnect integration
      IntegrationType: HTTP_PROXY
      IntegrationMethod: ANY
      IntegrationUri: https://api.mycompany.io/MyCompany.MyService/ws/disconnect
      RequestParameters:
        "integration.request.header.domainName": "context.domainName"
        "integration.request.header.stage": "context.stage"
        "integration.request.header.connectionId": "context.connectionId"
      PayloadFormatVersion: 1.0

  wsDisconnectRouteResponse:
    Type: AWS::ApiGatewayV2::RouteResponse
    Properties:
      ApiId: !Ref wsApiGateway
      RouteId: !Ref wsDisconnectRoute
      RouteResponseKey: $default

  wsApiStage:
    Type: AWS::ApiGatewayV2::Stage
    DependsOn:
      - wsConnectRoute
      - wsSendRoute
      - wsDisconnectRoute
    Properties:
      StageName: production
      Description: Autodeploy in production
      AutoDeploy: true
      ApiId: !Ref wsApiGateway
      AccessLogSettings:
        DestinationArn: !GetAtt wsApiGatewayLogGroup.Arn
        Format: '{"requestTime":"$context.requestTime","requestId":"$context.requestId","httpMethod":"$context.httpMethod","path":"$context.path","routeKey":"$context.routeKey","status":$context.status,"responseLatency":$context.responseLatency}'
  
  wsApiMapping:
    Type: AWS::ApiGatewayV2::ApiMapping
    DependsOn: wsGatewayDomain
    Properties: 
      ApiId: !Ref wsApiGateway
      DomainName: !FindInMap [gateways, ws, domainName]
      Stage: !Ref wsApiStage
  
  wsAliasDns:
    Type: AWS::Route53::RecordSet
    Properties:
      Comment: DNS entry for public custom domain to alias Websocket URL
      HostedZoneId: !Ref hostedZoneId
      Name: !FindInMap [gateways, ws, domainName]
      Type: A
      AliasTarget:
        DNSName: !GetAtt wsGatewayDomain.RegionalDomainName
        HostedZoneId: !GetAtt wsGatewayDomain.RegionalHostedZoneId
        EvaluateTargetHealth: true

Outputs:
  apiUri:
    Description: Http Api URL
    Value: 
      !Sub 
        - https://${domainName}
        - domainName: !FindInMap [gateways, api, domainName]
  wsUri:
    Description: Websocket Api URL
    Value:
      !Sub 
        - wss://${domainName}/web/index.html
        - domainName: !FindInMap [gateways, ws, domainName]

为了以防万一,我的服务目前是一个 C# ASP.NET 应用程序,中间件只能执行此操作

public async Task Invoke(HttpContext context)
{
  var request = context.Request;
  if (request.Path.ToString().Contains("/ws/", StringComparison.InvariantCultureIgnoreCase))
  {
    var path = request.Path.ToString();
    _logger.LogDebug("Received websocket message on {path}");
    await context.Response.WriteAsync($"OK: {path}");
  }
  else
  {
    // Call the next delegate/middleware in the pipeline
    await _next(context);
    var response = context.Response;
  }

更新 1: 我试过在我的服务中没有中间件。只是一个普通的旧 GET 端点 https://api.mycompany.io/MyCompany.MyService/api/sample 调用时 returns 200 OK。它已经启动 运行ning,而且我可以从 POSTman 验证它工作正常。

但是仍然无法连接websocket。这些是痕迹。

(f0sygHFWDoEFuyQ=) Extended Request Id: f0sygHFWDoEFuyQ=
(f0sygHFWDoEFuyQ=) Verifying Usage Plan for request: f0sygHFWDoEFuyQ=. API Key:  API Stage: rui1tsyre5/production
(f0sygHFWDoEFuyQ=) API Key  authorized because route '$connect' does not require API Key. Request will not contribute to throttle or quota limits
(f0sygHFWDoEFuyQ=) Usage Plan check succeeded for API Key  and API Stage rui1tsyre5/production
(f0sygHFWDoEFuyQ=) Starting execution for request: f0sygHFWDoEFuyQ=
(f0sygHFWDoEFuyQ=) WebSocket Request Route: [$connect]
(f0sygHFWDoEFuyQ=) Client [UserAgent: null, SourceIp: 91.194.63.143] is connecting to WebSocket API [rui1tsyre5].
(f0sygHFWDoEFuyQ=) Endpoint request URI: https://api.mycompany.io/MyCompany.MyService/api/sample
(f0sygHFWDoEFuyQ=) Endpoint request headers: {Sec-WebSocket-Key=rzw3hU/DdJC5x7ujeuRgCQ==, x-amzn-apigateway-api-id=rui1tsyre5, stage=production, domainName=ws.mycompany.io, X-Forwarded-Proto=https, User-Agent=AmazonAPIGateway_rui1tsyre5, connectionId=f0sygdPvDoECHRg=, Sec-WebSocket-Version=13, X-Forwarded-For=91.194.63.143, X-Forwarded-Port=443, X-Amzn-Trace-Id=Root=1-60ab5b43-5653a1fe637f11634b1e16ce, Sec-WebSocket-Extensions=permessage-deflate; client_max_window_bits}
(f0sygHFWDoEFuyQ=) Endpoint request body after transformations: 
(f0sygHFWDoEFuyQ=) Sending request to https://api.mycompany.io/MyCompany.MyService/api/sample
(f0sygHFWDoEFuyQ=) Received response. Status: 400, Integration latency: 4 ms
(f0sygHFWDoEFuyQ=) Endpoint response headers: {Server=awselb/2.0, Date=Mon, 24 May 2021 07:52:35 GMT, Content-Type=application/json; charset=utf-8, Content-Length=11, Connection=keep-alive}
(f0sygHFWDoEFuyQ=) Endpoint response body before transformations: Bad Request
(f0sygHFWDoEFuyQ=) Client [UserAgent: null, SourceIp: 91.194.63.143] failed to connect to API [rui1tsyre5]. 

为什么集成返回 400 错误?如果我能看到有关错误的更多详细信息,那就太好了。

我解决了。

指向我的服务的“真实”http 集成失败的原因是某些 http headers。

我通过访问云监视日志并查看实际 http 集成调用中使用的所有 http header 重现了该问题,并且在 PostMan 中向我的服务发出了准确的请求那些headers。不出所料,我收到了 400 Bad Request。

(fr1iDG7CjoEFWTg=) Endpoint request headers: {Sec-WebSocket-Key=7kp7mwI9YH+WjoGeiGjMEg==, x-amzn-apigateway-api-id=i4vdq18wg9, stage=production, domainName=ws.mycompany.io, X-Forwarded-Proto=https, User-Agent=AmazonAPIGateway_i4vdq18wg9, connectionId=fr1iDchODoECFIQ=, Sec-WebSocket-Version=13, X-Forwarded-For=91.194.63.143, X-Forwarded-Port=443, X-Amzn-Trace-Id=Root=1-60a7cfa6-45d1a7387a071f9c6547dfad, Sec-WebSocket-Extensions=permessage-deflate; client_max_window_bits}

首先,没有Host http header,不知道为什么。此外,即使存在 host header.

Sec-WebSocket-Version=13 似乎也会导致 400 错误

有几种方法可以解决这个问题。在服务中,我可以在中间件中操纵 http headers 来容纳它,或者我可以调整 API 网关。我做了后者,并为主机 header 设置了一个静态值,并将 Sec-WebSocket-Version 设置为 0 (我不知道如何删除 header,所以我用虚拟值覆盖它)

在 cloudFormation 的集成中

RequestParameters:
  "integration.request.header.domainName": "context.domainName"
  "integration.request.header.stage": "context.stage"
  "integration.request.header.connectionId": "context.connectionId"
  "integration.request.header.host": "'api.mycompany.io'"
  "integration.request.header.Sec-WebSocket-Version": "'0'"