AWS ECS - 退出代码 137
AWS ECS - exit code 137
我正在尝试使用 AWS ECS 和 docker 部署我的应用程序。 运行使用 docker-compose up 在我的机器上本地设置环境。然而,一旦转换并移动到 ECS,我就会收到错误 137
我已经搜索了退出代码,它可能与 ram 相关,所以我启动了一个更大的容器和 8gb 的 ram,仍然遇到同样的问题。 (只有两张图运行ning)
我通过连接到容器和 运行ning "docker run -it "imagename" /bin/bash 手动 运行 图像,它们 运行 没有问题。
任何人都可以看到原因或我如何解决这个问题吗?
这是我的任务:
{
"requiresAttributes": [
{
"value": null,
"name": "com.amazonaws.ecs.capability.docker-remote-api.1.17",
"targetId": null,
"targetType": null
},
{
"value": null,
"name": "com.amazonaws.ecs.capability.ecr-auth",
"targetId": null,
"targetType": null
}
],
"taskDefinitionArn": "arn:aws:ecs:eu-west-1:****:task-definition/it-app:1",
"networkMode": "bridge",
"status": "ACTIVE",
"revision": 1,
"taskRoleArn": null,
"containerDefinitions": [
{
"volumesFrom": [],
"memory": 300,
"extraHosts": null,
"dnsServers": null,
"disableNetworking": null,
"dnsSearchDomains": null,
"portMappings": [],
"hostname": null,
"essential": true,
"entryPoint": null,
"mountPoints": [],
"name": "it-app",
"ulimits": null,
"dockerSecurityOptions": null,
"environment": [],
"links": null,
"workingDirectory": "/itapp/itapp",
"readonlyRootFilesystem": null,
"image": "****.dkr.ecr.eu-west-1.amazonaws.com/itapp",
"command": [
"bash",
"-c",
"python manage.py collectstatic --noinput && python manage.py makemigrations && python manage.py migrate && exec gunicorn itapp.wsgi -b 0.0.0.0:8000"
],
"user": null,
"dockerLabels": null,
"logConfiguration": null,
"cpu": 0,
"privileged": null,
"memoryReservation": null
},
{
"volumesFrom": [],
"memory": 300,
"extraHosts": null,
"dnsServers": null,
"disableNetworking": null,
"dnsSearchDomains": null,
"portMappings": [
{
"hostPort": 80,
"containerPort": 8000,
"protocol": "tcp"
}
],
"hostname": null,
"essential": true,
"entryPoint": null,
"mountPoints": [],
"name": "itapp-ngnix",
"ulimits": null,
"dockerSecurityOptions": null,
"environment": [],
"links": [
"it-app"
],
"workingDirectory": null,
"readonlyRootFilesystem": null,
"image": "****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest",
"command": null,
"user": null,
"dockerLabels": null,
"logConfiguration": null,
"cpu": 0,
"privileged": null,
"memoryReservation": null
}
],
"placementConstraints": [],
"volumes": [],
"family": "it-app"
}
编辑:
我想我发现了问题,但不确定为什么会这样?
service itapp-web was unable to place a task because no container instance met all of its requirements.
The closest matching container-instance 315a7d85-1e1a-4a66-9173-e1fc2c17fda2 is already using a port required by your task.
For more information, see the Troubleshooting section.
这里是容器日志:
2017-09-18T15:10:07Z [INFO] Waiting for any previous stops to complete module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (NONE->RUNNING),]" seqnum="109"
2017-09-18T15:10:07Z [INFO] Wait over; ready to move towards status: RUNNING module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (NONE->RUNNING),]"
2017-09-18T15:10:07Z [INFO] Pulling container it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (NONE->RUNNING) concurrently. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (NONE->RUNNING),]
2017-09-18T15:10:08Z [INFO] Updating container reference it-app in Image State - sha256:a5a8ed72293792e2c1a51f2083d61049e9cda26b5d39a78d14087b0a2ffda675
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:08Z [INFO] Finished pulling container ****.dkr.ecr.eu-west-1.amazonaws.com/itapp in 124.116556ms. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (PULLED->RUNNING),]
2017-09-18T15:10:08Z [INFO] Creating container module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (PULLED->RUNNING),]" container="it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (PULLED->RUNNING)"
2017-09-18T15:10:08Z [INFO] Created container name mapping for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (PULLED->RUNNING),] - it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (PULLED->RUNNING) -> ecs-it-app-1-it-app-d280ddb2fed09c640000
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:08Z [INFO] Created docker container for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (CREATED->RUNNING),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (CREATED->RUNNING) -> c2ce8f61094cd93865c4bce71e12d8264d45c91e8c8ad1436a614380541e7e64
2017-09-18T15:10:08Z [INFO] Starting container module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (CREATED->RUNNING),]" container="it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (CREATED->RUNNING)"
2017-09-18T15:10:08Z [INFO] Redundant container state change for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (CREATED->RUNNING),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (CREATED->RUNNING) to CREATED, but already CREATED
2017-09-18T15:10:08Z [INFO] TaskHandler, batching container event: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c it-app -> RUNNING, Known Sent: NONE
2017-09-18T15:10:08Z [INFO] Redundant container state change for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (RUNNING->RUNNING),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (RUNNING->RUNNING) to RUNNING, but already RUNNING
2017-09-18T15:10:08Z [INFO] Pulling container itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (NONE->RUNNING) concurrently. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (RUNNING->RUNNING),]
2017-09-18T15:10:08Z [INFO] Updating container reference itapp-ngnix in Image State - sha256:fb38044bb0d9b78d8da038b54d5afc1b3091b35d04524976b716500b0fcabb4f
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:08Z [INFO] Finished pulling container ****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest in 105.243653ms. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (PULLED->RUNNING),it-app (RUNNING->RUNNING),]
2017-09-18T15:10:07Z [INFO] Pulling container it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (NONE->RUNNING) concurrently. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (NONE->RUNNING),]
2017-09-18T15:10:08Z [INFO] Updating container reference it-app in Image State - sha256:a5a8ed72293792e2c1a51f2083d61049e9cda26b5d39a78d14087b0a2ffda675
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:08Z [INFO] Finished pulling container ****.dkr.ecr.eu-west-1.amazonaws.com/itapp in 124.116556ms. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (PULLED->RUNNING),]
2017-09-18T15:10:08Z [INFO] Creating container module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (PULLED->RUNNING),]" container="it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (PULLED->RUNNING)"
2017-09-18T15:10:08Z [INFO] Created container name mapping for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (PULLED->RUNNING),] - it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (PULLED->RUNNING) -> ecs-it-app-1-it-app-d280ddb2fed09c640000
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:08Z [INFO] Created docker container for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (CREATED->RUNNING),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (CREATED->RUNNING) -> c2ce8f61094cd93865c4bce71e12d8264d45c91e8c8ad1436a614380541e7e64
2017-09-18T15:10:08Z [INFO] Starting container module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (CREATED->RUNNING),]" container="it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (CREATED->RUNNING)"
2017-09-18T15:10:08Z [INFO] Redundant container state change for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (CREATED->RUNNING),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (CREATED->RUNNING) to CREATED, but already CREATED
2017-09-18T15:10:08Z [INFO] TaskHandler, batching container event: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c it-app -> RUNNING, Known Sent: NONE
2017-09-18T15:10:08Z [INFO] Redundant container state change for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (RUNNING->RUNNING),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (RUNNING->RUNNING) to RUNNING, but already RUNNING
2017-09-18T15:10:08Z [INFO] Pulling container itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (NONE->RUNNING) concurrently. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (RUNNING->RUNNING),]
2017-09-18T15:10:08Z [INFO] Updating container reference itapp-ngnix in Image State - sha256:fb38044bb0d9b78d8da038b54d5afc1b3091b35d04524976b716500b0fcabb4f
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:08Z [INFO] Finished pulling container ****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest in 105.243653ms. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (PULLED->RUNNING),it-app (RUNNING->RUNNING),]
2017-09-18T15:10:08Z [INFO] Creating container module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (PULLED->RUNNING),it-app (RUNNING->RUNNING),]" container="itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (PULLED->RUNNING)"
2017-09-18T15:10:08Z [WARN] Warning, link with no linkalias module="api client" linkName="it-app" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (PULLED->RUNNING),it-app (RUNNING->RUNNING),]" container="itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (PULLED->RUNNING)"
2017-09-18T15:10:08Z [INFO] Created container name mapping for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (PULLED->RUNNING),it-app (RUNNING->RUNNING),] - itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (PULLED->RUNNING) -> ecs-it-app-1-itapp-ngnix-c4e29c838bb19df6cb01
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:09Z [INFO] Created docker container for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (CREATED->RUNNING) Containers: [itapp-ngnix (CREATED->RUNNING),it-app (RUNNING->RUNNING),]: itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (CREATED->RUNNING) -> d06b6e4300665f0f8c18f0be04345e4797827736574d977b2a81cfb5d7eb471a
2017-09-18T15:10:09Z [INFO] Starting container module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (CREATED->RUNNING) Containers: [itapp-ngnix (CREATED->RUNNING),it-app (RUNNING->RUNNING),]" container="itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (CREATED->RUNNING)"
2017-09-18T15:10:09Z [INFO] Task change event module="TaskEngine" event="{TaskArn:arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c Status:RUNNING Reason: Containers:[] Task:it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (RUNNING->RUNNING) Containers: [itapp-ngnix (RUNNING->RUNNING),it-app (RUNNING->RUNNING),]}"
2017-09-18T15:10:09Z [INFO] TaskHandler, batching container event: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c itapp-ngnix -> RUNNING, Ports [{8000 80 0.0.0.0 0}], Known Sent: NONE
2017-09-18T15:10:09Z [INFO] TaskHandler, Adding event: TaskChange: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c -> RUNNING, Known Sent: NONE
2017-09-18T15:10:09Z [INFO] TaskHandler, Sending task change: TaskChange: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c -> RUNNING, Known Sent: NONE
2017-09-18T15:10:09Z [INFO] Redundant container state change for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (RUNNING->RUNNING) Containers: [itapp-ngnix (RUNNING->RUNNING),it-app (RUNNING->RUNNING),]: itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (RUNNING->RUNNING) to RUNNING, but already RUNNING
2017-09-18T15:10:09Z [INFO] Stopping container module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (RUNNING->STOPPED) Containers: [itapp-ngnix (STOPPED->STOPPED),it-app (RUNNING->STOPPED),]" container="it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (RUNNING->STOPPED)"
2017-09-18T15:10:09Z [INFO] Error retrieving stats for container d06b6e4300665f0f8c18f0be04345e4797827736574d977b2a81cfb5d7eb471a: context canceled
2017-09-18T15:10:09Z [INFO] Container d06b6e4300665f0f8c18f0be04345e4797827736574d977b2a81cfb5d7eb471a is terminal, stopping stats collection
2017-09-18T15:10:09Z [INFO] TaskHandler, batching container event: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c itapp-ngnix -> STOPPED, Exit 1, , Known Sent: NONE
2017-09-18T15:10:18Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:40Z [WARN] Error converting stats for container c2ce8f61094cd93865c4bce71e12d8264d45c91e8c8ad1436a614380541e7e64: Invalid container statistics reported
2017-09-18T15:10:40Z [INFO] Task change event module="TaskEngine" event="{TaskArn:arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c Status:STOPPED Reason: Containers:[] Task:it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (STOPPED->STOPPED) Containers: [itapp-ngnix (STOPPED->STOPPED),it-app (STOPPED->STOPPED),]}"
2017-09-18T15:10:40Z [INFO] Error retrieving stats for container c2ce8f61094cd93865c4bce71e12d8264d45c91e8c8ad1436a614380541e7e64: context canceled
2017-09-18T15:10:40Z [INFO] TaskHandler, batching container event: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c it-app -> STOPPED, Exit 137, , Known Sent: NONE
2017-09-18T15:10:40Z [INFO] TaskHandler, Adding event: TaskChange: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c -> STOPPED, Known Sent: RUNNING
2017-09-18T15:10:40Z [INFO] TaskHandler, Sending task change: TaskChange: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c -> STOPPED, Known Sent: RUNNING
2017-09-18T15:10:40Z [INFO] Container c2ce8f61094cd93865c4bce71e12d8264d45c91e8c8ad1436a614380541e7e64 is terminal, stopping stats collection
2017-09-18T15:10:40Z [INFO] Redundant container state change for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (STOPPED->STOPPED) Containers: [itapp-ngnix (STOPPED->STOPPED),it-app (STOPPED->STOPPED),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (STOPPED->STOPPED) - Exit: 137 to STOPPED, but already STOPPED
2017-09-18T15:10:40Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:40Z [INFO] Redundant container state change for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (STOPPED->STOPPED) Containers: [itapp-ngnix (STOPPED->STOPPED),it-app (STOPPED->STOPPED),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (STOPPED->STOPPED) - Exit: 137 to STOPPED, but already STOPPED
2017-09-18T15:10:50Z [INFO] Saving state! module="statemanager"
在Linux中,有多个exit codes with Special Meanings,这里要注意的是128+n部分,这是一个进程的Kill级别。在这种情况下,137 = 128 + 9,所以这个进程被最高级别杀死。
这通常发生在 ECS 中,当 ECS 向进程发送 STOP,但它在 30 秒内没有退出。
我要检查的此类问题的两个常见原因:
- 在您的容器定义中,您有
"memory": 300
。这是对特定 Docker 任务可以使用的内存量的硬性限制。如果达到该限制,它将被终止(我不知道退出代码是什么,但 137 可能是合理的)。根据您的示例,您可以尝试 运行 类似于 docker run -m 300M -it "imagename" /bin/bash
的内容。如果这达到了内存限制,它也应该被终止。
- 如果您的容器连接到负载均衡器并启用了健康检查,请确保您的应用程序在健康检查间隔内响应健康检查。如果没有,则对容器进行计数,在配置的健康检查失败次数后,容器将被认为是不健康的,将创建一个新任务并销毁旧任务。
这里需要注意一点,ECS在任务定义中支持memory
和memoryReservation
,只需要设置其中一个即可。 memory
如上所述是一个硬性限制,任何被击中的东西都会被摧毁。 memoryReservation
是软限制,您的容器可以超过该限制,最高可达 ECS 实例的总内存。您还可以将两者结合起来,在这种情况下,memoryReservation
用于确定实例上可用于额外任务的内存量,而 memory
用于在超过 [=35= 时终止任务]结束运行提供的号码。
我最终删除了我的服务并重新开始,现在是 运行
在我的例子中,是因为任务定义中定义的docker自我状态检查失败。删除检查或使其正确执行。
我遇到了类似的问题,我的容器被杀死,退出代码 (137) 完全相同。
事实证明,由于我的应用程序负载均衡器中配置不正确的健康检查,它被杀死了。
我在通过 AWS CDK 使用负载均衡器设置我的容器时也遇到了这个问题。
我正在使用 applicationLoadBalancedFargateService
结构。
事实证明,有一个选项可以使用目标组设置健康检查,您可以设置自定义端口、路径等。我还增加了时间,因为默认时间设置为 60 秒,它也会抛出此错误。
alb: applicationLoadBalancedFargateService = .... (my service)
alb.targetGroup.configureHealthCheck({
path: '/health-check',
port: '80',
healthyHttpCodes: '200,304', // both success and non modified
enabled: true,
healthyThresholdCount: 5,
interval: Duration.seconds(120),
timeout: Duration.seconds(60)
})
尝试通过增加任务数来更新ECS服务。对我来说很管用。
我正在尝试使用 AWS ECS 和 docker 部署我的应用程序。 运行使用 docker-compose up 在我的机器上本地设置环境。然而,一旦转换并移动到 ECS,我就会收到错误 137
我已经搜索了退出代码,它可能与 ram 相关,所以我启动了一个更大的容器和 8gb 的 ram,仍然遇到同样的问题。 (只有两张图运行ning)
我通过连接到容器和 运行ning "docker run -it "imagename" /bin/bash 手动 运行 图像,它们 运行 没有问题。
任何人都可以看到原因或我如何解决这个问题吗?
这是我的任务:
{
"requiresAttributes": [
{
"value": null,
"name": "com.amazonaws.ecs.capability.docker-remote-api.1.17",
"targetId": null,
"targetType": null
},
{
"value": null,
"name": "com.amazonaws.ecs.capability.ecr-auth",
"targetId": null,
"targetType": null
}
],
"taskDefinitionArn": "arn:aws:ecs:eu-west-1:****:task-definition/it-app:1",
"networkMode": "bridge",
"status": "ACTIVE",
"revision": 1,
"taskRoleArn": null,
"containerDefinitions": [
{
"volumesFrom": [],
"memory": 300,
"extraHosts": null,
"dnsServers": null,
"disableNetworking": null,
"dnsSearchDomains": null,
"portMappings": [],
"hostname": null,
"essential": true,
"entryPoint": null,
"mountPoints": [],
"name": "it-app",
"ulimits": null,
"dockerSecurityOptions": null,
"environment": [],
"links": null,
"workingDirectory": "/itapp/itapp",
"readonlyRootFilesystem": null,
"image": "****.dkr.ecr.eu-west-1.amazonaws.com/itapp",
"command": [
"bash",
"-c",
"python manage.py collectstatic --noinput && python manage.py makemigrations && python manage.py migrate && exec gunicorn itapp.wsgi -b 0.0.0.0:8000"
],
"user": null,
"dockerLabels": null,
"logConfiguration": null,
"cpu": 0,
"privileged": null,
"memoryReservation": null
},
{
"volumesFrom": [],
"memory": 300,
"extraHosts": null,
"dnsServers": null,
"disableNetworking": null,
"dnsSearchDomains": null,
"portMappings": [
{
"hostPort": 80,
"containerPort": 8000,
"protocol": "tcp"
}
],
"hostname": null,
"essential": true,
"entryPoint": null,
"mountPoints": [],
"name": "itapp-ngnix",
"ulimits": null,
"dockerSecurityOptions": null,
"environment": [],
"links": [
"it-app"
],
"workingDirectory": null,
"readonlyRootFilesystem": null,
"image": "****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest",
"command": null,
"user": null,
"dockerLabels": null,
"logConfiguration": null,
"cpu": 0,
"privileged": null,
"memoryReservation": null
}
],
"placementConstraints": [],
"volumes": [],
"family": "it-app"
}
编辑:
我想我发现了问题,但不确定为什么会这样?
service itapp-web was unable to place a task because no container instance met all of its requirements.
The closest matching container-instance 315a7d85-1e1a-4a66-9173-e1fc2c17fda2 is already using a port required by your task.
For more information, see the Troubleshooting section.
这里是容器日志:
2017-09-18T15:10:07Z [INFO] Waiting for any previous stops to complete module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (NONE->RUNNING),]" seqnum="109"
2017-09-18T15:10:07Z [INFO] Wait over; ready to move towards status: RUNNING module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (NONE->RUNNING),]"
2017-09-18T15:10:07Z [INFO] Pulling container it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (NONE->RUNNING) concurrently. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (NONE->RUNNING),]
2017-09-18T15:10:08Z [INFO] Updating container reference it-app in Image State - sha256:a5a8ed72293792e2c1a51f2083d61049e9cda26b5d39a78d14087b0a2ffda675
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:08Z [INFO] Finished pulling container ****.dkr.ecr.eu-west-1.amazonaws.com/itapp in 124.116556ms. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (PULLED->RUNNING),]
2017-09-18T15:10:08Z [INFO] Creating container module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (PULLED->RUNNING),]" container="it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (PULLED->RUNNING)"
2017-09-18T15:10:08Z [INFO] Created container name mapping for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (PULLED->RUNNING),] - it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (PULLED->RUNNING) -> ecs-it-app-1-it-app-d280ddb2fed09c640000
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:08Z [INFO] Created docker container for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (CREATED->RUNNING),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (CREATED->RUNNING) -> c2ce8f61094cd93865c4bce71e12d8264d45c91e8c8ad1436a614380541e7e64
2017-09-18T15:10:08Z [INFO] Starting container module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (CREATED->RUNNING),]" container="it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (CREATED->RUNNING)"
2017-09-18T15:10:08Z [INFO] Redundant container state change for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (CREATED->RUNNING),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (CREATED->RUNNING) to CREATED, but already CREATED
2017-09-18T15:10:08Z [INFO] TaskHandler, batching container event: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c it-app -> RUNNING, Known Sent: NONE
2017-09-18T15:10:08Z [INFO] Redundant container state change for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (RUNNING->RUNNING),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (RUNNING->RUNNING) to RUNNING, but already RUNNING
2017-09-18T15:10:08Z [INFO] Pulling container itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (NONE->RUNNING) concurrently. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (RUNNING->RUNNING),]
2017-09-18T15:10:08Z [INFO] Updating container reference itapp-ngnix in Image State - sha256:fb38044bb0d9b78d8da038b54d5afc1b3091b35d04524976b716500b0fcabb4f
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:08Z [INFO] Finished pulling container ****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest in 105.243653ms. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (PULLED->RUNNING),it-app (RUNNING->RUNNING),]
2017-09-18T15:10:07Z [INFO] Pulling container it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (NONE->RUNNING) concurrently. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (NONE->RUNNING),]
2017-09-18T15:10:08Z [INFO] Updating container reference it-app in Image State - sha256:a5a8ed72293792e2c1a51f2083d61049e9cda26b5d39a78d14087b0a2ffda675
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:08Z [INFO] Finished pulling container ****.dkr.ecr.eu-west-1.amazonaws.com/itapp in 124.116556ms. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (PULLED->RUNNING),]
2017-09-18T15:10:08Z [INFO] Creating container module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (PULLED->RUNNING),]" container="it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (PULLED->RUNNING)"
2017-09-18T15:10:08Z [INFO] Created container name mapping for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (PULLED->RUNNING),] - it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (PULLED->RUNNING) -> ecs-it-app-1-it-app-d280ddb2fed09c640000
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:08Z [INFO] Created docker container for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (CREATED->RUNNING),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (CREATED->RUNNING) -> c2ce8f61094cd93865c4bce71e12d8264d45c91e8c8ad1436a614380541e7e64
2017-09-18T15:10:08Z [INFO] Starting container module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (CREATED->RUNNING),]" container="it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (CREATED->RUNNING)"
2017-09-18T15:10:08Z [INFO] Redundant container state change for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (CREATED->RUNNING),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (CREATED->RUNNING) to CREATED, but already CREATED
2017-09-18T15:10:08Z [INFO] TaskHandler, batching container event: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c it-app -> RUNNING, Known Sent: NONE
2017-09-18T15:10:08Z [INFO] Redundant container state change for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (RUNNING->RUNNING),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (RUNNING->RUNNING) to RUNNING, but already RUNNING
2017-09-18T15:10:08Z [INFO] Pulling container itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (NONE->RUNNING) concurrently. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (NONE->RUNNING),it-app (RUNNING->RUNNING),]
2017-09-18T15:10:08Z [INFO] Updating container reference itapp-ngnix in Image State - sha256:fb38044bb0d9b78d8da038b54d5afc1b3091b35d04524976b716500b0fcabb4f
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:08Z [INFO] Finished pulling container ****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest in 105.243653ms. Task: it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (PULLED->RUNNING),it-app (RUNNING->RUNNING),]
2017-09-18T15:10:08Z [INFO] Creating container module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (PULLED->RUNNING),it-app (RUNNING->RUNNING),]" container="itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (PULLED->RUNNING)"
2017-09-18T15:10:08Z [WARN] Warning, link with no linkalias module="api client" linkName="it-app" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (PULLED->RUNNING),it-app (RUNNING->RUNNING),]" container="itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (PULLED->RUNNING)"
2017-09-18T15:10:08Z [INFO] Created container name mapping for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (NONE->RUNNING) Containers: [itapp-ngnix (PULLED->RUNNING),it-app (RUNNING->RUNNING),] - itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (PULLED->RUNNING) -> ecs-it-app-1-itapp-ngnix-c4e29c838bb19df6cb01
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:08Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:09Z [INFO] Created docker container for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (CREATED->RUNNING) Containers: [itapp-ngnix (CREATED->RUNNING),it-app (RUNNING->RUNNING),]: itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (CREATED->RUNNING) -> d06b6e4300665f0f8c18f0be04345e4797827736574d977b2a81cfb5d7eb471a
2017-09-18T15:10:09Z [INFO] Starting container module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (CREATED->RUNNING) Containers: [itapp-ngnix (CREATED->RUNNING),it-app (RUNNING->RUNNING),]" container="itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (CREATED->RUNNING)"
2017-09-18T15:10:09Z [INFO] Task change event module="TaskEngine" event="{TaskArn:arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c Status:RUNNING Reason: Containers:[] Task:it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (RUNNING->RUNNING) Containers: [itapp-ngnix (RUNNING->RUNNING),it-app (RUNNING->RUNNING),]}"
2017-09-18T15:10:09Z [INFO] TaskHandler, batching container event: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c itapp-ngnix -> RUNNING, Ports [{8000 80 0.0.0.0 0}], Known Sent: NONE
2017-09-18T15:10:09Z [INFO] TaskHandler, Adding event: TaskChange: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c -> RUNNING, Known Sent: NONE
2017-09-18T15:10:09Z [INFO] TaskHandler, Sending task change: TaskChange: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c -> RUNNING, Known Sent: NONE
2017-09-18T15:10:09Z [INFO] Redundant container state change for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (RUNNING->RUNNING) Containers: [itapp-ngnix (RUNNING->RUNNING),it-app (RUNNING->RUNNING),]: itapp-ngnix(****.dkr.ecr.eu-west-1.amazonaws.com/itapp-ngnix:latest) (RUNNING->RUNNING) to RUNNING, but already RUNNING
2017-09-18T15:10:09Z [INFO] Stopping container module="TaskEngine" task="it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (RUNNING->STOPPED) Containers: [itapp-ngnix (STOPPED->STOPPED),it-app (RUNNING->STOPPED),]" container="it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (RUNNING->STOPPED)"
2017-09-18T15:10:09Z [INFO] Error retrieving stats for container d06b6e4300665f0f8c18f0be04345e4797827736574d977b2a81cfb5d7eb471a: context canceled
2017-09-18T15:10:09Z [INFO] Container d06b6e4300665f0f8c18f0be04345e4797827736574d977b2a81cfb5d7eb471a is terminal, stopping stats collection
2017-09-18T15:10:09Z [INFO] TaskHandler, batching container event: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c itapp-ngnix -> STOPPED, Exit 1, , Known Sent: NONE
2017-09-18T15:10:18Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:40Z [WARN] Error converting stats for container c2ce8f61094cd93865c4bce71e12d8264d45c91e8c8ad1436a614380541e7e64: Invalid container statistics reported
2017-09-18T15:10:40Z [INFO] Task change event module="TaskEngine" event="{TaskArn:arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c Status:STOPPED Reason: Containers:[] Task:it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (STOPPED->STOPPED) Containers: [itapp-ngnix (STOPPED->STOPPED),it-app (STOPPED->STOPPED),]}"
2017-09-18T15:10:40Z [INFO] Error retrieving stats for container c2ce8f61094cd93865c4bce71e12d8264d45c91e8c8ad1436a614380541e7e64: context canceled
2017-09-18T15:10:40Z [INFO] TaskHandler, batching container event: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c it-app -> STOPPED, Exit 137, , Known Sent: NONE
2017-09-18T15:10:40Z [INFO] TaskHandler, Adding event: TaskChange: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c -> STOPPED, Known Sent: RUNNING
2017-09-18T15:10:40Z [INFO] TaskHandler, Sending task change: TaskChange: arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c -> STOPPED, Known Sent: RUNNING
2017-09-18T15:10:40Z [INFO] Container c2ce8f61094cd93865c4bce71e12d8264d45c91e8c8ad1436a614380541e7e64 is terminal, stopping stats collection
2017-09-18T15:10:40Z [INFO] Redundant container state change for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (STOPPED->STOPPED) Containers: [itapp-ngnix (STOPPED->STOPPED),it-app (STOPPED->STOPPED),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (STOPPED->STOPPED) - Exit: 137 to STOPPED, but already STOPPED
2017-09-18T15:10:40Z [INFO] Saving state! module="statemanager"
2017-09-18T15:10:40Z [INFO] Redundant container state change for task it-app:1 arn:aws:ecs:eu-west-1:****:task/2d1f357a-1350-40bb-801c-d3d552225d3c, Status: (STOPPED->STOPPED) Containers: [itapp-ngnix (STOPPED->STOPPED),it-app (STOPPED->STOPPED),]: it-app(****.dkr.ecr.eu-west-1.amazonaws.com/itapp) (STOPPED->STOPPED) - Exit: 137 to STOPPED, but already STOPPED
2017-09-18T15:10:50Z [INFO] Saving state! module="statemanager"
在Linux中,有多个exit codes with Special Meanings,这里要注意的是128+n部分,这是一个进程的Kill级别。在这种情况下,137 = 128 + 9,所以这个进程被最高级别杀死。
这通常发生在 ECS 中,当 ECS 向进程发送 STOP,但它在 30 秒内没有退出。
我要检查的此类问题的两个常见原因:
- 在您的容器定义中,您有
"memory": 300
。这是对特定 Docker 任务可以使用的内存量的硬性限制。如果达到该限制,它将被终止(我不知道退出代码是什么,但 137 可能是合理的)。根据您的示例,您可以尝试 运行 类似于docker run -m 300M -it "imagename" /bin/bash
的内容。如果这达到了内存限制,它也应该被终止。 - 如果您的容器连接到负载均衡器并启用了健康检查,请确保您的应用程序在健康检查间隔内响应健康检查。如果没有,则对容器进行计数,在配置的健康检查失败次数后,容器将被认为是不健康的,将创建一个新任务并销毁旧任务。
这里需要注意一点,ECS在任务定义中支持memory
和memoryReservation
,只需要设置其中一个即可。 memory
如上所述是一个硬性限制,任何被击中的东西都会被摧毁。 memoryReservation
是软限制,您的容器可以超过该限制,最高可达 ECS 实例的总内存。您还可以将两者结合起来,在这种情况下,memoryReservation
用于确定实例上可用于额外任务的内存量,而 memory
用于在超过 [=35= 时终止任务]结束运行提供的号码。
我最终删除了我的服务并重新开始,现在是 运行
在我的例子中,是因为任务定义中定义的docker自我状态检查失败。删除检查或使其正确执行。
我遇到了类似的问题,我的容器被杀死,退出代码 (137) 完全相同。
事实证明,由于我的应用程序负载均衡器中配置不正确的健康检查,它被杀死了。
我在通过 AWS CDK 使用负载均衡器设置我的容器时也遇到了这个问题。
我正在使用 applicationLoadBalancedFargateService
结构。
事实证明,有一个选项可以使用目标组设置健康检查,您可以设置自定义端口、路径等。我还增加了时间,因为默认时间设置为 60 秒,它也会抛出此错误。
alb: applicationLoadBalancedFargateService = .... (my service)
alb.targetGroup.configureHealthCheck({
path: '/health-check',
port: '80',
healthyHttpCodes: '200,304', // both success and non modified
enabled: true,
healthyThresholdCount: 5,
interval: Duration.seconds(120),
timeout: Duration.seconds(60)
})
尝试通过增加任务数来更新ECS服务。对我来说很管用。