X 秒后重试：docker 私有注册推送错误
Retrying in X seconds: docker private registery push error
我已经设置了我的 docker 私有注册表，我正在尝试 push 一个标记图像。当我 运行 命令波纹管我得到无限错误 Retrying in X seconds.
docker push registry.domain/hellow-world:latest
我尝试查看容器日志中发生了什么，我看到以下消息：
ERRO[14335] response completed with error err.code=blob unknown err.detail=sha256:080aeede8114545b32a15a1cac9b13fa4b9754104916f72ce43ccd1f8d271744 err.message=blob unknown to registry go.version=go1.7.6 http.request.host=app-registry http.request.id=3841db89-4e17-4aa6-893c-9d66e270cb58 http.request.method=HEAD http.request.remoteaddr=10.42.122.106:37796 http.request.uri=/v2/speed-nginx/blobs/sha256:080aeede8114545b32a15a1cac9b13fa4b9754104916f72ce43ccd1f8d271744 http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.contenttype=application/json; charset=utf-8 http.response.duration=2.788485ms http.response.status=404 http.response.written=157 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d vars.digest=sha256:080aeede8114545b32a15a1cac9b13fa4b9754104916f72ce43ccd1f8d271744 vars.name=speed-nginx version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:48 +0000] "HEAD /v2/speed-nginx/blobs/sha256:080aeede8114545b32a15a1cac9b13fa4b9754104916f72ce43ccd1f8d271744 HTTP/1.0" 404 157 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
ERRO[14335] response completed with error err.code=blob unknown err.detail=sha256:2a72cbf407d67c7a7a76dd48e432091678e297140dce050ad5eccad918a9f8d6 err.message=blob unknown to registry go.version=go1.7.6 http.request.host=app-registry http.request.id=40832877-6978-4289-8ca1-b5f8031ba1e1 http.request.method=HEAD http.request.remoteaddr=10.42.122.106:37794 http.request.uri=/v2/speed-nginx/blobs/sha256:2a72cbf407d67c7a7a76dd48e432091678e297140dce050ad5eccad918a9f8d6 http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.contenttype=application/json; charset=utf-8 http.response.duration=3.010708ms http.response.status=404 http.response.written=157 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d vars.digest=sha256:2a72cbf407d67c7a7a76dd48e432091678e297140dce050ad5eccad918a9f8d6 vars.name=speed-nginx version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:48 +0000] "HEAD /v2/speed-nginx/blobs/sha256:2a72cbf407d67c7a7a76dd48e432091678e297140dce050ad5eccad918a9f8d6 HTTP/1.0" 404 157 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
ERRO[14335] response completed with error err.code=blob unknown err.detail=sha256:fefa2faca81fe5f2d465e38741227d44d07c0fedcaf5e92ced037f07afd291ff err.message=blob unknown to registry go.version=go1.7.6 http.request.host=app-registry http.request.id=3cc3b8c8-b886-4f39-be19-ed9f7bc330ef http.request.method=HEAD http.request.remoteaddr=10.42.122.106:37798 http.request.uri=/v2/speed-nginx/blobs/sha256:fefa2faca81fe5f2d465e38741227d44d07c0fedcaf5e92ced037f07afd291ff http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.contenttype=application/json; charset=utf-8 http.response.duration=2.764234ms http.response.status=404 http.response.written=157 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d vars.digest=sha256:fefa2faca81fe5f2d465e38741227d44d07c0fedcaf5e92ced037f07afd291ff vars.name=speed-nginx version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:48 +0000] "HEAD /v2/speed-nginx/blobs/sha256:fefa2faca81fe5f2d465e38741227d44d07c0fedcaf5e92ced037f07afd291ff HTTP/1.0" 404 157 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
INFO[14335] response completed go.version=go1.7.6 http.request.host=app-registry http.request.id=10435650-6769-4fb3-a5c3-6d2becb24658 http.request.method=POST http.request.remoteaddr=10.42.122.106:37812 http.request.uri=/v2/speed-nginx/blobs/uploads/ http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.duration=41.743786ms http.response.status=202 http.response.written=0 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:48 +0000] "POST /v2/speed-nginx/blobs/uploads/ HTTP/1.0" 202 0 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
INFO[14335] response completed go.version=go1.7.6 http.request.host=app-registry http.request.id=6a0428c8-ee18-4549-8064-80a56c04b6dd http.request.method=POST http.request.remoteaddr=10.42.122.106:37808 http.request.uri=/v2/speed-nginx/blobs/uploads/ http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.duration=67.665191ms http.response.status=202 http.response.written=0 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:48 +0000] "POST /v2/speed-nginx/blobs/uploads/ HTTP/1.0" 202 0 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
INFO[14335] response completed go.version=go1.7.6 http.request.host=app-registry http.request.id=a61801c7-c215-44b4-88c3-a5fd59469c9f http.request.method=POST http.request.remoteaddr=10.42.122.106:37810 http.request.uri=/v2/speed-nginx/blobs/uploads/ http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.duration=46.806906ms http.response.status=202 http.response.written=0 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:48 +0000] "POST /v2/speed-nginx/blobs/uploads/ HTTP/1.0" 202 0 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
INFO[14341] response completed go.version=go1.7.6 http.request.host=app-registry http.request.id=1471ae4f-b5d7-4c05-90cb-7c125837fbfe http.request.method=POST http.request.remoteaddr=10.42.122.106:37868 http.request.uri=/v2/speed-nginx/blobs/uploads/ http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.duration=73.810855ms http.response.status=202 http.response.written=0 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:54 +0000] "POST /v2/speed-nginx/blobs/uploads/ HTTP/1.0" 202 0 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
INFO[14341] response completed go.version=go1.7.6 http.request.host=app-registry http.request.id=86e88418-a141-4000-a3fd-3c9c6d4664d2 http.request.method=POST http.request.remoteaddr=10.42.122.106:37866 http.request.uri=/v2/speed-nginx/blobs/uploads/ http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.duration=61.720106ms http.response.status=202 http.response.written=0 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:54 +0000] "POST /v2/speed-nginx/blobs/uploads/ HTTP/1.0" 202 0 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
INFO[14341] response completed go.version=go1.7.6 http.request.host=app-registry http.request.id=0097346b-38c4-4b94-8e20-6e64dfd65c7a http.request.method=POST http.request.remoteaddr=10.42.122.106:37864 http.request.uri=/v2/speed-nginx/blobs/uploads/ http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.duration=176.55553ms http.response.status=202 http.response.written=0 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:54 +0000] "POST /v2/speed-nginx/blobs/uploads/ HTTP/1.0" 202 0 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
我的 docker 引擎版本是 17.12.0-ce
并拥有 nginx + 私有注册表。
问题出在 nginx.config. 在 proxy_pass 中使用上游导致 DNS lookup error。我替换了上游值并开始工作。好像是个bug！
docker服务状态：
Mar 30 20:47:03 docker-s-2vcpu-4gb-lon1-01 dockerd[1369]: time="2018-03-30T20:47:03.885277507Z" level=error msg="Upload failed, retrying: dial tcp: lookup app-registry on x.x.x.x:53: no such host"
我已经设置了我的 docker 私有注册表，我正在尝试 push 一个标记图像。当我 运行 命令波纹管我得到无限错误 Retrying in X seconds.
docker push registry.domain/hellow-world:latest
我尝试查看容器日志中发生了什么，我看到以下消息：
ERRO[14335] response completed with error err.code=blob unknown err.detail=sha256:080aeede8114545b32a15a1cac9b13fa4b9754104916f72ce43ccd1f8d271744 err.message=blob unknown to registry go.version=go1.7.6 http.request.host=app-registry http.request.id=3841db89-4e17-4aa6-893c-9d66e270cb58 http.request.method=HEAD http.request.remoteaddr=10.42.122.106:37796 http.request.uri=/v2/speed-nginx/blobs/sha256:080aeede8114545b32a15a1cac9b13fa4b9754104916f72ce43ccd1f8d271744 http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.contenttype=application/json; charset=utf-8 http.response.duration=2.788485ms http.response.status=404 http.response.written=157 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d vars.digest=sha256:080aeede8114545b32a15a1cac9b13fa4b9754104916f72ce43ccd1f8d271744 vars.name=speed-nginx version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:48 +0000] "HEAD /v2/speed-nginx/blobs/sha256:080aeede8114545b32a15a1cac9b13fa4b9754104916f72ce43ccd1f8d271744 HTTP/1.0" 404 157 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
ERRO[14335] response completed with error err.code=blob unknown err.detail=sha256:2a72cbf407d67c7a7a76dd48e432091678e297140dce050ad5eccad918a9f8d6 err.message=blob unknown to registry go.version=go1.7.6 http.request.host=app-registry http.request.id=40832877-6978-4289-8ca1-b5f8031ba1e1 http.request.method=HEAD http.request.remoteaddr=10.42.122.106:37794 http.request.uri=/v2/speed-nginx/blobs/sha256:2a72cbf407d67c7a7a76dd48e432091678e297140dce050ad5eccad918a9f8d6 http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.contenttype=application/json; charset=utf-8 http.response.duration=3.010708ms http.response.status=404 http.response.written=157 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d vars.digest=sha256:2a72cbf407d67c7a7a76dd48e432091678e297140dce050ad5eccad918a9f8d6 vars.name=speed-nginx version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:48 +0000] "HEAD /v2/speed-nginx/blobs/sha256:2a72cbf407d67c7a7a76dd48e432091678e297140dce050ad5eccad918a9f8d6 HTTP/1.0" 404 157 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
ERRO[14335] response completed with error err.code=blob unknown err.detail=sha256:fefa2faca81fe5f2d465e38741227d44d07c0fedcaf5e92ced037f07afd291ff err.message=blob unknown to registry go.version=go1.7.6 http.request.host=app-registry http.request.id=3cc3b8c8-b886-4f39-be19-ed9f7bc330ef http.request.method=HEAD http.request.remoteaddr=10.42.122.106:37798 http.request.uri=/v2/speed-nginx/blobs/sha256:fefa2faca81fe5f2d465e38741227d44d07c0fedcaf5e92ced037f07afd291ff http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.contenttype=application/json; charset=utf-8 http.response.duration=2.764234ms http.response.status=404 http.response.written=157 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d vars.digest=sha256:fefa2faca81fe5f2d465e38741227d44d07c0fedcaf5e92ced037f07afd291ff vars.name=speed-nginx version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:48 +0000] "HEAD /v2/speed-nginx/blobs/sha256:fefa2faca81fe5f2d465e38741227d44d07c0fedcaf5e92ced037f07afd291ff HTTP/1.0" 404 157 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
INFO[14335] response completed go.version=go1.7.6 http.request.host=app-registry http.request.id=10435650-6769-4fb3-a5c3-6d2becb24658 http.request.method=POST http.request.remoteaddr=10.42.122.106:37812 http.request.uri=/v2/speed-nginx/blobs/uploads/ http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.duration=41.743786ms http.response.status=202 http.response.written=0 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:48 +0000] "POST /v2/speed-nginx/blobs/uploads/ HTTP/1.0" 202 0 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
INFO[14335] response completed go.version=go1.7.6 http.request.host=app-registry http.request.id=6a0428c8-ee18-4549-8064-80a56c04b6dd http.request.method=POST http.request.remoteaddr=10.42.122.106:37808 http.request.uri=/v2/speed-nginx/blobs/uploads/ http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.duration=67.665191ms http.response.status=202 http.response.written=0 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:48 +0000] "POST /v2/speed-nginx/blobs/uploads/ HTTP/1.0" 202 0 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
INFO[14335] response completed go.version=go1.7.6 http.request.host=app-registry http.request.id=a61801c7-c215-44b4-88c3-a5fd59469c9f http.request.method=POST http.request.remoteaddr=10.42.122.106:37810 http.request.uri=/v2/speed-nginx/blobs/uploads/ http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.duration=46.806906ms http.response.status=202 http.response.written=0 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:48 +0000] "POST /v2/speed-nginx/blobs/uploads/ HTTP/1.0" 202 0 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
INFO[14341] response completed go.version=go1.7.6 http.request.host=app-registry http.request.id=1471ae4f-b5d7-4c05-90cb-7c125837fbfe http.request.method=POST http.request.remoteaddr=10.42.122.106:37868 http.request.uri=/v2/speed-nginx/blobs/uploads/ http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.duration=73.810855ms http.response.status=202 http.response.written=0 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:54 +0000] "POST /v2/speed-nginx/blobs/uploads/ HTTP/1.0" 202 0 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
INFO[14341] response completed go.version=go1.7.6 http.request.host=app-registry http.request.id=86e88418-a141-4000-a3fd-3c9c6d4664d2 http.request.method=POST http.request.remoteaddr=10.42.122.106:37866 http.request.uri=/v2/speed-nginx/blobs/uploads/ http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.duration=61.720106ms http.response.status=202 http.response.written=0 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:54 +0000] "POST /v2/speed-nginx/blobs/uploads/ HTTP/1.0" 202 0 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
INFO[14341] response completed go.version=go1.7.6 http.request.host=app-registry http.request.id=0097346b-38c4-4b94-8e20-6e64dfd65c7a http.request.method=POST http.request.remoteaddr=10.42.122.106:37864 http.request.uri=/v2/speed-nginx/blobs/uploads/ http.request.useragent=docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\)) http.response.duration=176.55553ms http.response.status=202 http.response.written=0 instance.id=dc472e1d-1267-4ea3-80ae-3cf905e4890d version=v2.6.2
10.42.122.106 - - [30/Mar/2018:17:14:54 +0000] "POST /v2/speed-nginx/blobs/uploads/ HTTP/1.0" 202 0 "" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.4.0-109-generic os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \(linux\))"
我的 docker 引擎版本是 17.12.0-ce
并拥有 nginx + 私有注册表。
问题出在 nginx.config. 在 proxy_pass 中使用上游导致 DNS lookup error。我替换了上游值并开始工作。好像是个bug！
docker服务状态：
Mar 30 20:47:03 docker-s-2vcpu-4gb-lon1-01 dockerd[1369]: time="2018-03-30T20:47:03.885277507Z" level=error msg="Upload failed, retrying: dial tcp: lookup app-registry on x.x.x.x:53: no such host"