Docker 使用 Letsencrypt 的 HAProxy SSL 终止

Question

我目前有一个 docker 设置，使用 haproxy 作为负载均衡器，将流量引导到容器运行我的网络应用程序。我正在尝试向 HAProxy 添加 SSL 终止并运行遇到了一些麻烦。当我将 DEFAULT_SSL_CERT 作为环境变量添加到我的 haproxy 容器时，我收到以下错误：

Mar 20 20:15:03 escapes-artist kernel:  [3804709.167813] aufs au_opts_verify:1597:dockerd[1595]: dirperm1 breaks the protection by the permission bits on the lower branch
Mar 20 20:15:03 escapes-artist kernel:  [3804709.213993] aufs au_opts_verify:1597:dockerd[1595]: dirperm1 breaks the protection by the permission bits on the lower branch
Mar 20 20:15:04 escapes-artist kernel:  [3804709.674840] aufs au_opts_verify:1597:dockerd[1595]: dirperm1 breaks the protection by the permission bits on the lower branch
Mar 20 20:15:04 escapes-artist kernel:  [3804709.688631] device vethebd7d1d entered promiscuous mode
Mar 20 20:15:04 escapes-artist kernel:  [3804709.688767] IPv6: ADDRCONF(NETDEV_UP): vethebd7d1d: link is not ready
Mar 20 20:15:04 escapes-artist systemd-udevd:  Could not generate persistent MAC address for veth5c0585c: No such file or directory
Mar 20 20:15:04 escapes-artist systemd-udevd:  Could not generate persistent MAC address for vethebd7d1d: No such file or directory
Mar 20 20:15:04 escapes-artist dockerd:  time="2017-03-21T02:15:04.671620998Z" level=warning msg="Your kernel does not support swap memory limit."
Mar 20 20:15:04 escapes-artist dockerd:  time="2017-03-21T02:15:04.672345380Z" level=warning msg="Your kernel does not support cgroup rt period"
Mar 20 20:15:04 escapes-artist dockerd:  time="2017-03-21T02:15:04.672732724Z" level=warning msg="Your kernel does not support cgroup rt runtime"
Mar 20 20:15:04 escapes-artist dockerd:  time="2017-03-21T02:15:04Z" level=info msg="Firewalld running: false"
Mar 20 20:15:05 escapes-artist kernel:  [3804710.392546] eth0: renamed from veth5c0585c
Mar 20 20:15:05 escapes-artist kernel:  [3804710.395273] IPv6: ADDRCONF(NETDEV_CHANGE): vethebd7d1d: link becomes ready
Mar 20 20:15:05 escapes-artist kernel:  [3804710.395303] br-5c6735a37ece: port 3(vethebd7d1d) entered forwarding state
Mar 20 20:15:05 escapes-artist kernel:  [3804710.395313] br-5c6735a37ece: port 3(vethebd7d1d) entered forwarding state
Mar 20 20:15:05 escapes-artist kernel:  [3804711.072047] br-5c6735a37ece: port 2(vethbaf33bd) entered forwarding state
Mar 20 20:15:08 escapes-artist kernel:  [3804713.819317] haproxy[29684]: segfault at 7f560000003b ip 00007f56f6ac74bb sp 00007ffe45011290 error 4 in libcrypto.so.1.0.0[7f56f69ce000+3f3000]
Mar 20 20:15:11 escapes-artist sshd:  Received disconnect from 122.194.229.7 port 21903:11:  [preauth]
Mar 20 20:15:11 escapes-artist sshd:  Disconnected from 122.194.229.7 port 21903 [preauth]
Mar 20 20:15:13 escapes-artist kernel:  [3804718.789238] haproxy[29686]: segfault at 7fbb0000003b ip 00007fbb747b74bb sp 00007ffc944fcc10 error 4 in libcrypto.so.1.0.0[7fbb746be000+3f3000]
Mar 20 20:15:17 escapes-artist kernel:  [3804722.944073] br-5c6735a37ece: port 1(veth610d1f4) entered forwarding state
Mar 20 20:15:18 escapes-artist kernel:  [3804723.790663] haproxy[29688]: segfault at 7ff10000003b ip 00007ff1ad6004bb sp 00007fffa6f03cb0 error 4 in libcrypto.so.1.0.0[7ff1ad507000+3f3000]
Mar 20 20:15:20 escapes-artist kernel:  [3804725.408060] br-5c6735a37ece: port 3(vethebd7d1d) entered forwarding state
Mar 20 20:15:23 escapes-artist kernel:  [3804728.792134] haproxy[29690]: segfault at 7f130000003b ip 00007f13210c54bb sp 00007ffcbe3f7670 error 4 in libcrypto.so.1.0.0[7f1320fcc000+3f3000]
Mar 20 20:15:28 escapes-artist kernel:  [3804733.823940] haproxy[29692]: segfault at 7f500000003b ip 00007f500b9d94bb sp 00007ffe6d044f10 error 4 in libcrypto.so.1.0.0[7f500b8e0000+3f3000]
Mar 20 20:15:33 escapes-artist kernel:  [3804738.780797] haproxy[29694]: segfault at 7f000000003b ip 00007f00310124bb sp 00007fffd6e979b0 error 4 in libcrypto.so.1.0.0[7f0030f19000+3f3000]

有谁知道如何解决这个问题？我已经尝试了数小时，尝试了证书文件、环境变量等的不同格式，但似乎无法弄清楚任何事情。这是我正在使用的 docker-compose.yml 文件：

version: '2'
services:
  db:
    image: mysql
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: docker
      MYSQL_USER: admin
      MYSQL_PASSWORD: password
    volumes:
      - /storage/docker/mysql-datadir:/var/lib/mysql
    ports:
      - 3306:3306
  web:
    image: myimage
    restart: always
    depends_on:
      - db
    volumes:
      - /home/docker/persistent/media/:/home/docker/code/media/
  lb:
    image: dockercloud/haproxy
    links:
      - web
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /etc/haproxy/certs:/certs
    environment:
      STATS_AUTH: admin:password
      RSYSLOG_DESTINATION: logs5.papertrailapp.com:41747
      DEFAULT_SSL_CERT: (I've tried both pasting cert here directly and a path to cert)
    ports:
      - 80:80
      - 443:443
      - 1936:1936

我在主机上设置了 Letsencrypt 以自动更新。我一直尝试使用的证书是 privkey.pem 和 fullchian.pem 的组合。我试过连接它们，使用 dockercloud/haproxy docs 建议的 awk 1 ORS='\n'，以及我能想到的几乎所有其他配置。任何帮助将不胜感激。

此外，如果我使用 CERT_FOLDER: /certs/ 而不是 DEFAULT_SSL_CERT 并将我的证书存储在 /certs/cert0.pem 中，我就会收到此错误...

Mar 20 21:19:38 escapes-artist dockerd:  time="2017-03-21T03:19:38.840340234Z" level=error msg="containerd: deleting container" error="exit status 1: \"container ce6c0b6df31419691b6593be6744d01c8ccecf5f38851106aa4bb8fac915a63a does not exist\none or more of the container deletions failed\n\""
Mar 20 21:19:38 escapes-artist kernel:  [3808584.302038] br-5c6735a37ece: port 3(veth8b1ea8e) entered disabled state
Mar 20 21:19:38 escapes-artist kernel:  [3808584.302192] veth0bcd06c: renamed from eth0
Mar 20 21:19:38 escapes-artist kernel:  [3808584.320863] br-5c6735a37ece: port 3(veth8b1ea8e) entered disabled state
Mar 20 21:19:38 escapes-artist kernel:  [3808584.321869] device veth8b1ea8e left promiscuous mode
Mar 20 21:19:38 escapes-artist kernel:  [3808584.321874] br-5c6735a37ece: port 3(veth8b1ea8e) entered disabled state
Mar 20 21:19:39 escapes-artist dockerd:  time="2017-03-21T03:19:39.055316431Z" level=error msg="Handler for GET /v1.25/exec/c79e3c9b77f0c84d849cc641a425950d55fcbb22bf566922d3fd12e6a0e12e07/json returned error: Container ce6c0b6df31419691b6593be6744d01c8ccecf5f38851106aa4bb8fac915a63a is not running: Exited (0) Less than a second ago"
Mar 20 21:19:39 escapes-artist kernel:  [3808584.964578] aufs au_opts_verify:1597:dockerd[23058]: dirperm1 breaks the protection by the permission bits on the lower branch
Mar 20 21:19:39 escapes-artist kernel:  [3808585.005699] aufs au_opts_verify:1597:dockerd[23058]: dirperm1 breaks the protection by the permission bits on the lower branch
Mar 20 21:19:40 escapes-artist kernel:  [3808585.489799] aufs au_opts_verify:1597:dockerd[1595]: dirperm1 breaks the protection by the permission bits on the lower branch
Mar 20 21:19:40 escapes-artist kernel:  [3808585.500609] device veth24d6316 entered promiscuous mode
Mar 20 21:19:40 escapes-artist systemd-udevd:  Could not generate persistent MAC address for veth24d6316: No such file or directory
Mar 20 21:19:40 escapes-artist kernel:  [3808585.505055] IPv6: ADDRCONF(NETDEV_UP): veth24d6316: link is not ready
Mar 20 21:19:40 escapes-artist systemd-udevd:  Could not generate persistent MAC address for vethedaad7c: No such file or directory
Mar 20 21:19:40 escapes-artist dockerd:  time="2017-03-21T03:19:40.259076690Z" level=warning msg="Your kernel does not support swap memory limit."
Mar 20 21:19:40 escapes-artist dockerd:  time="2017-03-21T03:19:40.260183880Z" level=warning msg="Your kernel does not support cgroup rt period"
Mar 20 21:19:40 escapes-artist dockerd:  time="2017-03-21T03:19:40.260663645Z" level=warning msg="Your kernel does not support cgroup rt runtime"
Mar 20 21:19:40 escapes-artist dockerd:  time="2017-03-21T03:19:40Z" level=info msg="Firewalld running: false"
Mar 20 21:19:40 escapes-artist kernel:  [3808585.904671] eth0: renamed from vethedaad7c
Mar 20 21:19:40 escapes-artist kernel:  [3808585.918744] IPv6: ADDRCONF(NETDEV_CHANGE): veth24d6316: link becomes ready
Mar 20 21:19:40 escapes-artist kernel:  [3808585.919040] br-5c6735a37ece: port 3(veth24d6316) entered forwarding state
Mar 20 21:19:40 escapes-artist kernel:  [3808585.919058] br-5c6735a37ece: port 3(veth24d6316) entered forwarding state
Mar 20 21:19:44 escapes-artist kernel:  [3808589.585674] haproxy[32235]: segfault at 341 ip 0000000000000341 sp 00007ffe732fe5b8 error 14 in haproxy[55f6998b1000+d1000]
Mar 20 21:19:49 escapes-artist kernel:  [3808594.704226] haproxy[32237]: segfault at 341 ip 0000000000000341 sp 00007ffcb4d1aa08 error 14 in haproxy[563827d10000+d1000]
Mar 20 21:19:54 escapes-artist kernel:  [3808599.669540] haproxy[32239]: segfault at 341 ip 0000000000000341 sp 00007ffd1e8bb1b8 error 14 in haproxy[562d926fa000+d1000]
Mar 20 21:19:55 escapes-artist kernel:  [3808600.928110] br-5c6735a37ece: port 3(veth24d6316) entered forwarding state
Mar 20 21:19:59 escapes-artist kernel:  [3808604.602704] haproxy[32241]: segfault at 341 ip 0000000000000341 sp 00007fff142d0898 error 14 in haproxy[5592e3a63000+d1000]

Answer 1

好的，了解问题所在。 dockercloud/haproxy 映像创建证书文件并将它们放入 /certs/。我在 /certs/ 中安装了一个卷，这把事情搞砸了。我将已安装的卷移至 /shared-certs/，一切正常！

Docker 使用 Letsencrypt 的 HAProxy SSL 终止

Docker HAProxy SSL termination with Letsencrypt

haproxy

docker

docker-compose

lets-encrypt