docker 上的 Kubernetes 创建具有空 serviceaccount 且没有令牌的容器，导致容器崩溃并重新启动

Question

1.3.0 和高达 1.4.0-alpha.0 仍然遇到类似问题

在我的例子中（docker 设置），trusty 或 kubedns 都会从 api 服务器获得授权。

奇怪的是，我发现实例中没有秘密，在路径 /var/run/secrets/kubernetes.io/serviceaccount

下

[root@ ... ]#  kubectl exec -it kube-dns-v13-htfjo ls /bin/sh
/ #
/ # ls /var/run/secrets/kubernetes.io/serviceaccount
/ #

虽然看起来它们在节点和代理实例中

tmpfs on /var/lib/kubelet/pods/3de53b0c-45bb-11e6-9f03-08002776167a/volumes/kubernetes.io~secret/default-token-8axd8 type
tmpfs on /var/lib/kubelet/pods/3de5591e-45bb-11e6-9f03-08002776167a/volumes/kubernetes.io~secret/default-token-8axd8 type
tmpfs on /var/lib/kubelet/pods/f29f35c7-45cc-11e6-9f03-08002776167a/volumes/kubernetes.io~secret/default-token-ql88q type

删除机密并删除 pods 然后重新创建它们无效
卸载并删除文件夹后重新启动集群也没有效果

自然会导致kubedns无法启动。在下面记录

I0709 09:04:11.578816       1 dns.go:394] Received DNS Request:kubernetes.default.svc.cluster.local., exact:false
I0709 09:04:11.578873       1 dns.go:427] records:[], retval:[], path:[local cluster svc default kubernetes]
I0709 09:04:11.579657       1 dns.go:394] Received DNS Request:kubernetes.default.svc.cluster.local., exact:false
I0709 09:04:11.579677       1 dns.go:427] records:[], retval:[], path:[local cluster svc default kubernetes]
E0709 09:04:11.786646       1 reflector.go:216] pkg/dns/dns.go:128: Failed to list *api.Service: serializer for text/html; charset=utf-8 doesn't exist
E0709 09:04:11.786995       1 reflector.go:216] pkg/dns/dns.go:127: Failed to list *api.Endpoints: serializer for text/html; charset=utf-8 doesn't exist
I0709 09:04:12.488674       1 dns.go:145] Ignoring error while waiting for service default/kubernetes: serializer for text/html; charset=utf-8 doesn't exist. Sleeping 1s before retrying.
E0709 09:04:12.879701       1 reflector.go:216] pkg/dns/dns.go:128: Failed to list *api.Service: serializer for text/html; charset=utf-8 doesn't exist
E0709 09:04:12.880000       1 reflector.go:216] pkg/dns/dns.go:127: Failed to list *api.Endpoints: serializer for text/html; charset=utf-8 doesn't exist
I0709 09:04:13.582561       1 dns.go:145] Ignoring error while waiting for service default/kubernetes: serializer for text/html; charset=utf-8 doesn't exist. Sleeping 1s before retrying.

Answer 1

这似乎是一个尚未解决的错误

https://github.com/kubernetes/kubernetes/issues/26943

可行的解决方法是在 kubelet 安装中添加 rslave 选项，如 --volume=/var/lib/kubelet:/var/lib/kubelet:rw,rslave 如图所示以下。

此解决方案还依赖于平台。阅读错误报告中的注释。

## Start kubernetes master
sudo docker run \
--volume=/:/rootfs:ro \
--volume=/sys:/sys:ro \
--volume=/var/lib/docker/:/var/lib/docker:rw \
**--volume=/var/lib/kubelet:/var/lib/kubelet:rw,rslave** \
--volume=/var/run:/var/run:rw \
--net=host \
--privileged=true \
--pid=host \
-d \
gcr.io/google_containers/hyperkube-amd64:${K8S_VERSION} \
/hyperkube kubelet \
    --allow-privileged=true \
    --api-servers=http://localhost:8080 \
    --v=2 \
    --address=0.0.0.0 \
    --enable-server \
    --hostname-override=127.0.0.1 \
    --config=/etc/kubernetes/manifests-multi \
    --containerized \
    --cluster-dns=10.0.0.10 \
    --cluster-domain=cluster.local

docker 上的 Kubernetes 创建具有空 serviceaccount 且没有令牌的容器，导致容器崩溃并重新启动

Kubernetes on docker creates containers with empty serviceaccount and no tokens leading to container crash and restarts

crash

unauthorized

docker

kubernetes

kube-dns