Kubernetes 中的 Pod 始终处于挂起状态

Question

我在 CloudStack 的 CentOS 虚拟机中运行遇到 Kubernetes 问题。我的 pods 仍处于待处理状态。打印 pod 的日志时收到以下错误消息：

    [root@kubernetes-master ~]# kubectl logs wildfly-rc-6a0fr
    Error from server: Internal error occurred: Pod "wildfly-rc-6a0fr" in namespace "default" : pod is not in 'Running', 'Succeeded' or 'Failed' state - State: "Pending"

如果我在 pod 上启动 describe 命令，结果如下：

[root@kubernetes-master ~]# kubectl describe pod wildfly-rc-6a0fr
Name:               wildfly-rc-6a0fr
Namespace:          default
Image(s):           jboss/wildfly
Node:               kubernetes-minion1/
Start Time:         Sun, 03 Apr 2016 15:00:20 +0200
Labels:             name=wildfly
Status:             Pending
Reason:             
Message:            
IP:             
Replication Controllers:    wildfly-rc (2/2 replicas created)
Containers:
  wildfly-rc-pod:
    Container ID:   
    Image:      jboss/wildfly
    Image ID:       
    QoS Tier:
      cpu:      BestEffort
      memory:       BestEffort
    State:      Waiting
    Ready:      False
    Restart Count:  0
    Environment Variables:
Volumes:
  default-token-0dci1:
    Type:   Secret (a secret that should populate this volume)
    SecretName: default-token-0dci1
Events:
  FirstSeen LastSeen    Count   From                SubobjectPath               Reason  Message
  ───────── ────────    ─────   ────                ─────────────               ──────  ───────
  8m        8m      1   {kubelet kubernetes-minion1}    implicitly required container POD   Pulled  Container image "registry.access.redhat.com/rhel7/pod-infrastructure:latest" already present on machine
  8m        8m      1   {kubelet kubernetes-minion1}    implicitly required container POD   Created Created with docker id 97c1a3ea4aa5
  8m        8m      1   {kubelet kubernetes-minion1}    implicitly required container POD   Started Started with docker id 97c1a3ea4aa5
  8m        8m      1   {kubelet kubernetes-minion1}    spec.containers{wildfly-rc-pod}     Pulling pulling image "jboss/wildfly"

Kubelet 有一些我打印的错误below.Is这可能是因为虚拟机只有 5GB 的存储空间？

systemctl status -l kubelet
● kubelet.service - Kubernetes Kubelet Server
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
   Active: active (running) since lun 2016-04-04 08:08:59 CEST; 9min ago
     Docs: https://github.com/GoogleCloudPlatform/kubernetes
 Main PID: 2112 (kubelet)
   Memory: 39.3M
   CGroup: /system.slice/kubelet.service
           └─2112 /usr/bin/kubelet --logtostderr=true --v=0 --api-servers=http://kubernetes-master:8080 --address=0.0.0.0 --allow-privileged=false --pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest

apr 04 08:13:33 kubernetes-minion1 kubelet[2112]: W0404 08:13:33.877859    2112 kubelet.go:1690] Orphaned volume "167d0ead-fa29-11e5-bddc-064278000020/default-token-0dci1" found, tearing down volume
apr 04 08:13:53 kubernetes-minion1 kubelet[2112]: W0404 08:13:53.887279    2112 kubelet.go:1690] Orphaned volume "9f772358-fa2b-11e5-bddc-064278000020/default-token-0dci1" found, tearing down volume
apr 04 08:14:35 kubernetes-minion1 kubelet[2112]: I0404 08:14:35.341994    2112 provider.go:91] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
apr 04 08:14:35 kubernetes-minion1 kubelet[2112]: E0404 08:14:35.397168    2112 manager.go:1867] Failed to create pod infra container: impossible: cannot find the mounted volumes for pod "wildfly-rc-oroab_default"; Skipping pod "wildfly-rc-oroab_default"
apr 04 08:14:35 kubernetes-minion1 kubelet[2112]: E0404 08:14:35.401583    2112 pod_workers.go:113] Error syncing pod 167d0ead-fa29-11e5-bddc-064278000020, skipping: impossible: cannot find the mounted volumes for pod "wildfly-rc-oroab_default"
apr 04 08:14:58 kubernetes-minion1 kubelet[2112]: E0404 08:14:58.076530    2112 manager.go:1867] Failed to create pod infra container: impossible: cannot find the mounted volumes for pod "wildfly-rc-1aimv_default"; Skipping pod "wildfly-rc-1aimv_default"
apr 04 08:14:58 kubernetes-minion1 kubelet[2112]: E0404 08:14:58.078292    2112 pod_workers.go:113] Error syncing pod 9f772358-fa2b-11e5-bddc-064278000020, skipping: impossible: cannot find the mounted volumes for pod "wildfly-rc-1aimv_default"
apr 04 08:15:23 kubernetes-minion1 kubelet[2112]: W0404 08:15:23.879138    2112 kubelet.go:1690] Orphaned volume "56257e55-fa2c-11e5-bddc-064278000020/default-token-0dci1" found, tearing down volume
apr 04 08:15:28 kubernetes-minion1 kubelet[2112]: E0404 08:15:28.574574    2112 manager.go:1867] Failed to create pod infra container: impossible: cannot find the mounted volumes for pod "wildfly-rc-43b0f_default"; Skipping pod "wildfly-rc-43b0f_default"
apr 04 08:15:28 kubernetes-minion1 kubelet[2112]: E0404 08:15:28.581467    2112 pod_workers.go:113] Error syncing pod 56257e55-fa2c-11e5-bddc-064278000020, skipping: impossible: cannot find the mounted volumes for pod "wildfly-rc-43b0f_default"

好心的人可以帮助我吗？提前致谢。

Answer 1

Kubernetes application troubleshooting guide 推荐运行 kubectl describe pod wildfly-rc-6a0fr，这应该说明为什么 pod 还没有被移出 pending 状态。

Answer 2

我遇到了同样的问题。我有两个微型实例作为控制器，还有 gpu 实例用于计算。我发现有一些 dns pods 正在等待中，所以我扩大了控制器节点池，然后 pods 开始工作了。

Answer 3

这主要是在 pod 无法连接到主服务器时出现的。当我们设置 EKS 集群时，这是一个很常见的错误。人们只是启用 public 端点访问并面临这个问题。要涵盖的几个重要方面：

启用专用端点访问。这样 VPC 下的工作节点或 pod 可以连接
设置安全组并在集群设置时与EKS集群映射。确保 pod n worker 安全组应该添加到具有 443 端口访问权限的入口规则中。

Answer 4

运行下面的命令获取事件。这将显示未安排 Pod 的问题（以及所有其他事件）。

kubectl get events

Answer 5

对我来说，kubectl get events 和 kubectl describe 都没有提供足够的信息，有用的是 kubectl logs pod_name（可选 -n namespace_name）

Answer 6

这可能是由于资源问题。尝试增加实例的容量。

Kubernetes 中的 Pod 始终处于挂起状态

Pod in Kubernetes always in pending state

kubernetes