CPU 坚持 XXs

CPU stuck for XXs

我发现 CPU 卡在虚拟机 (CentOS 7.2) 中。

[root@ha-node1 ~]# 
Message from syslogd@ha-node1 at Jul 31 13:41:56 ...
 kernel:BUG: soft lockup - CPU#6 stuck for 21s! [swapper/6:0]

所以我使用 dmesg 来检查 CPU 卡住了:

[root@ha-node1 ~]# dmesg |grep stuck
[141504.185667] BUG: soft lockup - CPU#1 stuck for 23s! [stonithd:57926]
[141532.172112] BUG: soft lockup - CPU#1 stuck for 23s! [stonithd:57926]
[141541.654004] BUG: soft lockup - CPU#3 stuck for 79s! [ksoftirqd/3:279]
[141541.654974] BUG: soft lockup - CPU#7 stuck for 66s! [corosync:57912]
[141549.948803] BUG: soft lockup - CPU#6 stuck for 87s! [systemd:1]
[141578.290675] BUG: soft lockup - CPU#2 stuck for 27s! [xfsaild/sda2:598]
[141578.290767] BUG: soft lockup - CPU#7 stuck for 27s! [corosync:57912]
[141578.290820] BUG: soft lockup - CPU#6 stuck for 26s! [ksoftirqd/6:294]
[141578.291153] BUG: soft lockup - CPU#3 stuck for 27s! [haproxy:1250]
[141578.303520] BUG: soft lockup - CPU#0 stuck for 27s! [ruby:92261]
[141584.185153] BUG: soft lockup - CPU#5 stuck for 22s! [nova-conductor:49156]
[141593.982198] BUG: soft lockup - CPU#4 stuck for 39s! [kworker/4:0:285]
[141593.982694] BUG: soft lockup - CPU#1 stuck for 39s! [kworker/1:2:4672]
[141604.368498] BUG: soft lockup - CPU#2 stuck for 23s! [2_scheduler:2989]
[141623.603542] BUG: soft lockup - CPU#3 stuck for 25s! [6_scheduler:2993]
[142237.825417] BUG: soft lockup - CPU#7 stuck for 23s! [corosync:57912]
[142392.705639] BUG: soft lockup - CPU#6 stuck for 23s! [swapper/6:0]
[436213.466318] BUG: soft lockup - CPU#6 stuck for 24s! [4_scheduler:2991]
[436214.828007] BUG: soft lockup - CPU#5 stuck for 22s! [aux:2996]
[473345.361656] BUG: soft lockup - CPU#6 stuck for 28s! [haproxy:1250]
[473346.785653] BUG: soft lockup - CPU#7 stuck for 29s! [corosync:57912]
[479503.672417] BUG: soft lockup - CPU#6 stuck for 21s! [swapper/6:0]

后来发现我的宿主机内存是32G,我有3台VM,每台内存都是16G,所以内存超载了。

我还检查了主机的内存使用情况:

所以我把虚拟机的内存改成8G,现在可以正常使用了,没有CPU卡死了。