"Too many open files" 在 Elasticsearch 上
"Too many open files" on Elasticsearch
我有一个集群 ES,从 2 天前开始我有这个错误:
[2016-03-14 15:08:48,342][WARN ][cluster.action.shard] [node-01] [logstash-2016.03.14][2] received shard failed for [logstash-2016.03.14][2], node[72oHnFiXTVqgXaUYKTAu2Q], [P], s[INITIALIZING], indexUUID [AKybrAEZRXebyRxcmzTqJQ], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[logstash-2016.03.14][2] failed recovery]; nested: EngineCreationFailureException[[logstash-2016.03.14][2] failed to open reader on writer]; nested: FileSystemException[/var/lib/elasticsearch/elasticsearch/nodes/0/indices/logstash-2016.03.14/2/index/_lg_Lucene410_0.dvm: Too many open files]; ]]
如果我 运行 curl -XGET 'http://localhost:9200/_nodes?os=true&process=true&pretty=true' | grep "max_file_descriptors"
我有这个输出:
"max_file_descriptors" : 65535,
"max_file_descriptors" : 65535,
到文件 /etc/security/limit.conf 我已经有了:
elasticsearch - nofile 65535
elasticsearch - memlock unlimited
如果我检查索引的状态,我有一些索引红色....因此 logstash 出错并全部崩溃
只需将 elasticsearch - nofile 设置为无限制。
并确保您 运行 是正确的用户。
我有一个集群 ES,从 2 天前开始我有这个错误:
[2016-03-14 15:08:48,342][WARN ][cluster.action.shard] [node-01] [logstash-2016.03.14][2] received shard failed for [logstash-2016.03.14][2], node[72oHnFiXTVqgXaUYKTAu2Q], [P], s[INITIALIZING], indexUUID [AKybrAEZRXebyRxcmzTqJQ], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[logstash-2016.03.14][2] failed recovery]; nested: EngineCreationFailureException[[logstash-2016.03.14][2] failed to open reader on writer]; nested: FileSystemException[/var/lib/elasticsearch/elasticsearch/nodes/0/indices/logstash-2016.03.14/2/index/_lg_Lucene410_0.dvm: Too many open files]; ]]
如果我 运行 curl -XGET 'http://localhost:9200/_nodes?os=true&process=true&pretty=true' | grep "max_file_descriptors"
我有这个输出:
"max_file_descriptors" : 65535,
"max_file_descriptors" : 65535,
到文件 /etc/security/limit.conf 我已经有了:
elasticsearch - nofile 65535
elasticsearch - memlock unlimited
如果我检查索引的状态,我有一些索引红色....因此 logstash 出错并全部崩溃
只需将 elasticsearch - nofile 设置为无限制。 并确保您 运行 是正确的用户。