filebeat 忽略多个探矿者中的 logiles

Question

我尝试用 multible prospectors 配置一个 filebeat。 Filebeat 注册了所有探矿者，但忽略了来自 appA 的本地主机日志文件和来自 appB

的日志文件

我的filebeat.yml:

    filebeat.prospectors:
    - type: log
      paths:
        - /vol1/appA_instance01/logs/wrapper_*.log
        - /vol1/appA_instance02/logs/wrapper_*.log
      fields: 
         log_type: "appAlogs"
         environment: "stage1"
      exclude_files: [".gz$"]
    - type: log
      paths:
        - /vol1/appA_instance01/logs/localhost.*.log
        - /vol1/appA_instance02/logs/localhost.*.log
      fields: 
         log_type: "localhostlogs"
         environment: "stage1"
      exclude_files: [".gz$"]
    - type: log
      paths:
        - /vol1/appB_instance01/logs/*.log
        - /vol1/appB_instance02/logs/*.log
      fields: 
         log_type: "appBlogs"
      environment: "stage1"
      exclude_files: [".gz$"]
   output.logstash:
   hosts: ["<HOST>:5044"]

filebeat 日志文件：

2017-11-15T17:32:56+01:00 INFO Home path: [/usr/share/filebeat] Config   path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2017-11-15T17:32:56+01:00 INFO Setup Beat: filebeat; Version: 5.6.3
2017-11-15T17:32:56+01:00 INFO Max Retries set to: 3
2017-11-15T17:32:56+01:00 INFO Activated logstash as output plugin.
2017-11-15T17:32:56+01:00 INFO Publisher name: host
2017-11-15T17:32:56+01:00 INFO Flush Interval set to: 1s
2017-11-15T17:32:56+01:00 INFO Max Bulk Size set to: 2048
2017-11-15T17:32:56+01:00 INFO filebeat start running.
2017-11-15T17:32:56+01:00 INFO Registry file set to: /var/lib/filebeat/registry
2017-11-15T17:32:56+01:00 INFO Loading registrar data from /var/lib  /filebeat/registry
2017-11-15T17:32:56+01:00 INFO States Loaded from registrar: 222
2017-11-15T17:32:56+01:00 INFO Loading Prospectors: 3
2017-11-15T17:32:56+01:00 INFO Starting Registrar
2017-11-15T17:32:56+01:00 INFO Start sending events to output
2017-11-15T17:32:56+01:00 INFO Starting spooler: spool_size: 2048; idle_timeout: 5s
2017-11-15T17:32:56+01:00 INFO Prospector with previous states loaded: 40
2017-11-15T17:32:56+01:00 INFO Starting prospector of type: log; id: 12115431240338587115 
2017-11-15T17:32:56+01:00 INFO Harvester started for file: /vol1/appA_instance01/logs/wrapper_20171115.log
2017-11-15T17:32:56+01:00 INFO Prospector with previous states loaded: 182
2017-11-15T17:32:56+01:00 INFO Starting prospector of type: log; id: 18163435272915459714 
2017-11-15T17:32:56+01:00 INFO Prospector with previous states loaded: 0
2017-11-15T17:32:56+01:00 INFO Starting prospector of type: log; id: 16959079668827945694 
2017-11-15T17:32:56+01:00 INFO Loading and starting Prospectors completed. Enabled prospectors: 3
2017-11-15T17:33:06+01:00 INFO Harvester started for file: /vol1/appA_instance02/logs/wrapper_20171115.log

filebeat 忽略 logiles 的原因是什么？

/vol1/appA_instance01/logs/localhost.*.log
/vol1/appA_instance02/logs/localhost.*.log
/vol1/appB_instance01/logs/*.log
/vol1/appB_instance02/logs/*.log

问候 niesel

Answer 1

附加的日志显示所有三个探矿者都已启动，注册表文件似乎有状态。您确定 Filebeat 之前没有读取过被忽略的日志文件吗？它是否从这些日志文件中读取新行？

Filebeat 不会重新读取日志文件。所以这些文件可能是以前读取过的。

filebeat 忽略多个探矿者中的 logiles

filebeat ignore logiles in multible prospectors

elasticsearch

logstash

filebeat