这个自定义日志模式的 Grok 模式是什么？

Question

以下是我日志的一小部分：

2018-12-06 18:55:20 INFO  epo - myfile.xml is loaded successfully
2018-12-06 18:55:20 INFO  epo - checking that whether the given file name is already present
2018-12-06 18:55:20 INFO  epo - some logging deatils
2018-12-06 18:55:20 INFO  epo - Entry has been added to table.
2018-12-06 18:55:20 INFO  epo - Total number of records processed 0000035
2018-12-06 18:55:20 INFO  epo - some logging deatils
2018-12-07 09:57:59 INFO  epo - myfile.xml is loaded successfully
2018-12-07 09:57:59 INFO  epo - [ElasticSearch] => PIN07122018F00001 request sent successfully.
2018-12-06 18:55:20 INFO  epo - myfile.xml is loaded successfully
2018-12-06 18:55:20 INFO  epo - checking that whether the given file name is already present
2018-12-06 18:55:20 INFO  epo - some logging deatils
2018-12-06 18:55:20 INFO  epo - Entry has been added to table.
2018-12-06 18:55:20 INFO  epo - Total number of records processed 0000035
2018-12-06 18:55:20 INFO  epo - some logging deatils
2018-12-07 09:57:59 INFO  epo - myfile.xml is loaded successfully
2018-12-07 09:57:59 INFO  epo - [ElasticSearch] => PIN07122018F00002 request sent unsuccessfully.

在此日志中，我想 select 行包含 PIN07122018F00001 和 PIN07122018F00002 等请求 ID，并将其发送到弹性搜索。

为此我正在使用 logstash，我的 grok 模式是：

input {
  . . .
}

filter {
  grok {
    patterns_dir => ["/myServer/mnt/appln/folder1/folder2/logstash/pattern"]
    match => { "message" => '^%{TIMESTAMP_ISO8601:timestamp} INFO  epo - \[ElasticSearch\] => %{REQ_ID:requestid} %{MSG:statusmsg}$' }
  }
}

output{
    . . .
}

其中 DEPOSITORY_REQ_ID 和 MSG 定义为：

MSG (A-Za-z0-9 )+
REQ_ID PIN[0-9]{8}[A-Z]{1}[0-9]{5}

但我仍然无法匹配所需的行，这种模式占用了所有行。请告诉我匹配该行的模式是什么：

2018-12-07 09:57:59 INFO epo - [ElasticSearch] => PIN07122018F00001 request sent successfully.

请帮忙。

Answer 1

问题出在 MSG 模式上。 () 表示一个捕获组，它将尝试匹配 () 的确切内容。你想在你的案例中使用的是 []，它表示一个字符 class，它将匹配来自 class 的所有字符。它还缺少出现在行尾的 .。

您的模式应该这样定义，这将解决您的问题：

MSG [A-Za-z0-9 \.]+

这个自定义日志模式的 Grok 模式是什么？

what will be the Grok pattern for this custom log pattern?

logstash

logstash-grok