用于 ping 的 Logstash Grok 模式

Question

我从日志中得到了这个样本：

Tue Mar 27 06:51:48 2018 PING www.google.com (172.217.169.100) 56(84) bytes of data.
64 bytes from sof02s31-in-f4.1e100.net (172.217.169.100): icmp_seq=1 ttl=128 time=17.4 ms

--- www.google.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 17.482/17.482/17.482/0.000 ms

我想为 logstash 创建一个 grok 模式并提取诸如 TIMESTAMP,IPV4,TTL 之类的东西，以及 RTT 值 min/avg/max 来自最后两行。

此日志大约每秒从一个 ping 脚本到同一 IP。我想我需要一个多行模式来一次获取这 6 行中每一行的值？

任何帮助都会很棒！！！

谢谢

Answer 1

如果使用 Oniguruma syntax 转义 newline 即 \n，则不需要多行。

例如，(?<newline>(.|\r|\n)*) 可以匹配两段之间日志中所有不必要的数据，即，

" time=17.4 ms\n\n--- www.google.com ping statistics ---\n1 packets transmitted, 1 received, 0% packet loss, time 0ms\n"

您最终的 grok 模式将如下所示，

%{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{YEAR} %{WORD:PING} %{HOSTNAME:host} \(%{IP:ip_address}\) %{DATA} ttl=%{INT:TTL}(?<newline>(.|\r|\n)*)rtt min/avg/max/mdev = %{NUMBER:min}/%{NUMBER:avg}/%{NUMBER:max}/%{NUMBER:mdev} ms

它将产生以下输出，

{
  "DAY": [
    [
      "Tue"
    ]
  ],
  "MONTH": [
    [
      "Mar"
    ]
  ],
  "MONTHDAY": [
    [
      "27"
    ]
  ],
  "TIME": [
    [
      "06:51:48"
    ]
  ],
  "HOUR": [
    [
      "06"
    ]
  ],
  "MINUTE": [
    [
      "51"
    ]
  ],
  "SECOND": [
    [
      "48"
    ]
  ],
  "YEAR": [
    [
      "2018"
    ]
  ],
  "PING": [
    [
      "PING"
    ]
  ],
  "host": [
    [
      "www.google.com"
    ]
  ],
  "ip_address": [
    [
      "172.217.169.100"
    ]
  ],
  "IPV6": [
    [
      null
    ]
  ],
  "IPV4": [
    [
      "172.217.169.100"
    ]
  ],
  "DATA": [
    [
      "56(84) bytes of data. 64 bytes from sof02s31-in-f4.1e100.net (172.217.169.100): icmp_seq=1"
    ]
  ],
  "TTL": [
    [
      "128"
    ]
  ],
  "newline": [
    [
      " time=17.4 ms\n\n--- www.google.com ping statistics ---\n1 packets transmitted, 1 received, 0% packet loss, time 0ms\n"
    ]
  ],
  "min": [
    [
      "17.482"
    ]
  ],
  "BASE10NUM": [
    [
      "17.482",
      "17.482",
      "17.482",
      "0.000"
    ]
  ],
  "avg": [
    [
      "17.482"
    ]
  ],
  "max": [
    [
      "17.482"
    ]
  ],
  "mdev": [
    [
      "0.000"
    ]
  ]
}

用于 ping 的 Logstash Grok 模式

Logstash Grok Pattern for ping

logstash

logstash-grok

logstash-configuration