如何在 elasticsearch 上使用 logstash 获取 url 路径

Question

我已经使用我的 logstash 配置进行了测试

127.0.0.1 - - [02/Jun/2016:15:38:57 +0900] "GET /ad/adInfos?id=1 HTTP/1.1" 404 68

filter {
  grok {
    match => { "message" => "%{COMMONAPACHELOG}" }
  }
}

运行如下

{
        "message" => "127.0.0.1 - - [02/Jun/2016:15:39:02 +0900] \"POST /ad/signIn?id=1 HTTP/1.1\" 200 26",
       "@version" => "1",
     "@timestamp" => "2016-06-02T06:39:02.000Z",
           "path" => "/opt/node-v4.3.1/logs/access.log",
           "host" => "0.0.0.0",
       "clientip" => "127.0.0.1",
          "ident" => "-",
           "auth" => "-",
      "timestamp" => "02/Jun/2016:15:39:02 +0900",
           "verb" => "POST",
        "request" => "/ad/signIn?id=1
    "httpversion" => "1.1",
       "response" => "200",
          "bytes" => "26"
}

但我只想获取 URL 路径，路径参数除外：/ad/signIn

因为请求计数每个 REST API。

我该怎么办？

Answer 1

您只需在第一个 grok 之后添加第二个 grok，如下所示：

grok {
  match => { "request" => "%{URIPATH:path}" }
  named_captures_only => false
}

这样做是获取您的 request 字段并使用 URIPATH 模式再次解析它，并将结果存储在 path 字段中（参见最后一个字段）。

{
        "message" => "127.0.0.1 - - [02/Jun/2016:15:38:57 +0900] \"GET /ad/adInfos?id=1 HTTP/1.1\" 404 68",
       "@version" => "1",
     "@timestamp" => "2016-06-03T04:54:49.631Z",
           "host" => "iMac-de-Consulthys.local",
       "clientip" => "127.0.0.1",
          "ident" => "-",
           "auth" => "-",
      "timestamp" => "02/Jun/2016:15:38:57 +0900",
           "verb" => "GET",
        "request" => "/ad/adInfos?id=1",
    "httpversion" => "1.1",
       "response" => "404",
          "bytes" => "68",
           "path" => "/ad/adInfos"
}

如何在 elasticsearch 上使用 logstash 获取 url 路径

How to get url path using logstash on elasticsearch

logstash