将 log4net 级别字段添加到 logstash.conf 文件

Question

我正在尝试添加 LEVEL 字段（以便它显示在 Kibana 中）。我的 logstash.conf

输入：

2018-03-18 15:43:40.7914 - INFO: Tick 
2018-03-18 15:43:40.7914 - ERROR: Tock

文件：

input {
  beats {
    port => 5044
  }
}
filter {
  grok {      
      match => { 
            "message" => "(?m)^%{TIMESTAMP_ISO8601:timestamp}~~\[%{DATA:thread}\]~~\[%{DATA:user}\]~~\[%{DATA:requestId}\]~~\[%{DATA:userHost}\]~~\[%{DATA:requestUrl}\]~~%{DATA:level}~~%{DATA:logger}~~%{DATA:logmessage}~~%{DATA:exception}\|\|"
        }
      match => {
        "levell" => "(?m)^%{DATA:level}"
      }
      add_field => { 
        "received_at" => "%{@timestamp}" 
        "received_from" => "%{host}"
        "level" => "levell"
      }
      remove_field => ["message"]      
    }
  date {
    match => [ "timestamp", "yyyy-MM-dd HH:mm:ss:SSS" ]
  }
}
output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    sniffing => true
    index => "filebeat-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
    #user => "elastic"
    #password => "changeme"
  }
  stdout { codec => rubydebug }
}

这会打印出 "levell" 而不是 "INFO/ERROR" 等等

编辑：输入：

2018-03-18 15:43:40.7914 - INFO: Tick

配置：

# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.

input {
  beats {
    port => 5044
  }
}
filter {
  grok {      
      match => { "message" => "(?m)^%{TIMESTAMP_ISO8601:timestamp}~~\[%{DATA:thread}\]~~\[%{DATA:user}\]~~\[%{DATA:requestId}\]~~\[%{DATA:userHost}\]~~\[%{DATA:requestUrl}\]~~%{DATA:level}~~%{DATA:logger}~~%{DATA:logmessage}~~%{DATA:exception}\|\|" }
      add_field => { 
        "received_at" => "%{@timestamp}" 
        "received_from" => "%{host}"
      } 
    }
  grok {      
      match => { "message" => "- %{LOGLEVEL:level}" }
      remove_field => ["message"]      
    }
  date {
    match => [ "timestamp", "yyyy-MM-dd HH:mm:ss:SSS" ]
  }
}
output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    sniffing => true
    index => "filebeat-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
    #user => "elastic"
    #password => "changeme"
  }
  stdout { codec => rubydebug }
}

我得到的输出。仍然缺少 received_at 和级别：

Answer 1

在那部分配置中：

  add_field => { 
    "received_at" => "%{@timestamp}" 
    "received_from" => "%{host}"
    "level" => "levell"
  }

使用 "level" => "levell" 时，只需将字符串 levell 放入字段 level 即可。要放置名为 levell 的字段的值，您必须使用 %{levell}。所以在你的情况下，它看起来像：

  add_field => { 
    "received_at" => "%{@timestamp}" 
    "received_from" => "%{host}"
    "level" => "%{levell}"
  }

也是grok#match，据documentation：

A hash that defines the mapping of where to look, and with which patterns.

因此尝试在 levell 字段上进行匹配是行不通的，因为它看起来还不存在。您用来匹配 message 字段的 grok 模式与您提供的示例不匹配。

将 log4net 级别字段添加到 logstash.conf 文件

Add log4net Level field to logstash.conf file

logstash

logstash-configuration