配置正则表达式的 Logstash 问题

Question

我有以下日志条目：

2017-08-29 01:10:11.111 [http-noo-111-exe-1] TRACE com.javasystemsolutions.xml.gateway.Actions - The XML Gateway encountered an error. The message was Server with id OPA is not configured.

The template in use was TEST_Create_Incident_elkmonitoring.

The server in use was OPA.

The input XML was <incident>
       <summary>Test Monitoring - Summary</summary>
       <notes>Test Monitoring - Summary</notes>
       <product>ELK FAQ</product> </incident> com.javasystemsolutions.xml.gateway.ServerNotFoundException: Server with id OPA is not configured
       at com.javasystemsolutions.xml.gateway.input.PostActions.doPost(PostActions.java:215) [jss-xmlgateway.jar:?]
       at com.javasystemsolutions.xml.gateway.input.PostActions.postAction(PostActions.java:86) [jss-xmlgateway.jar:?]

我想做的是使用正则表达式并识别事件标签之间的文本，但似乎有问题，尽管我的正则表达式适用于 regex101 网站和 configtest returns 配置好的。我的配置如下所示，有人知道哪里出了问题吗？

# The # character at the beginning of a line indicates a comment. Use
# comments to describe your configuration.
input {
    file {
        type => "logs"
        path => "C:/logs/*.log"
        add_field => [ "Application", "ELK_GW_Test" ]
        add_field => [ "secret", "1234" ]
        start_position => beginning

        codec => multiline {
            pattern => "(^%{TIMESTAMP_ISO8601})"
            #negate => true
            what => "previous"
        }
    }
}
filter {
    #multiline {
      #pattern => "(^%{TIMESTAMP_ISO8601})"
      #negate => true
      #what => "previous"
    #}
    #if "_grokparsefailure" in [tags] {
      #drop { }
    #}
    if [host] == "host1" {
        grok {
            match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{NOTSPACE} %{LOGLEVEL:Severity} %{GREEDYDATA:log_message}"}
        }
        grok {
        match => {"message" => "<incident>(?<incident>[\s\S]*)</incident>"}
    }
    }
}
output {
    tcp {
        host => "host1.com"
        port => 1234
        codec => "json_lines"
    }
    #if  "The message was Server with id " in [log_message]  {
    #email {
            #from => "log-mailer@XYZ.com"
            #subject => "Central logstash alert"
            #to => "my_email@ABC.com"
            #via => "smtp"
            #body => "The incident details are: %{incident} \nLog file: %{path}"
            #options => {
                #starttls => "true"
                #smtpIporHost => "email.XYZ.com"
                #port => "587"
                #userName => "log-mailer@XYZ.com" 
                # email-server-mail-id
                # password => "password"
                #authenticationType => "LOGIN"
            #}
        #}
    #}
}

Answer 1

这部分配置错误：

    grok {
        match => ["requested_incident", "(?s)<incident>.+?</incident>"]
    }

试试这个：

    grok {
        match => {"message" => "<incident>(?<incident>[\s\S]*)</incident>"}
    }

我使用了自定义模式，它将在消息字段中进行搜索。找到的内容将放在名为事件的字段中。

配置正则表达式的 Logstash 问题

Logstash issue with config regex

logstash

logstash-grok

elastic-stack