Logstash 时间戳问题 - 解析值比日志值晚一小时

Question

我正在使用以下代码读取 McAfee 日志（我选择使用 CSV 过滤器，因为 grok 过滤器原来很乱）

input {

 stdin{}
}

filter {

csv {
         columns => ["timestamp", "McAf_ThreatSeverity", "McAf_Event", "McAf_EventDescription", "McAf_EventCategory", "McAf_ThreatT$

          separator => "|"
    }


date {
        locale => "en"
        match => ["timestamp", "dd/MM/YYYY:HH:mm:ss"]
        timezone => "Europe/London"
        add_tag => "McAfee_ThreatEventLog"
   }

  }

output {
    elasticsearch {
       #action => "index"
        host => "localhost"
        }
    stdout {
        codec => rubydebug
     }
}

输入是这样的...

31/03/2015:12:59:07| SEVERITY_CRITICAL | 1093| Buffer Overflow detected and blocked| Host intrusion buffer overflow| 10.3.1.252| Blocked| buffer overflow

输出正常，除了时间戳比日志中的时间戳晚一小时。也就是说，

Using milestone 2 filter plugin 'csv'. This plugin should be stable, but if you see strange behavior, please let us know! For more information on plugin milestones, see http://logstash.net/docs/1.4.2/plugin-milestones {:level=>:warn}
31/03/2015:12:59:07| SEVERITY_CRITICAL | 1093| Buffer Overflow detected and blocked| Host intrusion buffer overflow| 10.3.1.252| Blocked| buffer overflow
{
                         "message" => [
        [0] "31/03/2015:12:59:07| SEVERITY_CRITICAL | 1093| Buffer Overflow detected and blocked| Host intrusion buffer overflow| 10.3.1.252| Blocked| buffer overflow"
    ],
                        "@version" => "1",
                      "@timestamp" => "2015-03-31T11:59:07.000Z",
                            "host" => "UOD-220076",
                       "timestamp" => "31/03/2015:12:59:07",
             "McAf_ThreatSeverity" => " SEVERITY_CRITICAL ",
                      "McAf_Event" => " 1093",
           "McAf_EventDescription" => " Buffer Overflow detected and blocked",
              "McAf_EventCategory" => " Host intrusion buffer overflow",
    "McAf_ThreatTargetIPv4Address" => " 10.3.1.252",
                "McAf_ActionTaken" => " Blocked",
                 "McAf_ThreatType" => " buffer overflow",
                            "tags" => [
        [0] "McAfee_ThreatEventLog"
    ]
}

在上面的输出中，日志中的时间是 12:59:07 但解析的值是 11:59:07.

有趣的是，我昨天使用以下代码解析了不同的日志（防火墙日志）：

input {

 stdin{}
}

filter {

csv {
         columns => ["timestamp", "Interface", "Origin", "Type", "Action", "Service", "SourcePort", "SourceIP", "DestinationIP", "P$
         separator => "|"
    }


date {
        locale => "en"
        match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss"]
        timezone => "Europe/London"
        add_tag => "checkpoint_fw"

   }


}

output {
    elasticsearch {
        host => "localhost"
        }
    stdout {

以及以下输入....

18/MAR/2015:15:00:00| eth3-04|熟料1|日志|接受|网址| 46718| 193.60.148.101| 23.194.230.180| TCP| 0| | | inzone：本地；外区：外部； service_id：http； message_info：隐含规则|安全 Gateway/Management

它给出了正确的时间戳输出，如下所示：

    [0] "18/MAR/2015:15:00:00| eth3-04| grog1| Log| Accept| http| 46718| 193.60.148.101| 23.194.230.180| tcp| 0| | | inzone: Local; outzone: External; service_id: http; message_info: Implied rule| Security Gateway/Management"
],
     "@version" => "1",
   "@timestamp" => "2015-03-18T15:00:00.000Z",
         "host" => "UOD-220076",
    "timestamp" => "18/MAR/2015:15:00:00",
    "Interface" => " eth3-04",
       "Origin" => " grog1",
         "Type" => " Log",
       "Action" => " Accept",
      "Service" => " http",
   "SourcePort" => " 46718",
     "SourceIP" => " 193.60.148.101",
"DestinationIP" => " 23.194.230.180",
     "Protocol" => " tcp",
         "Rule" => " 0",
     "RuleName" => " ",
  "CurrentRule" => " ",
  "Information" => " inzone: Local; outzone: External; service_id: http; message_info: Implied rule",
      "Product" => " Security Gateway/Management",
         "tags" => [
    [0] "checkpoint_fw"

McAfee 脚本和防火墙脚本之间的唯一区别（除了字段之外）是防火墙脚本使用 dd/MMM/YYYY 而 McAfee 脚本使用 dd/MM/YYYY - 我非常怀疑那会有显着差异。

有人知道如何让 McAfee 时间戳与日志中的内容完全一致吗？（我什至尝试过更改时区，但所做的只是增加时差）- 有没有办法在 logstash（不是 kibana）中说“+ 1 小时”？

谢谢。

Answer 1

答案很简单。 Europe/London 时区是从 3 月 29 日开始的英国夏令时 (GMT+100)。您的其他条目是从 18 日开始的，早于 BST 标记。

Logstash 时间戳问题 - 解析值比日志值晚一小时

Logstash timestamp issue - the parsed value is one hour behind the log value

timestamp

timezone-offset

logstash