logstash 配置中 JDBC 插件的问题

Question

我有这个配置文件（logstash）：

input {
  beats {
    port => 5044
    host => "0.0.0.0"
  }
}

filter {
  json {
    source => "message"
    target => "log"
  }
}

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "logs-%{+YYYY.MM.dd}"
  }

  jdbc {
    driver_jar_path => "/etc/logstash/mysql-connector-java-8.0.11.jar"
    driver_class => "com.mysql.cj.jdbc.Driver"
    connection_string => "jdbc:mysql://localhost:3306/cste?user=master&password=testets!123"
    statement => ["INSERT INTO cste_log (ip, log, event, created, inserted) VALUES(?,?,?,?,?)", "log.userid", "log", "log.event", "@timestamp", "@timestamp"]
  }

  stdout {
    codec => "rubydebug"
  }

用于将数据保存为 mySQL 数据库。但它不适用于错误消息（列 'ip'、'event' 不能为空）

我认为 'jdbc.statement' 的语法有误，我正在努力修正它。 'output.elasticsearch' 效果很好。

{
   "agent" => {
             "version" => "7.10.0",
                "name" => "DESKTOP-GEB1AGR",
                  "id" => "7e109ece-5874-4149-9842-21acb86c9da0",
                "type" => "filebeat",
            "hostname" => "DESKTOP-GEB1AGR",
        "ephemeral_id" => "0730755e-f234-48c4-b7f1-2d2339df0e86"
    },
      "@version" => "1",
    "@timestamp" => 2020-11-23T06:31:59.005Z,
           "log" => {
           "userid" => "192.111.11.111",
        "writetime" => "2020/11/23 15:31:51",
           "target" => "crackme.exe - PID: 5528 - Module: ntdll.dll - Thread: Main Thread 3240 (switched from 19C0)",
            "event" => "dbgRestart"
    },
         "input" => {
        "type" => "log"
    },
           "ecs" => {
        "version" => "1.6.0"
    },
       "message" => "{\"writetime\": \"2020/11/23 15:31:51\", \"userid\": \"111.111.111.111\", \"target\": \"crackme.exe - PID: 5528 - Module: ntdll.dll - Thread: Main Thread 3240 (switched from 19C0)\",  \"event\": \"dbgRestart\"} ",
          "host" => {
                "name" => "DESKTOP-GEB1AGR",
        "architecture" => "x86_64",
                  "os" => {
             "version" => "10.0",
                "name" => "Windows 10 Home",
               "build" => "16299.1087",
              "family" => "windows",
            "platform" => "windows",
              "kernel" => "10.0.16299.1087 (WinBuild.160101.0800)"
        },
                  "id" => "659f1b29-3-2cb22793a39c",
                  "ip" => [
            [0] "fe80::adb9:b",
            [1] "192.168.43.",
            [2] "2001:0:348b:",
            [3] "fe80::180947e"
        ],
            "hostname" => "DESKTOP-GEB1AGR",
                 "mac" => [
            [0] "00:0c:6c:d7",
            [1] "00:00:00:e0"
        ]
    },
          "tags" => [
        [0] "beats_input_codec_plain_applied"
    ]
}

如何使用值 'writetime' 和 'event'？请给我一些建议。

 "log" => {
           "userid" => "192.168.43.129",
        "writetime" => "2020/11/23 15:31:51",
           "target" => "crackme.exe - PID: 5528 - Module: ntdll.dll - Thread: Main Thread 3240 (switched from 19C0)",
            "event" => "dbgRestart"},

Answer 1

如果事件是日志对象中的一个字段，那么在 logstash 中您将其称为“[log][event]”。 [log.event] 指的是名称中包含句点的字段。与“[log][userid]”类似。

logstash 配置中 JDBC 插件的问题

Problem of JDBC Plugin in logstash config

mysql

jdbc

elasticsearch

logstash

elastic-stack