从 Logstash 过滤器中的输入引用字段

Question

我有以下输入，由 FileBeat 生成

{
  "@timestamp": "2018-12-04T09:21:33.360Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "doc",
    "version": "6.5.0"
  },
  "message": "the message",
  "prospector": {"type": "log"},
  "input": {"type": "log"},
  "beat": {
    "name": "linuxkit-025000000001",
    "hostname": "linuxkit-025000000001",
    "version": "6.5.0"
  },
  "host": {"name": "linuxkit-025000000001"},
  "source": "/opt/foo/logs/bar.log",
  "offset": 9893715,
  "log": {"flags": ["multiline"]}
}

我想在我的脚本中使用 message 字段，所以我尝试使用以下过滤器

filter {
    mutate {
        add_field => {
            "decoded_base64" => ruby {
                path => "scripts/my_script.rb"
                script_params => { "msg" => "${[message]}" }
            }
        }
    }
}

但我不断 NoMethodError 引用方法 "get"。

logstash    | [2018-12-11T14:05:47,765][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:cep_logs, 
:exception=>"NoMethodError", 
:message=>"undefined method `get' for #<Java
::OrgLogstashConfigIrImperative::PluginStatement:0x29fc9676>
\nDid you mean?  getClass\n               get_class\n               gem", 
:backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler/lscl.rb:210:in `expr'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler/lscl.rb:202:in `expr'", "/usr/share/logstash/logstash-core/lib/logstash/compiler/lscl.rb:133:in `expr'", "org/jruby/RubyArray.java:2486:in `map'",
"/usr/share/logstash/logstash-core/lib/logstash/compiler/lscl.rb:97:in `expr_attributes'", "/usr/share/logstash/logstash-core/lib/logstash/compiler/lscl.rb:75:in `expr'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/log
stash-core/lib/logstash/compiler/lscl.rb:68:in `expr'", "/usr/share/logstash/logstash-core/lib/logstash/compiler/lscl.rb:47:in `block in compile'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstas
h/compiler/lscl.rb:45:in `compile'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:45:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:49:in `compile_graph'", "/usr/share/logstash/logsta
sh-core/lib/logstash/compiler.rb:11:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:10:in `compile_sources'", "org/logstash/execution/AbstractPipelineE
xt.java:149:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:22:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:90:in `initialize'", "/usr/share/logstash/logstash-core/lib/logsta
sh/pipeline_action/create.rb:38:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:309:in `block in converge_state'"]}

有人能解释一下为什么会这样吗？

我认为这无关紧要，但以防万一我使用的脚本是

require "base64"

def register(params)
    @msg = params["msg"]
end

def filter(event)
    if @msg.matches(/<DataB64>(.*)<\/DataB64>/)
        return Base64.decode64()
    end
end

Answer 1

除非发生了我在文档中没有看到的更改，否则您不应该使用 mutate->add_field 到运行您的 ruby。尝试：

filter {
    ruby {
        ...
    }
}

并让您的 ruby 代码添加带有 [event.set][1] 的字段。

从 Logstash 过滤器中的输入引用字段

Referencing field from input in a Logstash filter

logstash

logstash-configuration