使用 ELK 解析复杂的 XML 文件

Question

我最近开始学习 ELK，但很难理解如何解析 XML 数据。我想解析我的 XML 文件，它看起来像这样：

<Name nameID="xxxx">
  <Type p="1">xxxxxx</Type>
  <Type p="2">xxxxxx</Type>
    .
    .
  <Type p="9">xxxxx</Type>
  <Value obj="1"> 
    <r p="1">5.94</r>
    <r p="2">62.19</r>
    .
    .
    <r p="9">7.19</r>
  </Value>
  <Value obj="2"> 
    <r p="1">5.94</r>
    <r p="2">62.19</r>
    .
    .
    <r p="9">7.19</r>
  </Value>
</Name>
<Name nameID="yyyy">
  <Type p="1">yyyyy</Type>
  <Type p="2">yyyyyy</Type>
  <Type p="3">yyyy</Type>
  <Value obj="1"> 
    <r p="1">54.94</r>
    <r p="2">6.19</r>
    <r p="3">0</r>
  </Value>
</Name>

我想得到类似的东西：在输出中

"NameID = name1
Type = Type1
obj = obj1
Value = xx
"
"NameID = name1
Type = Type2
obj = obj1
Value = xx
"
"NameID = name1
Type = Type3
obj = obj1
Value = xx
"
...etc
and then
"NameID = name1
Type = Type1
obj = obj2
Value = xx
"
"NameID = name1
Type = Type2
obj = obj2
Value = xx
"
....etc

我使用了这个 logstash.conf 但我没有得到我真正需要的东西（我为每个字段得到一个数组）

input {
    file {
        path => "/home/test/data.xml"
        start_position => beginning
        sincedb_path => "/dev/null"
        codec => multiline
        {
            pattern => "<Name"
            negate => true
            what => "previous"
        }
    }
}
filter
{
    xml {
        source => "message"
        target => "parsed"
        add_tag => "xml"
        xpath => [
            "//Name/@nameID","Name",
            "//Type/@p","TypeID",
            "//Type/text()","Type",
            "//Value/@obj","Obj",
            "//r/text()","value"]

Answer 1

在代码中使用 logstash ruby filter plugin. (need to require a gem？）
和rubyparseXML，到
构建 json 文档准备索引

Answer 2

解决方案：

filter { xml { source => "message" store_xml => true target => "theXML" force_array => false } }
split { field => "[theXML][Type]" }
split { field => "[theXML][Value]" }
split { field => "[theXML][Value][r]" }

然后在输出中使用：

output{
if [theXML][Type][p]==[theXML][Value][p]{
elasticsearch ....}}

希望对某人有所帮助 ;)

使用 ELK 解析复杂的 XML 文件

Parse complex XML file using ELK

xml

elasticsearch

logstash

kibana

elastic-stack