在 nodejs 中使用 grok 模块时遇到问题

Question

我正在尝试使用 grok 解析节点中的一些日志，它似乎在 grok 调试器中工作，但当我在节点中运行它时却不行。

在http://grokdebug.herokuapp.com/我做：

输入：[2016-02-01 15:29:02,039] INFO [Replica state machine on controller 0]: Invoking state change to OnlineReplica for replicas [Topic=elk-test,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)

模式：\[%{TIMESTAMP_ISO8601:timestamp}\] %{LOGLEVEL:level} \[%{DATA:message1}\]: %{GREEDYDATA:message2}

输出这个：

{
  "timestamp": [
    [
      "2016-02-01 15:29:02,039"
    ]
  ],
  "YEAR": [
    [
      "2016"
    ]
  ],
  "MONTHNUM": [
    [
      "02"
    ]
  ],
  "MONTHDAY": [
    [
      "01"
    ]
  ],
  "HOUR": [
    [
      "15",
      null
    ]
  ],
  "MINUTE": [
    [
      "29",
      null
    ]
  ],
  "SECOND": [
    [
      "02,039"
    ]
  ],
  "ISO8601_TIMEZONE": [
    [
      null
    ]
  ],
  "level": [
    [
      "INFO"
    ]
  ],
  "message1": [
    [
      "Replica state machine on controller 0"
    ]
  ],
  "message2": [
    [
      "Invoking state change to OnlineReplica for replicas [Topic=elk-test,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)"
    ]
  ]
}

在nodejs中，我尝试这样做：

'use strict';

var nodegrok = require('node-grok');
var Regex = require("regex");
var zlib = require('zlib');

var str2 = '[2016-02-01 15:29:02,039] INFO [Replica state machine on controller 0]: Invoking state change to OnlineReplica for replicas [Topic=elk-test,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)'

var p2 = '\[%{TIMESTAMP_ISO8601:timestamp}\] %{LOGLEVEL:level} \[%{DATA:message1}\]: %{GREEDYDATA:message2}'


var patterns = require('node-grok').loadDefaultSync();
var pattern = patterns.createPattern(p2)
console.log('pattern:', pattern.parseSync(str2));

但随后出现此错误：

/Users/usrxxx/kafka_process_lambda/node_modules/node-grok/node_modules/oniguruma/lib/onig-reg-exp.js:9
      this.scanner = new OnigScanner([this.source]);
                     ^

Error: empty range in char class
    at Error (native)
    at new OnigRegExp (/Users/usrxxx/kafka_process_lambda/node_modules/node-grok/node_modules/oniguruma/lib/onig-reg-exp.js:9:22)
    at GrokPattern.t.parseSync (/Users/usrxxx/kafka_process_lambda/node_modules/node-grok/lib/index.js:38:24)
    at Object.<anonymous> (/Users/usrxxx/kafka_process_lambda/index.js:12:33)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Function.Module.runMain (module.js:441:10)
    at startup (node.js:139:18)

Answer 1

从 example 可以明显看出，您需要正确转义字符 [ 和 ]。因此，在您的代码中，更改...

var p2 = '\[%{TIMESTAMP_ISO8601:timestamp}\] %{LOGLEVEL:level} \[%{DATA:message1}\]: %{GREEDYDATA:message2}'

...至：

var p2 = '\[%{TIMESTAMP_ISO8601:timestamp}\] %{LOGLEVEL:level} \[%{DATA:message1}\]: %{GREEDYDATA:message2}'

然后你的代码输出：

$ node app.js 
pattern: { timestamp: '2016-02-01 15:29:02,039',
  level: 'INFO',
  message1: 'Replica state machine on controller 0',
  message2: 'Invoking state change to OnlineReplica for replicas [Topic=elk-test,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)' }

在 nodejs 中使用 grok 模块时遇到问题

trouble using grok module in nodejs

node.js

logstash-grok