使用 Logstash 7.2.0 的 Snow API

Question

我正在尝试在 Logstash 7.2.0 中使用以下配置查询雪 API，

   input {
  http_poller {
    urls => {
    snowinc => {
    url => "https://l.service-now.com/api/now/table/incident?sysparm_limit=10"
    user => "logstash_user"
    password => "hello123"
    headers => {Accept => "application/json"}
    }
}
request_timeout => 60
metadata_target => "http_poller_metadata"
schedule => { cron => "* * * * * UTC"}
codec => "json"
}
}
output {
  elasticsearch {
    hosts => ["10.116.55.24:9200"]
    index => "snowinc"
}
        stdout { codec => rubydebug }
}

但是，我在索引中看不到任何数据。（索引创建时没有数据）当我尝试 curl 相同时，我能够看到来自 url 的数据。所以我确定 API URL.

中有数据

日志显示以下错误：

[2019-12-24T01:28:01,510][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"snowinc", :_type=>"_doc", :routing=>nil}, #LogStash::Event:0x13fbd67c], :response=>{"index"=>{"_index"=>"snowinc", "_type"=>"_doc", "id"=>"OkiXNm8Be2NFZrotRzc", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"mapper [result.assigned_to] of different type, current_type [text], merged_type [ObjectMapper]"}}}}

请帮助找出我哪里出错了。

提前致谢

Answer 1

尝试使用过滤器 http 插件，如下所示，

filter {
http {
    url => "https://service-now.com/api/now/table/incident"
    verb => "GET"
    user => "logstash_user"
    password => "user222"
    query => {
          "sysparm_limit" => 10
        }
        target_body => "[snowData]"
    headers => {
    "Content-Type" => "application/json"
    Accept => "application/json"
    }
    }
}

Answer 2

问题出在 URL 数据本身，它有几个 reference_link 不允许正确转换的列。

将 URL 更改为 sysparm_exclude_reference_link=true，效果惊人！

感谢大家的帮助。

使用 Logstash 7.2.0 的 Snow API

Snow API with Logstash 7.2.0

api

logstash

elastic-stack