Logstash 和 Elastic 升级

Question

我在 5.1 版上有一个功能正常的 Logstash 和 Elasticsearch。

我删除了所有索引，然后升级到 6.1。

现在，当 Logstash 从 Filebeat（仍然是 5.1 版）接收到一些事件时，它会抛出此错误：

[2017-12-27T17:29:16,463][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch.
{
  :status => 400,
  :action => ["index", {:_id=>nil, :_index=>"logstash-2017.12.27", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x34de85bd>],
  :response => {
    "index" => {
      "_index" => "logstash-2017.12.27",
      "_type" => "doc",
      "_id" => nil,
      "status" => 400,
      "error" => {
        "type" => "mapper_parsing_exception",
        "reason" => "Failed to parse mapping [_default_]: [include_in_all] is not allowed for indices created on or after version 6.0.0 as [_all] is deprecated. As a replacement, you can use an [copy_to] on mapping fields to create your own catch all field.",
        "caused_by" => {
          "type" => "mapper_parsing_exception",
          "reason" => "[include_in_all] is not allowed for indices created on or after version 6.0.0 as [_all] is deprecated. As a replacement, you can use an [copy_to] on mapping fields to create your own catch all field."
        }
      }
    }
  }
}

我什至尝试过使用极其简单的管道，如您所见：

input {
  beats {
    port => 5044
  }
}   

filter {
    json {
      source => "message"
    }   
}   

output {
  elasticsearch { hosts => ["localhost:9200"] }
}

然而它一遍又一遍地抛出这个错误。

知道这里有什么问题吗？

Answer 1

看看changes in mapping, introduced in elasticsearch 6.0

您需要从索引模板中删除 include_in_all 映射参数。

你能在这里粘贴你的 template/mapping 吗？

Answer 2

这个答案只是对@alexanderlz 所说内容的扩展。从 kibana 的 DevTools 页面我运行这个：

GET /_template/

列出所有模板

这里是我们需要删除/修改的模板（部分）：

"logstash": {
    "order": 0,
    "version": 60001,
    "index_patterns": [
      "logstash-*"
    ],

那么运行

DELETE /_template/logstash

完成后重新启动 logstash，它将重新安装新的正确模板。

Logstash 和 Elastic 升级

Logstash and Elastic upgrade

elasticsearch

logstash

logstash-configuration