如何使用 logstash 处理由空格分隔的文本文件？

Question

我的日志文件

maker model mileage manufacture_year engine_displacement engine_power body_type color_slug stk_year transmission door_count seat_count fuel_type date_created date_last_seen price_eur ford galaxy 151000 2011 2000 103 None man 5 7 diesel 2015-11-14 18:10:06.838319+00 2016-01-27 20:40:15.46361+00 10584.75 skoda octavia 143476 2012 2000 81 None man 5 5 diesel 2015-11-14 18:10:06.853411+00 2016-01-27 20:40:15.46361+00 8882.31 bmw 97676 2010 1995 85 None man 5 5 diesel 2015-11-14 18:10:06.861792+00 2016-01-27 20:40:15.46361+00 12065.06 skoda fabia 111970 2004 1200 47 None man 5 5 gasoline 2015-11-14 18:10:06.872313+00 2016-01-27 20:40:15.46361+00 2960.77 skoda fabia 128886 2004 1200 47 None man 5 5 gasoline 2015-11-14 18:10:06.880335+00 2016-01-27 20:40:15.46361+00 2738.71

错误如下

[2018-03-23T11:35:20,226][ERROR][logstash.agent]Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::PluginLoadingError", :message=>"Couldn't find any filter plugin named 'txt'. Are you sure this is correct?

我的配置文件如下

input {file {path => "/home/elk/data/logklda.txt"start_position => "beginning" sincedb_path => "/dev/null"}}
filter{txt {separator => " "columns => ["name","type","category","date","error_log"]}} output {elasticsearch {hosts => "localhost"index => "logklda"document_type => "category"}stdout{}}

Answer 1

有名为 "txt" 的过滤器不存在。我认为您必须查看此 logstash 指南站点并更正您的代码。 https://www.elastic.co/guide/en/logstash/current/filter-plugins.html

Answer 2

过滤器文本文件不存在。从您的配置来看，您似乎想使用 csv filter。在您的情况下，将 txt 替换为 csv，配置将如下所示：

csv {
  separator => " "
  columns => ["name","type","category","date","error_log"]
}

如何使用 logstash 处理由空格分隔的文本文件？

How to process a text file separated by spaces using logstash?

config

elasticsearch

logstash