如何在logstash中查看源是kafka还是beat？

Question

我的日志有两个数据源。一个是 beat，一个是 kafka，我想根据源创建 ES 索引。 if kafka -> 前缀 index_name with kafka, and if beat 索引名称前缀 with beat.

input {
  beats {
    port => 9300
  }
}

input {
  kafka {
    bootstrap_servers => "localhost:9092"
    topics => ["my-topic"]
    codec => json
  }
}

output {
    # if kafka
      elasticsearch {
        hosts => "http://localhost:9200"
        user => "elastic"
        password => "password"
        index  => "[kafka-topic]-my-index"
     }
    # else if beat
    elasticsearch {
        hosts => "http://localhost:9200"
        user => "elastic"
        password => "password"
        index  => "[filebeat]-my-index"
     }
}

Answer 1

在您的输入中添加标签并使用它们来过滤输出。

input {
  beats {
    port => 9300
    tags => ["beats"]
  }
}

input {
  kafka {
    bootstrap_servers => "localhost:9092"
    topics => ["my-topic"]
    codec => json
    tags => ["kafka"]
  }
}

output {
    if "beats" in [tags] { 
        output for beats 
    }
    if "kafka" in [tags] { 
        output for kafka 
    }
}

如何在logstash中查看源是kafka还是beat？

How to check if the source is kafka or beat in logstash?

apache-kafka

logstash

filebeat