使用具有大量表的数据库在 elasticsearch 中提取数据

Question

input {
  jdbc {
    jdbc_driver_library => "mysql-connector-java-5.1.36-bin.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql://localhost:3306/mydb"
    jdbc_user => "mysql"
    parameters => { "favorite_artist" => "Beethoven" }
    schedule => "* * * * *"
    statement => "SELECT * from songs where artist = :favorite_artist"
  }
}

在上面的logstash配置文件中如何摄取数据？
当我必须 select 多个表时该怎么办？

Answer 1

数据将根据 "Select statement query" 获取。如果你想有来自多个表的数据，那么你可以结合所有的连接查询表和查询的相关输出将被提取到 ES.It 一切都取决于在您的特定用例上。这是一些粘贴下来的示例供您参考。

input {
jdbc {
jdbc_driver_library => "mysql-connector-java-5.1.36-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://localhost:3306/mydb"
jdbc_user => "mysql"
parameters => { "favorite_artist" => "Beethoven" }
schedule => "* * * * *"
statement => "SELECT * FROM songs INNER JOIN song_folder using song_number ORDER BY 
song_title;"
}
}

output{
elasticsearch{
hosts=>"http://xx:XX:XX:XX:9200"
index=>"song"
document_type=>"songname"
document_id=>"song_title"
}
stdout{codec=>rubydebug}
}


Please let me know , if you have any further queries.

使用具有大量表的数据库在 elasticsearch 中提取数据

Data ingestion in elasticsearch with database having large number of tables

mysql

jdbc

elasticsearch

logstash