我想用logstash将数据导入elasticsearch。数据是压缩格式,因为它是一个大数据。
这里是我到目前为止已经试过:如何使用logstash将压缩的json导入elasticsearch?
input {
file {
path => "C:/Users/lenovo-pc/Desktop/test.zip"
start_position => "beginning"
type=> "mytest"
codec => "gzip_lines"
}
}
filter {
json { source => "message" }
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "testing"
}
stdout { codec => rubydebug }
}
我跑logstash -f import.conf
后得到了什么,是在这里:
Could not find log4j2 configuration at path /Bitnami/elk-5.2.2-0/logstash/config/log4j2.properties. Using default config which logs to console
13:53:11.640 [main] FATAL logstash.runner - An unexpected error occurred! {:error=>#<ArgumentError: Could not coerce (9600-9700) into a port range>, :backtrace=>["C:/Bitnami/elk-5.2.2-0/logstash/logstash-core/lib/logstash/settings.rb:352:in `coerce'", "C:/Bitnami/elk-5.2.2-0/logstash/logstash-core/lib/logstash/settings.rb:237:in `set'", "C:/Bitnami/elk-5.2.2-0/logstash/logstash-core/lib/logstash/settings.rb:61:in `set_value'", "C:/Bitnami/elk-5.2.2-0/logstash/logstash-core/lib/logstash/settings.rb:80:in `merge'", "org/jruby/RubyHash.java:1342:in `each'", "C:/Bitnami/elk-5.2.2-0/logstash/logstash-core/lib/logstash/settings.rb:80:in `merge'", "C:/Bitnami/elk-5.2.2-0/logstash/logstash-core/lib/logstash/settings.rb:115:in `validate_all'", "C:/Bitnami/elk-5.2.2-0/logstash/logstash-core/lib/logstash/runner.rb:210:in `execute'", "C:/Bitnami/elk-5.2.2-0/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "C:/Bitnami/elk-5.2.2-0/logstash/logstash-core/lib/logstash/runner.rb:183:in `run'", "C:/Bitnami/elk-5.2.2-0/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "C:\\Bitnami\\elk-5.2.2-0\\logstash\\lib\\bootstrap\\environment.rb:71:in `(root)'"]}
好心帮我解决这个问题。
样品我test.zip文件:Check here
你能分享一个简单的文件,我可以试试。我怀疑这个问题不是来自zip文件。 – Adonis
@asettouf好的让我编辑共享示例的问题。 –
请检查编辑后的问题 –