消费kafka话题使用logstash elasticSearch

Question

我开始消费从kafka消息logstash，我想发送完整的主题elasticSearch，但我没有得到任何消息logstash，我可以看到在kafka消息来了，但从卡夫卡方面我没有看到任何东西，什么是正确的方式来配置它？

input { 
    kafka { 
    zk_connect => "localhost:2181" 
    topic_id => "event" 
    } 
} 

output{ 
    stdout{ 
    codec => rubydebug 
    } 
    elasticsearch{ 
    index => "event-%{+YYYY.MM.dd}" 
    hosts => ["localhost:9201"] 
    codec => json 
    } 
} 

curl localhost:9201 
{ 
    "name" : "Flex", 
    "cluster_name" : "elasticsearch", 
    "version" : { 
    "number" : "2.3.4", 
    "build_hash" : "e455fd0c13dceca8dbbdbb1665d068ae55dabe3f", 
    "build_timestamp" : "2016-06-30T11:24:31Z", 
    "build_snapshot" : false, 
    "lucene_version" : "5.5.0" 
    }, 
    "tagline" : "You Know, for Search" 
} 

命令：

/kafka-console-consumer.sh --zookeeper localhost:2181 --topic event 

产生的结果不时。

Answer 1

尝试像这样与auto_offset_reset和reset_beginning：

kafka { 
    topic_id => "event" 
    zk_connect => "localhost:2181" 
    group_id => "event-group" 
    auto_offset_reset => "smallest" 
    reset_beginning => true 
    consumer_threads => 1 
    }