fluentd无法/显着连接到elasticsearch当ES安装在不同的端口

在我的码头环境中有弹性搜索（ES-1）暴露于端口9200，这是主要ES维护DB数据，所以为了日志记录我们使用fluentd-elasticsearch-kibna堆栈，并在端口9700（ES-2）上运行新的弹性搜索。fluentd无法/显着连接到elasticsearch当ES安装在不同的端口

我无法fluentd连接到这个新的弹性搜索（ES-2）端口9700。让我知道如何连接fluentd到上课比其他端口“9200”

下面

是使用泊坞窗的配置细节对于FEK

fluentd.config : 
    <source> 
    @type forward 
    port 24224 
    bind 0.0.0.0 
</source> 

<match *.**> 
    @type copy 
    <store> 
    @type elasticsearch 
    host elasticsearchkibana 
    port 9200 
    logstash_format true 
    logstash_prefix fluentd 
    logstash_dateformat %Y%m%d 
    include_tag_key true 
    type_name access_log 
    tag_key @log_name 
    flush_interval 1s 
    </store> 
    <store> 
    @type stdout 
    </store> 
</match> 


docker-compose.yml : 

    version: '2' 
services: 
    web: 
    image: httpd 
    ports: 
     - "80:80" 
    links: 
     - fluentd 
    logging: 
     driver: "fluentd" 
     options: 
     fluentd-address: localhost:24224 
     tag: httpd.access 

    elasticsearchkibana: 
    image: elasticsearch 
    environment: 
     - "ES_JAVA_OPTS=-Xms512m -Xmx512m" 
    expose: 
     - 9800 
    ports: 
     - "9700:9300" 
     - "9800:9200" 

    fluentd: 
    build: ./fluentd 
    volumes: 
     - ./fluentd/conf:/fluentd/etc 
    links: 
     - "elasticsearchkibana" 
    ports: 
     - "24224:24224" 
     - "24224:24224/udp" 

    kibana: 
    image: kibana 
    environment: 
     - ELASTICSEARCH_URL=http://elasticsearchkibana:9200 
#  - "ES_HOST=192.168.241.82" 
#  - "ES_PORT=9700" 
    ports: 
     - "5601:5601"

下面是错误消息我收到当我使用上面的配置

fluentd_1    | 2017-09-25 15:16:36 +0000 fluent.warn: {"next_retry":"2017-09-25 15:16:37 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:37 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
fluentd_1    | 2017-09-25 15:16:37 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:39 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4" 
fluentd_1    | 2017-09-25 15:16:37 +0000 [warn]: suppressed same stacktrace 
fluentd_1    | 2017-09-25 15:16:37 +0000 fluent.warn: {"next_retry":"2017-09-25 15:16:39 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:39 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
fluentd_1    | 2017-09-25 15:16:39 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:43 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4" 
fluentd_1    | 2017-09-25 15:16:39 +0000 [warn]: suppressed same stacktrace 
fluentd_1    | 2017-09-25 15:16:39 +0000 fluent.warn: {"next_retry":"2017-09-25 15:16:43 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:43 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
fluentd_1    | 2017-09-25 15:16:43 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:50 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4" 
fluentd_1    | 2017-09-25 15:16:43 +0000 [warn]: suppressed same stacktrace 
fluentd_1    | 2017-09-25 15:16:43 +0000 fluent.warn: {"next_retry":"2017-09-25 15:16:50 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:50 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
fluentd_1    | 2017-09-25 15:16:50 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:17:04 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4" 
fluentd_1    | 2017-09-25 15:16:50 +0000 [warn]: suppressed same stacktrace 
fluentd_1    | 2017-09-25 15:16:50 +0000 fluent.warn: {"next_retry":"2017-09-25 15:17:04 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:17:04 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
fluentd_1    | 2017-09-25 15:17:04 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:17:39 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4" 
fluentd_1    | 2017-09-25 15:17:04 +0000 [warn]: suppressed same stacktrace 
fluentd_1    | 2017-09-25 15:17:04 +0000 fluent.warn: {"next_retry":"2017-09-25 15:17:39 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:17:39 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
fluentd_1    | 2017-09-25 15:17:39 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:18:46 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4" 
fluentd_1    | 2017-09-25 15:17:39 +0000 fluent.warn: {"next_retry":"2017-09-25 15:18:46 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:18:46 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
fluentd_1    | 2017-09-25 15:17:39 +0000 [warn]: suppressed same stacktrace 
fluentd_1    | 2017-09-25 15:18:46 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:20:49 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4" 
fluentd_1    | 2017-09-25 15:18:46 +0000 [warn]: suppressed same stacktrace 
fluentd_1    | 2017-09-25 15:18:46 +0000 fluent.warn: {"next_retry":"2017-09-25 15:20:49 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:20:49 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
[email protected]:/home/tlirlogin/docker-compose-efk-master2# cd fluentd/conf/

感谢

来源

2017-09-25 k.explorer

您的fluentd运行在容器网络内而不是主机。所以即使您在主机上映射到9700，端口仍然需要9200。因为在这种情况下连接取决于容器端口 –

@tarun Laiwani 更改时仍然出错 fluentd_1 fluent.warn：error_class“：”Fluent :: ElasticsearchOutput :: ConnectionFailure“，”error“：”无法到达Elasticsearch集群（{：host => \“elasticsearchkibana \”，：port => 9200，：scheme => \“http \”}）！“，”plugin_id“：”object：2ac5be462aa4“，”message“：”暂时失败刷新缓冲区。 next_retry = 2017-09-26 05:14:24 +0000 error_class = \“Fluent :: ElasticsearchOutput :: ConnectionFailure \”error = \“无法到达Elasticsearch集群 –

您能分享fluentd dockerfile吗？所以我可以给你写作文件一射 –

你需要改变你的fluentd配置如下

fluentd.config : 
<source> 
    @type forward 
    port 24224 
    bind 0.0.0.0 
</source> 

<match *.**> 
    @type copy 
    <store> 
    @type elasticsearch 
    host elasticsearchkibana 
    port 9200 
    logstash_format true 
    logstash_prefix fluentd 
    logstash_dateformat %Y%m%d 
    include_tag_key true 
    type_name access_log 
    tag_key @log_name 
    flush_interval 1s 
    </store> 
    <store> 
    @type stdout 
    </store> 
</match>

所以基本上添加host elasticsearchkibana并更改为port 9200。这是因为我们应该关注容器端口而不是主机端口

来源

2017-09-25 17:59:46

fluentd无法/显着连接到elasticsearch当ES安装在不同的端口

回答

相关问题