Logstash不与filebeat一起运行

Question

我正在关注数字海洋指南以安装ELK堆栈。它正在运行，但现在我遇到了logstash的问题。

当我运行调试，我得到这个::

tail -f /var/log/logstash/logstash.log 

{:timestamp=>"2017-02-10T10:38:05.169000-0500", :message=>"Error: Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "} 
{:timestamp=>"2017-02-10T10:38:05.201000-0500", :message=>"You may be interested in the '--configtest' flag which you can\nuse to validate logstash's configuration before you choose\nto restart a running system."} 
{:timestamp=>"2017-02-14T10:51:46.921000-0500", :message=>"fetched an invalid config", :config=>"input {\n lumberjack {\n port => 5044\n type => syslog\n ssl_certificate => \"/etc/pki/tls/certs/logstash-forwarder.crt\"\n ssl_key => \"/etc/pki/tls/private/logstash-forwarder.key\"\n }\n}\n\ninput {\n beats {\n port => 5044\n ssl => true\n ssl_certificate => \"/etc/pki/tls/certs/logstash-forwarder.crt\"\n ssl_key => \"/etc/pki/tls/private/logstash-forwarder.key\"\n }\n}\n\nfilter {\n if [type] == \"syslog\" {\n grok {\n  match => { \"message\" => \"%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{GREEDYDATA:syslog_message}\" }\n  add_field => [ \"received_at\", \"%{@timestamp}\" ]\n  add_field => [ \"received_from\", \"%{host}\" ]\n }\n syslog_pri { }\n date {\n  match => [ \"syslog_timestamp\", \"MMM d HH:mm:ss\", \"MMM dd HH:mm:ss\" ]\n }\n }\n}\n\ninput{\n beats {\n port => 5044\n }\n}\n\noutput {\n elasticsearch {\n hosts => \"10.84.234.224:9200\"\n manage_template => false\n index => \"%{[@metadata][beat]}-%{+YYYY.MM.dd}\"\n document_type => \"%{[metadata][type]}\"\n }\n}\n\noutput {\n elasticsearch {\n hosts => [\"localhost:9200\"]\n sniffing => true\n manage_template => false\n index => \"%{[@metadata][beat]}-%{+YYYY.MM.dd}\"\n document_type => \"%{[@metadata][type]}\"\n }\n\u007Fstdout { codec => rubydebug }\n}\n\noutput {\n elasticsearch { host => localhost }\n stdout { codec => rubydebug }\n}\n\ninput {\n tcp {\n port => 5400\n type => syslog\n }\n udp {\n port => 5400\n type => syslog\n }\n}\n\nfilter {\n if [type] == \"syslog\" {\n grok {\n  match => { \"message\" => \"%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{GREEDYDATA:syslog_message}\" }\n  add_field => [ \"received_at\", \"%{@timestamp}\" ]\n  add_field => [ \"received_from\", \"%{host}\" ]\n }\n date {\n  match => [ \"syslog_timestamp\", \"MMM d HH:mm:ss\", \"MMM dd HH:mm:ss\" ]\n }\n }\n}\n\noutput {\n elasticsearch { hosts => [\"localhost:9200\"] }\n stdout { codec => rubydebug }\n}\n\n\n", :reason=>"Expected one of #, if, \", ', } at line 56, column 1 (byte 1278) after output {\n elasticsearch {\n hosts => [\"localhost:9200\"]\n sniffing => true\n manage_template => false\n index => \"%{[@metadata][beat]}-%{+YYYY.MM.dd}\"\n document_type => \"%{[@metadata][type]}\"\n }\n", :level=>:error} 

不过，我想删除我的伐木工人confs，我无法找到任何我CONFIGS的问题。有谁知道是否有办法重新安装logstash。我想我已经搞砸了太多confs，logstash，logstash，转发等

的conf文件：

input { 
    beats { 
    port => 5044 
    ssl => true 
    ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt" 
    ssl_key => "/etc/pki/tls/private/logstash-forwarder.key" 
    } 
} 

filter { 
    if [type] == "syslog" { 
    grok { 
     match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" } 
     add_field => [ "received_at", "%{@timestamp}" ] 
     add_field => [ "received_from", "%{host}" ] 
    } 
    syslog_pri { } 
    date { 
     match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ] 
    } 
    } 
} 

output { 
    elasticsearch { 
    host => ["localhost:9200"] 
    sniffing => true 
    manage_template => false 
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}" 
    document_type => "%{[@metadata][type]}" 
    } 
} 

UPDATE 当我运行一个configtest，我得到这个错误：

sudo service logstash configtest 

/etc/init.d/logstash: 156: /etc/init.d/logstash: /opt/logstash/bin/logstash: not found 

Answer 1

注意logstash作为服务从/etc/logstash/conf.d阅读所有文件，因此，如果您编辑只有一个，这并不意味着它会重新加载并修复行吟诗人r文件。

我不确定你正在编辑什么文件，但是你有很多logstash一起追加的文件。

例如，一个文件有这个

input { 
    tcp { 
    port => 5400 
    type => syslog 
    } 
    udp { 
    port => 5400 
    type => syslog 
    } 
} 

另一个有这个

input{ 
    beats { 
    port => 5044 
    } 
} 

等等......所以，检查所有文件/etc/logstash/conf.d为单独使用logstash --configtest错误，因为错误说。

---

如果你看看你的错误消息的结束......

:reason=>"Expected one of #, if, \", ', } at line 56, column 1 (byte 1278) after ...

output { 
    elasticsearch { 
    hosts => [\"localhost:9200\"] 
    sniffing => true 
    manage_template => false 
    index => \"%{[@metadata][beat]}-%{+YYYY.MM.dd}\" 
    document_type => \"%{[@metadata][type]}\" 
    } 

您似乎缺少output {}一个右括号，或者说上线56坏/缺少的，它认为。

---

我不知道什么是错的configtest，但你并不需要该服务。命令logstash --configtest可以测试你自己的文件。