可以logstash同时处理多个输出吗？

Question

我很新的logstash和弹性搜索。我试图在elasticsearch和平面文件中存储日志文件。我知道logstash支持这两种输出。但他们是同时处理？或者是通过工作定期完成的？

Answer 1

是的，您可以通过在托运人配置中使用“add_tag”命令标记和克隆您的输入来完成此操作。

input 
{ 
    tcp  { type => "linux" port => "50000" codec => plain { charset => "US-ASCII" } } 
    tcp  { type => "apache_access" port => "50001" codec => plain { charset => "US-ASCII" } } 
    tcp  { type => "apache_error" port => "50002" codec => plain { charset => "US-ASCII" } } 
    tcp  { type => "windows_security" port => "50003" codec => plain { charset => "US-ASCII" } } 
    tcp  { type => "windows_application" port => "50004" codec => plain { charset => "US-ASCII" } } 
    tcp  { type => "windows_system" port => "50005" codec => plain { charset => "US-ASCII" } } 
udp { type => "network_equipment" port => "514" codec => plain { charset => "US-ASCII" } } 
udp { type => "firewalls" port => "50006" codec => plain } 
} 
filter 
{ 
    grok { match => [ "host", "%{IPORHOST:ipaddr}(:%{NUMBER})?" ] } 
    mutate { replace => [ "fqdn", "%{ipaddr}" ] } 
    dns  { reverse => [ "fqdn", "fqdn" ] action => "replace" } 
    if [type] == "linux"     { clone { clones => "linux.log" add_tag => "savetofile" } } 
    if [type] == "apache_access"   { clone { clones => "apache_access.log" add_tag => "savetofile" } } 
    if [type] == "apache_error"    { clone { clones => "apache_error.log" add_tag => "savetofile" } } 
    if [type] == "windows_security"   { clone { clones => "windows_security.log" add_tag => "savetofile" } } 
    if [type] == "windows_application"  { clone { clones => "windows_application.log" add_tag => "savetofile" } } 
    if [type] == "windows_system"   { clone { clones => "windows_system.log" add_tag => "savetofile" } } 
    if [type] == "network_equipment"  { clone { clones => "network_%{fqdn}.log" add_tag => "savetofile" } } 
if [type] == "firewalls"  { clone { clones => "firewalls.log" add_tag => "savetofile" } } 
} 
output 
{ 
    #stdout { debug => true } 
    #stdout { codec => rubydebug } 
    redis { host => "1.1.1.1" data_type => "list" key => "logstash" } 
} 

并在您的主logstash比如你可以这样做：

input { 
    redis { 
    host => "1.1.1.1" 
    data_type => "list" 
    key => "logstash" 
    type=> "redis-input" 
    # We use the 'json' codec here because we expect to read json events from redis. 
    codec => json 
      } 
    } 
    output 
    { 
     if "savetofile" in [tags] { 
      file { 
       path => [ "/logs/%{fqdn}/%{type}" ] message_format => "%{message}" 
      } 
     } 
     else { elasticsearch { host => "2.2.2.2" } 
    } 
} 

Answer 2

仅供参考，你可以学习The life of logstash event有关logstash事件。

输出工作者模型当前是单个线程。输出将按照它们在配置文件中定义的顺序接收事件。

但是输出可能决定在发布它们之前临时缓冲事件。例如：输出缓冲2或3个事件，然后只写入文件。