2017-04-09 62 views
0

我想将logstash输出发送到csv,但列没有写入文件中。无法输出从logstash到csv的字段

这是我的logstash配置:

input 
{ 
    http 
    { 
    host => "0.0.0.0" 
    port => 31311 
    } 
} 

filter 
{ 

    grok { 
      match => { "id" => "%{URIPARAM:id}?" } 
     } 

    kv 
    { 
     field_split => "&?" 
     source => "[headers][request_uri]" 
    } 
} 

output 
{ 

    stdout { codec => rubydebug } 

    csv 
    { 
     fields => ["de,cd,dl,message,bn,ua"] 
     path => "/tmp/logstash-bq/text.csv" 
     flush_interval => 0 
     csv_options => {"col_sep" => ";" "row_sep" => "\r\n"} 
    } 


} 

这是我输入:

curl -X POST 'http://localhost:31311/?id=9decaf95-20a5-428e-a3ca-50485edb9f9f&uid=1-fg4fuqed-j0hzl5q2&ev=pageview&ed=&v=1&dl=http://dev.xxx.com.br/&rl=http://dev.xxxx.com.br/&ts=1491758180677&de=UTF-8&sr=1600x900... 

这是logstash答案:

{ 
     "headers" => { 
      "http_accept" => "*/*", 
      "request_path" => "/", 
      "http_version" => "HTTP/1.1", 
     "request_method" => "POST", 
       "http_host" => "localhost:31311", 
      "request_uri" => "/?id=xxx...", 
     "http_user_agent" => "curl/7.47.1" 
    }, 
      "de" => "UTF-8", 
      "cd" => "24", 
      "dl" => "http://dev.xxx.com.br/", 
     "message" => "", 
      "bn" => "Chrome%2057", 
      "ua" => "Mozilla/5.0%20(Macintosh;%20Intel%20Mac%20OS%20X%2010_11_3)%20AppleWebKit/537.36%20(KHTML,%20like%20Gecko)%20Chrome/57.0.2987.133%20Safari/537.36", 
      "dt" => "xxxx", 
      "uid" => "1-fg4fuqed-j0hzl5q2", 
      "ev" => "pageview", 
    "@timestamp" => 2017-04-09T17:41:03.083Z, 
      "v" => "1", 
      "md" => "false", 
     "@version" => "1", 
      "host" => "0:0:0:0:0:0:0:1", 
      "rl" => "http://dev.xxx.com.br/", 
      "vp" => "1600x236", 
      "id" => "9decaf95-20a5-428e-a3ca-50485edb9f9f", 
      "ts" => "1491758180677", 
      "sr" => "1600x900" 
} 
[2017-04-09T14:41:03,137][INFO ][logstash.outputs.csv  ] Opening file {:path=>"/tmp/logstash-bq/text.csv"} 

但是当我打开的/ tmp/logstash- bq/text.csv我看到这个:

2017-04-09T16:26:17.464Z 127.0.0.1 abc2017-04-09T17:19:19.690Z 0:0:0:0:0:0:0:1 2017-04-09T17:23:12.117Z 0:0:0:0:0:0:0:1 2017-04-09T17:24:08.067Z 0:0:0:0:0:0:0:1 2017-04-09T17:31:39.269Z 0:0:0:0: 0:0:0:1 2017-04-09T17:38:02.624Z 0:0:0:0:0:0:0:1 2017-04-09T17:41:03.083Z 0:0:0:0:0:0:0:1 
+0

嗨。我认为你应该以这种格式定义字段:'fields => [“de”,“cd”,“dl”,“message”,“bn”,“ua”]' – Kadir

回答

0

对于logstash 5.x,CSV输出存在错误。我不得不安装logstash 2.4.1。