在Logstash中分解json数组字符串

Question

我有一个RabbitMQ以JSON格式发送3个字段，它正在被Logstash rabbitmq输入插件使用。

其中一个字段是一个字符串JSON数组如下：

"content": [ 
    "1111", 
    "2222222", 
    "Test 06", 
    "3", 
    "3232", 
    "SomeValue1" 
] 

1. 我怎么能有这样的字符串每次进入一个领域，这样我可以快速发现并从Kibana可视化可用的字段？现在我看到那个完整字符串的“内容”。

2. JSON数组字符串大小根据另一个字段eventID而变化。根据eventID，是否可以将该字符串中的值动态映射到特定名称？ 如：

   "eventID": 1, 
       "content": [ 
       "name1": "1111", 
       "name2": "2222222", 
       "name3": "Test 06", 
       "name4": "3", 
       "name5": "3232", 
       "name6": "SomeValue1" 
   ] 
   
   "eventID": 2, 
       "content": [ 
       "othername1": "3434", 
       "othername2": "Test 10", 
       "othername3": "876", 
       "othername4": "Some String7" 
   ] 
   

我想有名字*和*英文别名在可用字段。 任何帮助，将不胜感激。

Answer 1

首先，我将假设您的输入已经被正确解析为一个数组了。出于测试目的，这意味着这样的：

echo '{"eventID":1,"content":["a","b","c","d"]}' | bin/logstash -f test.conf 

其中test.conf是：

input { 
    stdin { codec => json } 
} 
output { 
    stdout { codec => rubydebug } 
} 

将输出这样的：

{ 
     "eventID" => 1, 
    "@timestamp" => 2017-08-03T19:39:13.054Z, 
     "@version" => "1", 
      "host" => "xxxxxx.local", 
     "content" => [ 
     [0] "a", 
     [1] "b", 
     [2] "c", 
     [3] "d" 
    ] 
} 

如果是这样的话，那么你就需要做这样的事情：

filter { 
    if [eventID] == 1 { 
     mutate { 
      add_field => { 
       "eventName" => "type1 event" 
       "one0" => "%{[content][0]}" 
       "one1" => "%{[content][1]}" 
       "one2" => "%{[content][2]}" 
       "one3" => "%{[content][3]}" 
      } 
      remove_field => [ "content" ] 
     } 
    } else if [eventID] == 2 { 
     mutate { 
      add_field => { 
       "eventName" => "type2 event" 
       "two0" => "%{[content][0]}" 
       "two1" => "%{[content][1]}" 
       "two2" => "%{[content][2]}" 
       "two3" => "%{[content][3]}" 
      } 
      remove_field => [ "content" ] 
     } 
    } 
} 

which将生成如下事件：

{ 
     "eventID" => 1, 
    "@timestamp" => 2017-08-03T19:51:02.946Z, 
     "@version" => "1", 
      "host" => "xxxxxxx.local", 
      "one2" => "c", 
    "eventName" => "type1 event", 
      "one3" => "d", 
      "one0" => "a", 
      "one1" => "b" 
}