我遇到了logstash 5.2.2过滤器的问题。我需要将字符串转换为整数并且失败。我正在采取的做法可能并不理想。以下是问题陈述: JMX MBean值将“metric_value_string”报告为“XX.X秒”。我希望能够使用此值作为可视化中Kibana聚合的数字。Logstash 5.2.2 - 修改值的问题 - 将字符串的添加字段转换为整数
我已经试了一下:
定义: 源字段是 “metric_value_string”。一个字符串。值看起来像“26.0秒”。 期望的目标字段是“time_in_seconds”。一个整数。
我试着创建一个脚本字段。简单地(无痛)将“metric_value_string”转换为看起来像Kibana发现结果中的数字的数字,可用于在可视化中作为数字进行聚合。当可视化运行时,会发生错误。这是一个强制性异常,它显示“metric_value_string”中的值,而不是我转换后的“time_in_seconds”。这是Kibana 5.2.2。 IDK如果这是一个错误,所以我尝试了另一种方法。
我试着在logstash过滤器中创建并转换一个字段。
我已经试过这两种方法:
filter {
if "TimeSince" in [metric_path] or "Delay" in [metric_path] {
mutate { add_field => { "time_in_seconds" => "%{metric_value_string}"} }
mutate { gsub => ["time_in_seconds", ".0 secs", ""] }
mutate { convert => { "time_in_seconds", "integer" } }
}
}
和
filter {
if "TimeSince" in [metric_path] or "Delay" in [metric_path] {
ruby {
code =>
"event.set('time_in_seconds', event.get('metric_value_string'))"
}
mutate { gsub => ["time_in_seconds", ".0 secs", ""] }
mutate { convert => { "time_in_seconds", "integer" } }
}
}
有条件的东西就可以了。当我注释掉
mutate { convert => { "time_in_seconds", "integer" } }
代码时,它在输出中看起来和预期的一样,并且不会失败。
我不确定是什么原因导致失败。错误说这是一个语法错误,但如果我只是注释掉转换行,那么一切都可以。作为完整性检查,我在gsub行后添加了更多代码,以确保该代码没有问题。
这里的STDOUT从logstash:
C:\Elastic\logstash-5.2.2\bin>cls
C:\Elastic\logstash-5.2.2\bin>logstash -f config/logstash.conf --config.reload.automatic
JAVA_OPTS was set to [ -Dlog4j.configurationFile=C:\Elastic\logstash-5.2.2\config\log4j2.properties -Xmx1g -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="C:\Elastic\logstash-5.2.2/heapdump.hprof"]. Logstash will trust these options, and not set any defaults that it might usually set
[2017-09-14T14:14:14,056][ERROR][logstash.agent ] Cannot load an invalid configuration {:reason=>"Expected one of #, => at line 14, column 42 (byte 355) after filter {\n\tif \"TimeSince\" in [metric_path] or \"Delay\" in [metric_path] {\n\t\tmutate { add_field => { \"time_in_seconds\" => \"%{metric_value_string}\"} }\n\t\tmutate { gsub => [\"time_in_seconds\", \".0 secs\", \"\"] }\n\t\tmutate { convert => { \"time_in_seconds\""}
这里的配置文件的内容从错误上面:
input {
jmx {
path => "plugins/jmx"
polling_frequency => 60
type => "jmx"
nb_thread => 4
}
}
filter {
if "TimeSince" in [metric_path] or "Delay" in [metric_path] {
mutate { add_field => { "time_in_seconds" => "%{metric_value_string}"} }
mutate { gsub => ["time_in_seconds", ".0 secs", ""] }
mutate { convert => { "time_in_seconds", "integer" } }
}
}
output {
stdout { codec => rubydebug }
}
没有问题,如果我注释掉转换线..
上午我以错误的方式去解决这个问题?我更新这个堆栈。如果这是C#/ SQL,我只是做一个替换+转换/转换。我是否在错误的地方做这件事?
编辑:
这里的时候我注释掉转换线,所以你可以看到实际数据的标准输出:上CONVERT
C:\Elastic\logstash-5.2.2\bin>cls
C:\Elastic\logstash-5.2.2\bin>logstash -f config/logstash.conf --config.reload.automatic
JAVA_OPTS was set to [ -Dlog4j.configurationFile=C:\Elastic\logstash-5.2.2\config\log4j2.properties -Xmx1g -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="C:\Elastic\logstash-5.2.2/heapdump.hprof"]. Logstash will trust these options, and not set any defaults that it might usually set
[2017-09-14T14:23:13,456][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}
[2017-09-14T14:23:13,568][INFO ][logstash.inputs.jmx ] Create queue dispatching JMX requests to threads
[2017-09-14T14:23:13,573][INFO ][logstash.inputs.jmx ] Compile regexp for group alias object replacement
[2017-09-14T14:23:13,574][INFO ][logstash.pipeline ] Pipeline main started
[2017-09-14T14:23:13,576][INFO ][logstash.inputs.jmx ] Initialize 4 threads for JMX metrics collection
[2017-09-14T14:23:13,648][INFO ][logstash.inputs.jmx ] Loading configuration files in path {:path=>"plugins/jmx"}
[2017-09-14T14:23:13,743][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
{
"path" => "plugins/jmx",
"environment" => "TEST",
"@timestamp" => 2017-09-14T18:23:14.154Z,
"@version" => "1",
"host" => "MY HOST",
"metric_path" => "xxStatus",
"type" => "jmx",
"metric_value_string" => "idle"
}
{
"path" => "plugins/jmx",
"time_in_seconds" => "191",
"environment" => "TEST",
"@timestamp" => 2017-09-14T18:23:14.200Z,
"@version" => "1",
"host" => "MY HOST",
"metric_path" => "xxTimeSincexx",
"type" => "jmx",
"metric_value_string" => "191.0 secs"
}
跨贴到https的文档@https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-convert
不正确的://discuss.elastic .CO /吨/ logstash-5-2-2-发生变异字符串值到数/ 100877 –