Logary与Logstash到弹性搜索 - 简单的配置

Question

我一直在试图建立日志记录使用Logary> Logstash> ElasticSearch几乎整个一天，我的.NET应用程序。我已经尝试了我能想到的所有配置变化，并通过了几个小时的文档无济于事。

这里有什么问题？

var logary = LogaryFactory.New("Logary Specs", with => with 

       .Target<Logstash.Builder>("ls", conf => conf.Target 
         .Hostname("localhost") 
         .Port(1939) 
         .EventVersion(Logstash.EventVersion.One) 
         .Done()) 

       .Target<Debugger.Builder>("debugger") 
      ); 

这里的logstash.conf：

input { 
    stdin {} 

    tcp { 
     codec => json_lines { charset => "UTF-8" } 
     host => localhost 
     port => 1939 
     type => "app" 
    } 
} 

output { 

    stdout {} 
    elasticsearch { 
     host => localhost 
     protocol => "http" 
    } 
} 

我在文档不output.elasticsearch.protocol试图与主机=> “0.0.0.0” 等，中间的每一个组合。

我有ES了对9200和Kibana装起来，我可以看到日志显示了通过标准输入/输出但不使用Logary我的应用程序日志。它也验证了Logary调试器目标的工作原理。

出了什么问题我在这里的管道？

Logstash 1.4.2

ES 1.3.4

Answer 1

您需要一些组件来调试它。

我写了一个示例程序供您用来调试它;这是可以工作的最简单的东西：

open System 

open Logary 
open Logary.Configuration 
open Logary.Targets 

[<EntryPoint>] 
let main argv = 
    use logary = 
    withLogary' "logstash-sample" (
     withTargets [ 
     Logstash.create (Logstash.LogstashConf.Create "10.0.0.120") "logstash" 
     Debugger.create Debugger.empty "debugger" 
     ] >> 
     withRules [ 
     Rule.createForTarget "logstash" 
     Rule.createForTarget "debugger" 
     ] >> 
     withInternalTargets Verbose [ 
     Debugger.create (Debugger.empty) "console" 
     ] 
    ) 

    let logger = logary.GetLogger "logstash-sample" 
    Logger.debug logger "Hello World" 

    Console.ReadKey true |> ignore 
    0 

这是它的预期调试器输出（不是控制台）。

'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled. 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'Y:\dev\ip\_scratch\logstash-sample\bin\Debug\logstash_sample.exe', Symbols loaded. 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'Y:\dev\ip\_scratch\logstash-sample\bin\Debug\Logary.dll', Symbols loaded. 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'Y:\dev\ip\_scratch\logstash-sample\bin\Debug\NodaTime.dll' 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'C:\Windows\Microsoft.Net\assembly\GAC_MSIL\FSharp.Core\v4.0_4.3.0.0__b03f5f7f11d50a3a\FSharp.Core.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled. 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled. 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled. 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Xml\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled. 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'Y:\dev\ip\_scratch\logstash-sample\bin\Debug\Newtonsoft.Json.dll' 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'Y:\dev\ip\_scratch\logstash-sample\bin\Debug\Intelliplan.JsonNet.dll' 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'Y:\dev\ip\_scratch\logstash-sample\bin\Debug\Intelliplan.JsonNet.NodaTime.dll' 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'Y:\dev\ip\_scratch\logstash-sample\bin\Debug\FSharp.Actor.dll' 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Configuration\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Configuration.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled. 
D 2014-10-18T10:25:42.4192736+00:00: supervisor status: Running [Logary.Registry.runRegistry] 
I 2014-10-18T10:25:42.4192736+00:00: initing logstash target, connecting to 10.0.0.120:1936... [Logary.Targets.Logstash.logstashLoop] 
D 2014-10-18T10:25:42.4383230+00:00: scheduler status: Running [Logary.Registry.runRegistry] 
D 2014-10-18T10:25:42.4473276+00:00: registry status: Running [Logary.Registry.runRegistry] 
D 2014-10-18T10:25:42.4977127+00:00: will poll metrics every 0:00:00:00.5 [Logary.Registry.registry] 
D 2014-10-18T10:25:42.5335455+00:00: Hello World [logstash-sample] 
I 2014-10-18T10:25:42.5515369+00:00: initing logstash target, connected [Logary.Targets.Logstash.logstashLoop] 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Numerics\v4.0_4.0.0.0__b77a5c561934e089\System.Numerics.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled. 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.ComponentModel.DataAnnotations\v4.0_4.0.0.0__31bf3856ad364e35\System.ComponentModel.DataAnnotations.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled. 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Runtime.Serialization\v4.0_4.0.0.0__b77a5c561934e089\System.Runtime.Serialization.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled. 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Xml.Linq\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.Linq.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled. 
'logstash_sample.exe' (Managed (v4.0.30319)): Loaded 'C:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled. 
D 2014-10-18T10:25:42.7268749+00:00: polling metrics [Logary.Registry.registry] 
D 2014-10-18T10:25:42.8571421+00:00: send-recv stream not open, opening [Logary.Targets.Logstash.logstashLoop] 
I 2014-10-18T10:25:43.1096339+00:00: start shutdown [Logary.Configuration.Config.shutdown] 
I 2014-10-18T10:25:43.1186538+00:00: flush start [Logary.Registry.registry] 
I 2014-10-18T10:25:43.1667422+00:00: flush Ack [Logary.Registry.registry] 
I 2014-10-18T10:25:43.1787861+00:00: shutdown metrics polling [Logary.Registry.registry] 
I 2014-10-18T10:25:43.1798152+00:00: shutdown schedules [Logary.Registry.registry] 
I 2014-10-18T10:25:43.1798152+00:00: shutdown targets [Logary.Registry.registry] 
I 2014-10-18T10:25:43.2018104+00:00: shutdown Ack [Logary.Registry.registry] 
I 2014-10-18T10:25:43.2018104+00:00: shutting down immediately [Logary.Registry.registry] 
I 2014-10-18T10:25:43.2118258+00:00: stop shutdown [Logary.Configuration.Config.shutdown] 
The program '[4820] logstash_sample.exe: Managed (v4.0.30319)' has exited with code 0 (0x0). 

此代码是在这里：

https://github.com/logary/logary/commit/0dca450677dc1ca691292d379a734c7051743c1c

你可以做Install-Package Intelliplan.Logary -Pre，然后复制在Logary.dll -dll文件。确保你有相同的输出。

如果您可以根据上述输出成功连接到logstash，则可以通过打印到其日志来验证它是否接收事件或失败;

tail -f /var/log/logstash/logstash-agent.log 

Answer 2

有一对夫妇的事情，你可以试试。首先仔细检查您的elasticsearch集群是否实际上是绿色/黄色。我浪费了好几个小时，因为我在配置的elasticsearch索引模板中指定了更多的副本。

当然检查，对任何错误elasticsearch和logstash日志。

接下来，请尝试使用

elasticsearch_http { host => "localhost" }

此使用HTTP协议，而不是内部协议，并释放你不必有相同的弹性搜索版本与logstash捆绑在一起。

最后，如果还是不行，我建议在logstash目录打探和修改，实现插件的Ruby脚本。当它工作的时候它很好，但是当它没有时它的记录有点缺乏;这可以通过一些日志语句轻松解决。