2016-02-04 73 views
6

我使用的Apache Spark的spark-shell与mesos和zookeeper,这似乎工作得很好,除了我从zookeeper得到更多的日志记录比我想,这是一个有点分散注意力:本地更改动物园管理员C客户端的日志级别

Welcome to 
     ____    __ 
    /__/__ ___ _____/ /__ 
    _\ \/ _ \/ _ `/ __/ '_/ 
    /___/ .__/\_,_/_/ /_/\_\ version 1.5.0 
     /_/ 

Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_60) 
Type in expressions to have them evaluated. 
Type :help for more information. 
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):[email protected][email protected]: Client environment:zookeeper.version=zookeeper C client 3.4.6 
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):[email protected][email protected]: Client environment:host.name=myhost 
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):[email protected][email protected]: Client environment:os.name=Linux 
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):[email protected][email protected]: Client environment:os.arch=3.2.0-34-generic 
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):[email protected][email protected]: Client environment:os.version=#53-Ubuntu SMP Thu Nov 15 10:48:16 UTC 2012 
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):[email protected][email protected]: Client environment:user.name=myname 
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):[email protected][email protected]: Initiating client connection, host=localhost:9999 sessionTimeout=10000 watcher=0xffffffffff sessionId=0 sessionPasswd=<null> context=0x777777777777 flags=0 
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):[email protected][email protected]: initiated connection to server [127.0.0.1:9999] 
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):[email protected][email protected]: session establishment complete on server [127.0.0.1:9999], sessionId=0x11111111111111, negotiated timeout=10000 
Spark context available as sc. 
SQL context available as sqlContext. 

scala> 

我试图改变$SPARK_CONF_DIR/log4j.properties

log4j.logger.org.apache.zookeeper=WARN 

,没有工作,但我并不感到惊讶,因为在mesos documentation mentions that the zookeeper logging goes directly to stderr

注意:第三方日志消息(例如, ZooKeeper)只写入stderr!

加入广口瓶中,引发的是具有与外国功能接口调用zookeeper C api function to change the log level静态块好像它可能工作classpath中,但我不知道,我没有与调用任何经验来自JVM的C代码。

回答

1

除,

log4j.logger.org.apache.zookeeper=WARN 

配置stderr日志级别仅打印优先ERROR(及以上)日志。

# Send WARN or higher to stderr 
log4j.appender.stderr = org.apache.log4j.ConsoleAppender 
log4j.appender.stderr.Threshold = ERROR 
log4j.appender.stderr.Target = System.err 

另外一个LevelRangeFilter也可以如图here证明是有用的。

+0

不,这对动物园管理员的'ZOO_INFO'日志没有任何影响,这些日志仍然在stderr上吐出来。 – rampion

1

在这个上找不到任何东西,但对于JNI(Java-to-C)可以提供this。看起来没有zookeeper没有mesos有网络服务或shell(zookeeper有zkshell虽然,但在我们的情况下不可用),可以使用。