关闭tomcat服务器时无法加载org.apache.hadoop.util.ShutdownHookManager

Question

我有一个运行在tomcat上的简单Web应用程序。此Web应用程序读取和写入文件到HDFS。

我面临的问题是每当我通过使用./bin/shutdown.sh停止服务器。我得到无法加载hadoop的shutdownHookManager异常。

我确定hadoop-common（包含ShutDownManager）在tomcat classpath中。

任何人都可以帮我吗？

例外，我得到了：

Oct 14, 2013 5:57:54 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks 
SEVERE: The web application [] created a ThreadLocal with key of type [com.ebay.kernel.calwrapper.CalTransactionHelper$1] (value [com.ebay.kernel.calwrapper.CalTransactionHelper$1@3d5a1c5f]) and a value of type [com.ebay.kernel.calwrapper.CalTransactionHelper.Stack] (value [[]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak. 
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol stop 
INFO: Stopping ProtocolHandler ["http-bio-8080"] 
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol stop 
INFO: Stopping ProtocolHandler ["ajp-bio-8009"] 
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol destroy 
INFO: Destroying ProtocolHandler ["http-bio-8080"] 
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol destroy 
INFO: Destroying ProtocolHandler ["ajp-bio-8009"] 
Oct 14, 2013 5:57:54 PM org.apache.catalina.loader.WebappClassLoader loadClass 
INFO: Illegal access: this web application instance has been stopped already. Could not load org.apache.hadoop.util.ShutdownHookManager$2. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact. 
java.lang.IllegalStateException 
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1600) 
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559) 
    at org.apache.hadoop.util.ShutdownHookManager.getShutdownHooksInOrder(ShutdownHookManager.java:124) 
    at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:52) 
Exception in thread "Thread-9" java.lang.NoClassDefFoundError: org/apache/hadoop/util/ShutdownHookManager$2 
    at org.apache.hadoop.util.ShutdownHookManager.getShutdownHooksInOrder(ShutdownHookManager.java:124) 
    at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:52) 
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.ShutdownHookManager$2 
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1714) 
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559) 
    ... 2 more 

Answer 1

我有一个Maven的Mojo我建立了同样的问题。我认为这个问题是由在您的Web应用程序中加载hadoop类的上下文类加载器引起的。 WebAppClassLoader类加载器不可见这些定义。当发生JVM关闭时，关闭钩子正在执行，并调用ShutdownHookManager，但它被加载到现在销毁的上下文类加载器中，并且出现问题。

把hadoop jar放在tomcat的lib目录下可能会有帮助。或者在启动tomcat时设置一些* _CLASSPATH java选项。这样WebAppClassLoader就会定义你的类。

否则，您可以在销毁上下文之前手动执行hadoop的关闭挂钩并忽略上述异常。