2017-02-09 90 views
1

我发现一个奇怪的behaviore其中REST POST线程有时挂起或卡死,并保持在RUNNABLE状态永远在方法阳光下。 nio.ch.PollArrayWrapper.poll0(本机方法)Java的REST线程都处于RUNNABLE状态永远在我们的生产服务器wildfly 10.0

几天后,野蛮应用服务器变得无响应并且消耗高CPU。

这里是打印屏幕构成了我们javamelody监控,显示一些在RUNNABLE状态挂线程的时间超过50小时(和计数)。

enter image description here

要确保我花了一些jstack跟踪文件,我发现了同样的线程仍然thier和RUNNABLE状态。

默认任务-302线程堆栈(从线程转储文件)如下..

"default task-302" #1054 prio=5 os_prio=0 tid=0x00007fd55c19b800 nid=0x666b runnable [0x00007fd5161a6000] 
    java.lang.Thread.State: RUNNABLE 
    at sun.nio.ch.PollArrayWrapper.poll0(Native Method) 
    at sun.nio.ch.PollArrayWrapper.poll(PollArrayWrapper.java:115) 
    at sun.nio.ch.PollSelectorImpl.doSelect(PollSelectorImpl.java:87) 
    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) 
    - locked <0x0000000612c0f7e0> (a sun.nio.ch.Util$3) 
    - locked <0x0000000612c0f7d0> (a java.util.Collections$UnmodifiableSet) 
    - locked <0x0000000612c07e10> (a sun.nio.ch.PollSelectorImpl) 
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) 
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) 
    at org.xnio.nio.SelectorUtils.await(SelectorUtils.java:46) 
    at org.xnio.nio.NioSocketConduit.awaitReadable(NioSocketConduit.java:345) 
    at io.undertow.protocols.ssl.SslConduit.awaitReadable(SslConduit.java:281) 
    at org.xnio.conduits.AbstractSourceConduit.awaitReadable(AbstractSourceConduit.java:66) 
    at org.xnio.conduits.AbstractSourceConduit.awaitReadable(AbstractSourceConduit.java:66) 
    at io.undertow.conduits.ReadDataStreamSourceConduit.awaitReadable(ReadDataStreamSourceConduit.java:101) 
    at io.undertow.conduits.FixedLengthStreamSourceConduit.awaitReadable(FixedLengthStreamSourceConduit.java:285) 
    at org.xnio.conduits.ConduitStreamSourceChannel.awaitReadable(ConduitStreamSourceChannel.java:151) 
    at io.undertow.channels.DetachableStreamSourceChannel.awaitReadable(DetachableStreamSourceChannel.java:77) 
    at io.undertow.server.HttpServerExchange$ReadDispatchChannel.awaitReadable(HttpServerExchange.java:2092) 
    at org.xnio.channels.Channels.readBlocking(Channels.java:295) 
    at io.undertow.servlet.spec.ServletInputStreamImpl.readIntoBuffer(ServletInputStreamImpl.java:170) 
    at io.undertow.servlet.spec.ServletInputStreamImpl.read(ServletInputStreamImpl.java:146) 
    at org.jboss.resteasy.plugins.providers.ProviderHelper.readString(ProviderHelper.java:66) 
    at org.jboss.resteasy.plugins.providers.StringTextStar.readFrom(StringTextStar.java:40) 
    at org.jboss.resteasy.plugins.providers.StringTextStar.readFrom(StringTextStar.java:22) 
    at org.jboss.resteasy.core.interception.AbstractReaderInterceptorContext.readFrom(AbstractReaderInterceptorContext.java:61) 
    at org.jboss.resteasy.core.interception.ServerReaderInterceptorContext.readFrom(ServerReaderInterceptorContext.java:60) 
    at org.jboss.resteasy.core.interception.AbstractReaderInterceptorContext.proceed(AbstractReaderInterceptorContext.java:53) 
    at org.jboss.resteasy.security.doseta.DigitalVerificationInterceptor.aroundReadFrom(DigitalVerificationInterceptor.java:34) 
    at org.jboss.resteasy.core.interception.AbstractReaderInterceptorContext.proceed(AbstractReaderInterceptorContext.java:55) 
    at org.jboss.resteasy.plugins.interceptors.encoding.GZIPDecodingInterceptor.aroundReadFrom(GZIPDecodingInterceptor.java:59) 
    at org.jboss.resteasy.core.interception.AbstractReaderInterceptorContext.proceed(AbstractReaderInterceptorContext.java:55) 
    at org.jboss.resteasy.core.MessageBodyParameterInjector.inject(MessageBodyParameterInjector.java:151) 
    at org.jboss.resteasy.core.MethodInjectorImpl.injectArguments(MethodInjectorImpl.java:91) 
    at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:114) 
    at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) 
    at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) 
    at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) 
    at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:395) 
    at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202) 
    at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) 
    at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) 
    at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) 
    at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) 
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129) 
    at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:201) 
    at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:178) 
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) 
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) 
    at com.unilever.merchandizer.views.filters.GzipResponseFilter.doFilter(GzipResponseFilter.java:176) 
    at org.omnifaces.filter.HttpFilter.doFilter(HttpFilter.java:108) 
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) 
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) 
    at com.unilever.merchandizer.views.filters.AuthorizationFilter.doFilter(AuthorizationFilter.java:33) 
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) 
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) 
    at com.unilever.merchandizer.views.filters.AuthenticationFilter.doFilter(AuthenticationFilter.java:40) 
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60) 
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) 
    at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) 
    at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) 
    at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) 
    at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) 
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) 
    at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) 
    at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) 
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) 
    at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:51) 
    at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) 
    at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) 
    at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:56) 
    at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) 
    at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) 
    at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) 
    at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) 
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) 
    at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) 
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) 
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) 
    at io.undertow.server.handlers.MetricsHandler.handleRequest(MetricsHandler.java:62) 
    at io.undertow.servlet.core.MetricsChainHandler.handleRequest(MetricsChainHandler.java:59) 
    at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284) 
    at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263) 
    at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) 
    at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174) 
    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) 
    at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
    at java.lang.Thread.run(Thread.java:745) 

任何想法或暗示有关该问题的根本原因? 它是wild or还是隐藏问题? 是JDK 8的错误吗?什么在我的应用程序代码,以检查

环境:

  • JDK:jdk1.8.0_112
  • 操作系统:CentOS的7
  • Wildfly 10.0(与1.3.15版本的暗流最终)
  • 的Java EE 7的应用程序
+0

堆栈会告诉我你的服务器在某些资源上等待再次变得可读(来自其中的所有'awaitReadable'方法)。先试着看看你的应用程序中是否有资源泄漏。例如,如果您不关闭“Connection”或“InputStream/OutputStream”或“Reader/Writer”。 –

+0

不幸的是,我在我的简单REST POST方法中没有发现任何泄漏 –

回答

0

这个问题的根本原因是由于使用改装库与邻互动的机器人活动你的网络服务。但在某些情况下,它会打开与服务器的连接,因此服务器线程处于可运行状态。

要从服务器端解决这个问题,我们在wildfly的HTTP和HTTPS模块中添加了超时设置。这个行动解决了这个案子。

相关问题