2016-08-24 90 views
1

我在一堆Linux服务器上设置自动Cron作业,这需要调用IBM WebSphere Application Server提供的一些PUT/POST API (8.5.5)。我用Java编写了自己的REST客户端,但奇怪的是,有时API执行会挂起下面的异常。当有效负载超过特定大小时,Java REST API调用失败,异常“异步操作超时”

com.ibm.ws.webcontainer.webapp.WebApp logServletError SRVE0293E: [Servlet Error]-[JAX-RS Servlet]: java.net.SocketTimeoutException: Async operation timed out 
    at com.ibm.ws.tcp.channel.impl.AioTCPReadRequestContextImpl.processSyncReadRequest(AioTCPReadRequestContextImpl.java:191) 
    at com.ibm.ws.tcp.channel.impl.TCPReadRequestContextImpl.read(TCPReadRequestContextImpl.java:111) 
    at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.fillABuffer(HttpServiceContextImpl.java:4212) 
    at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.readSingleBlock(HttpServiceContextImpl.java:3440) 
    at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.readBodyBuffer(HttpServiceContextImpl.java:3546) 
    at com.ibm.ws.http.channel.inbound.impl.HttpInboundServiceContextImpl.getRequestBodyBuffer(HttpInboundServiceContextImpl.java:1822) 
    at com.ibm.ws.webcontainer.channel.WCCByteBufferInputStream.bufferIsGood(WCCByteBufferInputStream.java:371) 
    at com.ibm.ws.webcontainer.channel.WCCByteBufferInputStream.read(WCCByteBufferInputStream.java:265) 
    at com.ibm.ws.webcontainer.srt.http.HttpInputStream.read(HttpInputStream.java:322) 
    at sun.nio.cs.StreamDecoder$CharsetSD.readBytes(StreamDecoder.java:464) 
    at sun.nio.cs.StreamDecoder$CharsetSD.implRead(StreamDecoder.java:506) 
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:234) 
    at java.io.InputStreamReader.read(InputStreamReader.java:188) 
    at com.ibm.json.java.JSONArray.parse(JSONArray.java:150) 
    at com.ibm.websphere.jaxrs.providers.json4j.JSON4JArrayProvider.readFrom(JSON4JArrayProvider.java:44) 
    at com.ibm.websphere.jaxrs.providers.json4j.JSON4JArrayProvider.readFrom(JSON4JArrayProvider.java:25) 
    at org.apache.wink.server.internal.registry.ServerInjectableFactory$EntityParam.getValue(ServerInjectableFactory.java:197) 
    at org.apache.wink.common.internal.registry.InjectableFactory.instantiate(InjectableFactory.java:68) 
    at org.apache.wink.server.internal.handlers.CreateInvocationParametersHandler.handleRequest(CreateInvocationParametersHandler.java:44) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22) 
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:75) 
    at org.apache.wink.server.internal.handlers.FindResourceMethodHandler.handleSubResourceMethod(FindResourceMethodHandler.java:188) 
    at org.apache.wink.server.internal.handlers.FindResourceMethodHandler.handleRequest(FindResourceMethodHandler.java:110) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22) 
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:75) 
    at org.apache.wink.server.internal.handlers.FindRootResourceHandler.handleRequest(FindRootResourceHandler.java:95) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22) 
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:75) 
    at org.apache.wink.server.internal.handlers.HeadMethodHandler.handleRequest(HeadMethodHandler.java:53) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22) 
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:75) 
    at org.apache.wink.server.internal.handlers.OptionsMethodWADLHandler.handleRequest(OptionsMethodWADLHandler.java:51) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22) 
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:75) 
    at org.apache.wink.server.internal.handlers.SearchResultHandler.handleRequest(SearchResultHandler.java:33) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22) 
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:75) 
    at org.apache.wink.server.internal.log.ResourceInvocation.handleRequest(ResourceInvocation.java:92) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22) 
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:75) 
    at org.apache.wink.server.internal.log.Requests.handleRequest(Requests.java:76) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:26) 
    at org.apache.wink.server.handlers.RequestHandlersChain.handle(RequestHandlersChain.java:22) 
    at org.apache.wink.server.handlers.AbstractHandlersChain.doChain(AbstractHandlersChain.java:75) 
    at org.apache.wink.server.handlers.AbstractHandlersChain.run(AbstractHandlersChain.java:60) 
    at org.apache.wink.server.internal.RequestProcessor.handleRequestWithoutFaultBarrier(RequestProcessor.java:207) 
    at org.apache.wink.server.internal.RequestProcessor.handleRequest(RequestProcessor.java:154) 
    at org.apache.wink.server.internal.servlet.RestServlet.service(RestServlet.java:124) 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:668) 
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1230) 
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:779) 
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:478) 
    at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178) 
    at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1071) 
    at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:87) 
    at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:909) 
    at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1662) 
    at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:200) 
    at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:459) 
    at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:526) 
    at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:312) 
    at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:283) 
    at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214) 
    at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113) 
    at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175) 
    at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217) 
    at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161) 
    at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138) 
    at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204) 
    at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775) 
    at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905) 
    at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1862) 
Caused by: com.ibm.io.async.AsyncTimeoutException(Async operation timed out, [Timeout, rc=0]) 
    at com.ibm.io.async.AbstractAsyncFuture.waitForCompletion(AbstractAsyncFuture.java:359) 
    at com.ibm.io.async.AsyncFuture.getByteCount(AsyncFuture.java:218) 
    at com.ibm.ws.tcp.channel.impl.AioSocketIOChannel.readAIOSync(AioSocketIOChannel.java:215) 
    at com.ibm.ws.tcp.channel.impl.AioTCPReadRequestContextImpl.processSyncReadRequest(AioTCPReadRequestContextImpl.java:184) 
    ... 78 more 

这只发生在一些Linux服务器上,当我在Windows工作站上开发它时它工作得很好。所以在WAS服务器中设置超时应该不成问题。我尝试使用卷曲如下

curl -vX PUT http://<was_server_ip>:9081/RTCWebClient/v0.1/api/testResult/syncTestSuites?testCategory=Kitchen -d @test.json --header "Content-Type: application/json" 

调试,发现当所述有效载荷超过1306个字节(见下文卷曲输出内容长度),该API开始挂并给出上述异常。

> PUT /RTCWebClient/v0.1/api/testResult/syncTestSuites?testCategory=Kitchen 
HTTP/1.1 
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2 
> Host: 9.51.163.190:9081 
> Accept: */* 
> Content-Type: application/json 
> Content-Length: 1307 
> Expect: 100-continue 
> 
< HTTP/1.1 100 Continue 
< Content-Length: 0 
< Date: Wed, 24 Aug 2016 07:35:23 GMT 
< HTTP/1.1 500 Internal Server Error 
< Date: Wed, 24 Aug 2016 07:35:23 GMT 
< X-Powered-By: Servlet/3.0 
< Content-Type: text/html;charset=ISO-8859-1 
< $WSEP: 
< Content-Language: en-US 
< Content-Length: 102 
< Connection: Close 
< 
Error 500: javax.servlet.ServletException: java.net.SocketTimeoutException: Async operation timed out 
* Closing connection #0 

当有效负载小于1307字节时,API得到正确执行。

> PUT /RTCWebClient/v0.1/api/testResult/syncTestSuites?testCategory=Kitchen HTTP/1.1 
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2 
> Host: 9.51.163.190:9081 
> Accept: */* 
> Content-Type: application/json 
> Content-Length: 1306 
> Expect: 100-continue 
> 
< HTTP/1.1 100 Continue 
< Content-Length: 0 
< Date: Wed, 24 Aug 2016 07:37:51 GMT 
< HTTP/1.1 200 OK 
< Date: Wed, 24 Aug 2016 07:37:51 GMT 
< X-Powered-By: Servlet/3.0 
< Content-Type: application/json 
< Access-Control-Allow-Headers: application/json 
< Access-Control-Allow-Methods: GET, POST, PUT, DELETE 
< Access-Control-Allow-Origin: * 
< Content-Language: en-US 
< Transfer-Encoding: chunked 
< 
* Connection #0 to host 9.51.163.190 left intact 
* Closing connection #0 
{"Identifier":"syncTestSuites completed","Result":{"Newly Added Test Suites":[]}} 

我需要与他们的管理员检查在那些有问题的Linux机器上可能导致这种情况的设置。但我也在这里发布,看看是否有人能够就这个问题给出一些深思熟虑的见解。

+0

一些相关的Stack问题尚未解答。 1. [异步操作超时](http://stackoverflow.com/questions/35476406/async-operation-timed-out)2. [请求没有得到正确提交](http://stackoverflow.com/questions/ 36810130/request-is-not-getting-submitted-properly -ww-gettingexception-java-net-sockett) –

+1

我怀疑“异步操作超时”是WebSphere的方式:“我没有得到响应”。但奇怪的是,这取决于消息的大小,它不像你发送兆字节。它可能需要使用pcap或wireshark来查看通信发生了什么以达到此目的。 –

回答

0

这个问题很久以前就解决了,但忘了在这里更新它。

在WAS应用程序服务器的MTU大小从1500位降低到1320之后,问题消失了。根据网络管理员的说法,这使服务器的数据段变小了一些,允许一些应用程序以更好的方式处理数据。

这是一篇关于最大传输单元https://craigocon.wordpress.com/2012/10/03/whats-mtu-and-how-do-i-change-it-in-windows-2008-r2/的非常好的博客文章。

我对网络知识是全新的,因此很高兴能得到我们网络团队的支持。