Apache Nutch 2.3.1远程命令失败

Question

我已经完成了hadoop和hbase的nutch设置。如果我通过命令行（终端）运行作业，如果成功。但是当我想通过nutch wepapp服务器运行相同的命令发生异常时。

2016-09-07 12:25:31,800 ERROR impl.RemoteCommandExecutor - Remote command failed 
java.util.concurrent.TimeoutException 
    at java.util.concurrent.FutureTask.get(FutureTask.java:205) 
    at org.apache.nutch.webui.client.impl.RemoteCommandExecutor.executeRemoteJob(RemoteCommandExecutor.java:61) 
    at org.apache.nutch.webui.client.impl.CrawlingCycle.executeCrawlCycle(CrawlingCycle.java:58) 
    at org.apache.nutch.webui.service.impl.CrawlServiceImpl.startCrawl(CrawlServiceImpl.java:69) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:483) 
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) 
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) 
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) 
    at org.springframework.aop.interceptor.AsyncExecutionInterceptor$1.call(AsyncExecutionInterceptor.java:97) 
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
    at java.lang.Thread.run(Thread.java:745) 
2016-09-07 12:25:31,850 INFO impl.CrawlingCycle - Executed remote command data: INJECT status: FAILED 

我已经为应用服务器即nutchserver和webapp开始拖车服务。我已经以用户模式和root用户身份运行这些服务。但结果相同。

Answer 1

这是超时异常的RemoteCommandExecuter的.java

尝试增加的最大超时等待获得的结果在执行工作。它最多等待计算完成的给定时间，然后检索结果。

private static final int DEFAULT_TIMEOUT_SEC = 60; 

    public JobInfo executeRemoteJob(RemoteCommand command) { 
     try { 
      String jobId = client.executeJob(command.getJobConfig()); 
      Future<JobInfo> chekerFuture = executor 
       .submit(new JobStateChecker(jobId)); 
      return chekerFuture.get(getTimeout(command), TimeUnit.MILLISECONDS); 
     } catch (Exception e) { 
      log.error("Remote command failed", e); 
      JobInfo jobInfo = new JobInfo(); 
      jobInfo.setState(State.FAILED); 
      jobInfo.setMsg(ExceptionUtils.getStackTrace(e)); 
      return jobInfo; 
     } 
     } 

    private long getTimeout(RemoteCommand command) { 
     if (command.getTimeout() == null) { 
      return DEFAULT_TIMEOUT_SEC * DateTimeConstants.MILLIS_PER_SECOND; 
     } 
     return command.getTimeout().getMillis(); 
     } 

将DEFAULT_TIMEOUT_SEC更改为更高的值。