Spring批处理分区步骤未运行

Question

我试图配置我的第一个多线程作业。我们需要处理大约200,000条记录的主文件。我想将文件分解成10个文件并处理它们。分割文件tasklet工作正常&它将bug文件分成10个小文件。我将作业参数键中的文件路径传递给作业“urlFilesPath”，并将其值设为“file：/ scrapper/spliturl *”。我验证了在MultiResourcePartitioner bean中正在正确读取文件。

主步骤在我的配置中运行，但从站步骤不运行。以下是我的配置。

分区程序：

<bean id="partitioner" 
    class="org.springframework.batch.core.partition.support.MultiResourcePartitioner" 
    scope="step"> 
    <property name="resources" value="#{jobParameters['urlFilesPath']}" /> 
</bean> 

MultiResourceItemReader：

<bean id="multiResourceItemReader" 
    class="org.springframework.batch.item.file.MultiResourceItemReader" 
    scope="step"> 
    <property name="resources" value="#{jobParameters['urlFilesPath']}" /> 
    <property name="delegate" ref="urlFileItemReader" /> 
    <property name="strict" value="true" /> 
    <property name="saveState" value="false" /> 
</bean> 

FlatFileItemWriter：

<bean id="urlFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader" 
scope="step"> 
    <property name="lineMapper" ref="passThroughLineMapper" />  
    <property name="resource" value="#{stepExecutionContext['fileName']}" /> 
    <property name="saveState" value="true" /> 
</bean> 

作业配置：

<batch:job id="importJob" job-repository="jobRepository"> 
    <batch:step id="fileSplitter" next="readURLFileRunner"> 
     <batch:tasklet ref="fileSplittingTasklet" 
      transaction-manager="transactionManager" /> 
    </batch:step> 

    <batch:step id="readURLFileRunner"> 
     <batch:partition step="readURLFile" partitioner="partitioner"> 
      <batch:handler grid-size="10" task-executor="taskExecutor" /> 
     </batch:partition> 
    </batch:step> 
</batch:job> 

从步骤配置：

<batch:step id="readURLFile"> 
    <batch:tasklet transaction-manager="transactionManager" 
     task-executor="taskExecutor" throttle-limit="10"> 
     <batch:chunk reader="multiResourceItemReader" processor="urlFileItemProcessor" 
      writer="validURLItemWriter" commit-interval="200" skip-limit="100"> 
      <batch:skippable-exception-classes> 
       <batch:include class="java.net.MalformedURLException" /> 
       <batch:include class="java.net.URISyntaxException" /> 
       <batch:include class="java.net.UnknownHostException" /> 
      </batch:skippable-exception-classes> 
     </batch:chunk> 
     <batch:listeners> 
      <batch:listener ref="malformedURLExceptionListener" /> 
      <batch:listener ref="uriSyntaxExceptionListener" /> 
      <batch:listener ref="unknownHostExceptionListener" /> 
     </batch:listeners> 
    </batch:tasklet> 
    <batch:end on="COMPLETED" /> 
</batch:step> 

请告知我在做什么错误。我没有看到处理器urlFileItemProcessor &作家validURLItemWriter正在执行。

更新 我遵循@dimzak给出的答案。但是，我仍然看不到步骤readURLFile正在执行从urlFileItemProcessor & validURLItemWriter记录器不会被打印到控制台。作业挂在后面的日志记录器

org.springframework.batch.repeat.support.TaskExecutorRepeatTemplate - Starting repeat context. 

我对配置步骤范围的方式有疑问。

StepScope配置

<bean class="org.springframework.batch.core.scope.StepScope"> 
    <property name="autoProxy" value="true"/> 
    <property name="proxyTargetClass" value="true"/> 
</bean> 

在春季论坛，我读过的multiResourceItemReader的委托财产不必一步作用域。当我从urlFileItemReader中删除scope =“step”时，我得到以下例外。

INFO : 26 Sep 2014 00:10:58,811 - org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor - Initializing ExecutorService 'taskExecutor' 
INFO : 26 Sep 2014 00:10:59,066 - org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor - Shutting down ExecutorService 'taskExecutor' 
Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'urlFileItemReader' defined in class path resource [beansBatchService.xml]: Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanExpressionException: Expression parsing failed; nested exception is org.springframework.expression.spel.SpelEvaluationException: EL1008E:(pos 0): Field or property 'stepExecutionContext' cannot be found on object of type 'org.springframework.beans.factory.config.BeanExpressionContext' 
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:547) 
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475) 
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:304) 
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228) 
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:300) 
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:195) 
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:703) 
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:760) 
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482) 
    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139) 
    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:83) 
    at com.chw.hma.service.batch.jobrunner.MainJobRunner.run(MainJobRunner.java:29) 
    at com.chw.hma.service.batch.jobrunner.MainJobRunner.main(MainJobRunner.java:21) 
Caused by: org.springframework.beans.factory.BeanExpressionException: Expression parsing failed; nested exception is org.springframework.expression.spel.SpelEvaluationException: EL1008E:(pos 0): Field or property 'stepExecutionContext' cannot be found on object of type 'org.springframework.beans.factory.config.BeanExpressionContext' 
    at org.springframework.context.expression.StandardBeanExpressionResolver.evaluate(StandardBeanExpressionResolver.java:146) 
    at org.springframework.beans.factory.support.AbstractBeanFactory.evaluateBeanDefinitionString(AbstractBeanFactory.java:1364) 
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.evaluate(BeanDefinitionValueResolver.java:214) 
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:186) 
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1456) 
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1197) 
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537) 
    ... 12 more 
Caused by: org.springframework.expression.spel.SpelEvaluationException: EL1008E:(pos 0): Field or property 'stepExecutionContext' cannot be found on object of type 'org.springframework.beans.factory.config.BeanExpressionContext' 
    at org.springframework.expression.spel.ast.PropertyOrFieldReference.readProperty(PropertyOrFieldReference.java:217) 
    at org.springframework.expression.spel.ast.PropertyOrFieldReference.getValueInternal(PropertyOrFieldReference.java:85) 
    at org.springframework.expression.spel.ast.PropertyOrFieldReference.getValueInternal(PropertyOrFieldReference.java:78) 
    at org.springframework.expression.spel.ast.CompoundExpression.getValueRef(CompoundExpression.java:49) 
    at org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:85) 
    at org.springframework.expression.spel.ast.SpelNodeImpl.getValue(SpelNodeImpl.java:102) 
    at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:94) 
    at org.springframework.context.expression.StandardBeanExpressionResolver.evaluate(StandardBeanExpressionResolver.java:143) 
    ... 18 more 

请指教。

更新2014年9月27日

发布任务执行等，由@dimzak

<bean id="taskExecutor" 
    class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor"> 
    <property name="corePoolSize" value="10" /> 
    <property name="maxPoolSize" value="10" /> 
</bean> 

<bean id="jobRepository" 
    class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean"> 
    <property name="transactionManager" ref="transactionManager" /> 
    <property name="dataSource" ref="dataSource" /> 
    <property name="databaseType" value="mySQL" /> 
</bean> 

<bean id="jobLauncher" 
    class="org.springframework.batch.core.launch.support.SimpleJobLauncher"> 
    <property name="jobRepository" ref="jobRepository" /> 
    <property name="taskExecutor" ref="taskExecutor" /> 
</bean> 

Answer 1

MultiResourcePartitioner需要file:/scrapper/spliturl*和分裂样spliturl1，spliturl2，在不同的步骤每个文件建议。

之后，在从属步骤中，您只需读取此步骤的特定文件。 所以你的配置应该没有MultiResourceItemReader。

<batch:step id="readURLFile"> 
    <batch:tasklet transaction-manager="transactionManager" 
     task-executor="taskExecutor" throttle-limit="10"> 
     <batch:chunk reader="urlFileItemReader" processor="urlFileItemProcessor" 
      writer="validURLItemWriter" commit-interval="200" skip-limit="100"> 
      <batch:skippable-exception-classes> 
       <batch:include class="java.net.MalformedURLException" /> 
       <batch:include class="java.net.URISyntaxException" /> 
       <batch:include class="java.net.UnknownHostException" /> 
      </batch:skippable-exception-classes> 
     </batch:chunk> 
     <batch:listeners> 
      <batch:listener ref="malformedURLExceptionListener" /> 
      <batch:listener ref="uriSyntaxExceptionListener" /> 
      <batch:listener ref="unknownHostExceptionListener" /> 
     </batch:listeners> 
    </batch:tasklet> 
    <batch:end on="COMPLETED" /> 
</batch:step>