Spring数据仓库多线程性能

我遇到了多个线程访问同一个弹簧数据存储库对象的性能问题。许多线程在等待资源库对象的锁时被阻塞。大多数线程都在存储库对象上执行相同的查询。当这个线程块运行时，CPU在所有内核上都被最大化。偶尔它会下降，我认为这是来自等待锁存器对象的阻塞线程。我通过分析验证了多个线程正在等待调用存储库对象中的相同方法。通过改变使用返回列表的方法的方法，我确实看到了性能的提升。但锁定仍然是一个瓶颈。Spring数据仓库多线程性能

更新：经过更多的研究，我得出结论，存储库对象是一个单身人士。这个对象在每个线程访问它时都被锁定。我如何建立资源库对象的原型？（我会为这个用例创建一个只读存储库。）配置是否需要更改？ Spring数据已经做到了吗？

MWE：

public interface EntityJpaRepository extends JpaRepository<Entity, Integer> { 
    @Query(value = "select * from SomeTable where id = (?1);", nativeQuery = true) 
    Entity findById(int id); 

    //Method that returns a list of Entities 
    @Query(value = "select * from SomeTable where id in (?1);", nativeQuery = true) 
    List<Entity> findAllWithIds(List<Integer> ids); 
} 

@Component 
@Scope("prototype") 
public class AThread implements Runnable { 
    @Autowired 
    EntityJpaRepository myRepository; 

    final int someId;   

    public AThread(int someId) { 
     this.someId = someId; 
    } 

    @Override 
    public void run() { 
     //may call subMethod 1 
     myRepository.findById(someId); 
     //may call subMethod 1 
     List<Integer> ids = someMethodWhichReturnsIDs(); 
     myRepository.findAllWithIds(ids); 
     //may call subMethod 1 
    } 

    public void subMethod1(){ 
     //sometimes loop 
     subMethod2(); 
     //may call 
    } 
    public void subMethod2(){ 
     //more stuff 
     List<Integer> ids = someMethodWhichReturnsIDs(); 
     //more stuff 
    } 
} 

public static void main(String[] args){ 
ThreadPoolTaskExecutor taskExecutor = (ThreadPoolTaskExecutor) ctx.getBean("taskExecutor"); 

List<int> someInts;//assume this is full of ints. 
for(int someId: someInts){ 
    taskExecutor.execute((Runnable)ctx.getBean("AThread", someId)); 
} 
waitThreads(taskExecutor);

我会说我得到的性能公平位出了什么我现在有。我也不确定我是否已正确设置配置以使用多个线程/连接访问数据库。我不认为这是问题，但我提供了完整的配置。欢迎任何性能提示。

@Configuration 
@EnableJpaRepositories(basePackages= {"org.repository"}) 
@ContextConfiguration(locations={"classpath:META-INF/spring/app-context.xml"}) 
@ComponentScan(basePackages = "org.somepackage") 
public class JpaConfiguration { 

    @Value("#{mainDataSource}") 
    private javax.sql.DataSource dataSource; 

    @Bean 
    public Map<String, Object> jpaProperties() { 
     Map<String, Object> props = new HashMap<String, Object>(); 
     props.put("hibernate.dialect", MySQLDialect.class.getName()); 
     props.put("hibernate.format_sql", true); 
     return props; 
    } 

    @Bean 
    public JpaVendorAdapter jpaVendorAdapter() { 
     HibernateJpaVendorAdapter hibernateJpaVendorAdapter = new HibernateJpaVendorAdapter(); 
     hibernateJpaVendorAdapter.setDatabase(Database.MYSQL); 
     return hibernateJpaVendorAdapter; 
    } 

    @Bean 
    public PlatformTransactionManager transactionManager() { 
     return new JpaTransactionManager(entityManagerFactory().getObject()); 
    } 

    @Bean 
    public LocalContainerEntityManagerFactoryBean entityManagerFactory() { 
     LocalContainerEntityManagerFactoryBean lef = new LocalContainerEntityManagerFactoryBean(); 
     lef.setDataSource(this.dataSource); 
     lef.setJpaPropertyMap(this.jpaProperties()); 
     lef.setJpaVendorAdapter(this.jpaVendorAdapter()); 
     return lef; 
    } 

    @Bean 
    public ThreadPoolTaskExecutor taskExecutor() { 
     ThreadPoolTaskExecutor pool = new ThreadPoolTaskExecutor(); 
     pool.setCorePoolSize(100); 
     pool.setMaxPoolSize(500); 
     pool.setWaitForTasksToCompleteOnShutdown(true); 
     pool.setKeepAliveSeconds(1800); 
     return pool; 
    } 
}

这是一个线程的堆栈跟踪，当我暂停进程/应用程序时，它被阻塞。我也有pro-filer的样本输出。当它运行更长时间时，阻塞的时间加起来。它显然被@Autowired存储库对象上的另一个线程阻塞。我以为我通过使用原型范围避免了这一点。

taskExecutor-1 BLOCKED 4.68003 6320 34 2062 1 [email protected] taskExecutor-9 


java.lang.ClassLoader.loadClass(Unknown Source) 
sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) 
java.lang.ClassLoader.loadClass(Unknown Source) 
org.springframework.util.ClassUtils.isVisible(ClassUtils.java:1209) 
org.springframework.util.ClassUtils.getAllInterfacesForClassAsSet(ClassUtils.java:1136) 
org.springframework.util.ClassUtils.getAllInterfacesForClassAsSet(ClassUtils.java:1143) 
org.springframework.util.ClassUtils.getAllInterfacesForClass(ClassUtils.java:1099) 
org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:302) 
com.sun.proxy.$Proxy41.createNativeQuery(Unknown Source) 
org.springframework.data.jpa.repository.query.NativeJpaQuery.createJpaQuery(NativeJpaQuery.java:65) 
org.springframework.data.jpa.repository.query.AbstractStringBasedJpaQuery.doCreateQuery(AbstractStringBasedJpaQuery.java:72) 
org.springframework.data.jpa.repository.query.AbstractJpaQuery.createQuery(AbstractJpaQuery.java:165) 
org.springframework.data.jpa.repository.query.JpaQueryExecution$SingleEntityExecution.doExecute(JpaQueryExecution.java:197) 
org.springframework.data.jpa.repository.query.JpaQueryExecution.execute(JpaQueryExecution.java:74) 
org.springframework.data.jpa.repository.query.AbstractJpaQuery.doExecute(AbstractJpaQuery.java:98) 
org.springframework.data.jpa.repository.query.AbstractJpaQuery.execute(AbstractJpaQuery.java:89) 
org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.doInvoke(RepositoryFactorySupport.java:421) 
org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:381) 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 
org.springframework.data.repository.core.support.RepositoryFactorySupport$DefaultMethodInvokingMethodInterceptor.invoke(RepositoryFactorySupport.java:512) 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 
org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) 
org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281) 
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 
org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136) 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 
org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodIntercceptor.invoke( CrudMethodMetadataPostProcessor.java:122) 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) 
com.sun.proxy.$Proxy81.findAllWithIds(Unknown Source) 
sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source) 
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) 
java.lang.reflect.Method.invoke(Unknown Source) 
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) 
com.sun.proxy.$Proxy90.findAllWithIds(Unknown Source) 
org.somepackage.AThread.subMethod2(AThread.java:696) 
org.somepackage.AThread.subMethod1(AThread.java:346) 
org.somepackage.AThread.run(AThread.java:132) 
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 
java.lang.Thread.run(Unknown Source)

来源

2017-09-16 Joshua Winfrey

如何得到'EntityJpaRepository'在main（用'CTX。 getBean'）并用'new AThread（theEntityJpaRepository）'创建你的线程。也不能使用某种方法来取代'List '而不是？ – 2017-09-16 04:17:30

我已经使用列表方法来获得一些性能改进。但我仍然看到相同的行为。多个线程正在等待进入存储库方法。我想认为@Scope（“prototype”）注释会给我一个Repository对象的副本。我开始认为@Autowired注释是一个共享（单例）对象。 –

你的代码看起来非常合理。它允许对数据库执行多达100个并发请求。不过，我不明白为什么你的线程被阻止。如果run方法的实现仅对数据库执行单一只读请求，并且不能被隔离级别低于可序列化的其他线程阻塞。你能否详细解释一下在运行方法的实际实现中，“线程在等待资源库对象锁定时被阻塞”的问题？ –

这看起来非常像反复加载类查询的问题。同一类的重复加载可能会导致类加载器锁争用过多。（这里例如：https://plumbr.eu/blog/locked-threads/classloading-and-locking）

的sharedEntityManager在加载线302类 - 这将在ClassLoader的上线404碰上synchronized块

来源

2017-09-19 09:23:43

问题原来是我的错误。我改变了程序的优先级，仍然使用它运行的机器来完成其他任务。当我将任务的优先级改回正常时，问题就消失了。当数据库进程和有关程序都处于同一优先级时，它也不存在。 –

Spring数据仓库多线程性能

回答

相关问题