在cassandra中读取超过百万条记录

我使用Cassandra来存储发生在我的应用程序中的所有事件。6个月后它拥有超过百万的数据。在cassandra中读取超过百万条记录

我正在同时从卡桑德拉阅读所有的记录下面的错误

CLI -

cqlsh> use abc; 
cqlsh:abc> select count(*) from interaction; 
OperationTimedOut: errors={}, last_host=localhost

从春季数据（使用允许过滤）

Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency ONE (1 responses were required but only 0 replica responded) 
    at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:69) 
    at com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:256) 
    at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:172) 
    at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52) 
    at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:36) 
    at com.germinait.replica.CassandraTransfer.startTransfer(CassandraTransfer.java:43) 
    at com.germinait.replica.DemoApplication.run(DemoApplication.java:22) 
    at org.springframework.boot.SpringApplication.runCommandLineRunners(SpringApplication.java:673) 
    ... 5 more 
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency ONE (1 responses were required but only 0 replica responded) 
    at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:69) 
    at com.datastax.driver.core.Responses$Error.asException(Responses.java:94) 
    at com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:108) 
    at com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:235) 
    at com.datastax.driver.core.RequestHandler.onSet(RequestHandler.java:379) 
    at com.datastax.driver.core.Connection$Dispatcher.messageReceived(Connection.java:584) 
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) 
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) 
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) 
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) 
    at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70) 
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) 
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) 
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) 
    at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) 
    at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) 
    at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) 
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) 
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) 
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:

我不知道如何读取大量数据。

让我知道如何使用spring-data或其他替代方法来读取这些大数据。

来源

2016-09-16 Vaibhav Shah

卡桑德拉善于为特定键或范围返回值。它返回数百万行不好。事实上，未绑定的查询是一种反模式。 – Aaron

在这里回答：http://stackoverflow.com/questions/29394382/operation-time-out-error-in-cqlsh-console-of-cassandra/29394935#29394935如果有帮助 –

谢谢@ChrisLohfink –

如果您想知道表格包含多少行，您可以有一个计数器表格，它在插入记录时递增，在删除行时递减。

来源

2016-09-16 14:47:13

变化和看到读取驱动程序超时

https://datastax.github.io/java-driver/manual/socket_options/

Cluster cluster = Cluster.builder() 
    .addContactPoint("127.0.0.1") 
    .withSocketOptions(
      new SocketOptions() 
        .setConnectTimeoutMillis(2000)) 
    .build();

变化卡桑德拉服务器配置，以允许大的读和写

read_request_ timeout_in_ms，range_request_timeout_in_ms， write_request_timeout_in_ms，cas_contention_timeout_in_ms， truncate_request_timeout_in_ms

而且，看看这个答案以获取更多信息。

Cassandra read timeout

来源

2016-09-16 14:49:13 Sreekar

我试过这个解决方案它是返回的结果集，但它用java处理非常庞大。我正在寻找分页类的解决方案 –

在cassandra中读取超过百万条记录

回答

相关问题