搜索时显示的数据错误

Question

我拥有超过一百万行的数据集。我已经使用logstash与Mysql集成了elasticsearch。 当我键入以下URL在邮递员来取，

http://localhost:9200/persondetails/Document/_search?q= *

我得到如下：

{ 
"took": 1, 
"timed_out": false, 
"_shards": { 
    "total": 5, 
    "successful": 5, 
    "failed": 0 
}, 
"hits": { 
    "total": 2, 
    "max_score": 1, 
    "hits": [ 
     { 
      "_index": "persondetails", 
      "_type": "Document", 
      "_id": "%{idDocument}", 
      "_score": 1, 
      "_source": { 
       "iddocument": 514697, 
       "@timestamp": "2017-08-31T05:18:46.916Z", 
       "author": "vaibhav", 
       "expiry_date": null, 
       "@version": "1", 
       "description": "ly that", 
       "creation_date": null, 
       "type": 1 
      } 
     }, 
     { 
      "_index": "persondetails", 
      "_type": "Document_count", 
      "_id": "AV4o0J3OJ5ftvuhV7i0H", 
      "_score": 1, 
      "_source": { 
       "query": { 
        "term": { 
         "author": "rishav" 
        } 
       } 
      } 
     } 
    ] 
} 

}

它是错误的，因为排在我的表数超过100万，这表明总数只有2个。我无法找到这里的错误。

当我键入http://localhost:9200/_cat/indices?v 这表明该

1. 健康：黄

2. 状态：开放

3. 指数：persondetails

4. UUID：4FiGngZcQfS0Xvu6IeHIfg

个

5. PRI：5

6. 代表：1

7. docs.count：2

8. docs.deleted：1054

9. store.size：125.4kb

10. pri.store.size：125.4kb

这是我logstash.conf文件

input { 
jdbc { 
    jdbc_connection_string => "jdbc:mysql://127.0.0.1:3306/persondetails" 
    jdbc_user => "root" 
    jdbc_password => "" 
    schedule => "* * * * *" 
    jdbc_validate_connection => true 
    jdbc_driver_library => "/usr/local/Cellar/logstash/5.5.2/mysql-connector-java-3.1.14/mysql-connector-java-3.1.14-bin.jar" 
    jdbc_driver_class => "com.mysql.jdbc.Driver" 
    statement => "SELECT * FROM Document" 
    type => "persondetails" 
} 
} 
output { 
elasticsearch { 
    #protocol=>http 
    index =>"persondetails" 
    document_type => "Document" 
    document_id => "%{idDocument}" 
    hosts => ["http://localhost:9200"] 
    stdout{ codec => rubydebug} 
} 
} 

Answer 1

从你的结果，它看起来像有是这是造成因为没有得到产生DOCUMENT_ID要覆盖文档您logstash配置的问题，并有效只有一个文件与文档ID的指数为“％{} idDocument”

请参阅从结果如下_source片段到您提供的搜索查询：

"_source": { 
      "iddocument": 514697, 
      "@timestamp": "2017-08-31T05:18:46.916Z", 
      "author": "vaibhav", 
      "expiry_date": null, 
      "@version": "1", 
      "description": "ly that", 
      "creation_date": null, 
      "type": 1 
} 

即使看索引的小尺寸，看起来好像还没有更多的文档。你应该看看你的jdbc输入是否提供了“idDocument”字段。