如何使用Java在mongodb上载json文件？

Question

我想使用Java将大的JSON文件（newclicklogs.json）上传到MongoDB中。这里是我的JSON文件看起来像：

{"preview":false,"result":{"search_term":"rania","request_time":"Sat Apr 01 12:47:04 -0400 2017","request_ip":"127.0.0.1","stats_type":"stats","upi":"355658761","unit":"DR","job_title":"Communications Officer","vpu":"INP","organization":"73","city":"Wash","country":"DC","title":"Tom","url":"www.demo.com","tab_name":"People-Tab","page_name":"PEOPLE","result_number":"5","page_num":"0","session_id":"df234f468cb3fe8be","total_results":"5","filter":"qterm=rina","_time":"2017-04-01T12:47:04.000-0400"}} 
{"preview"......} 
{"preview"......} 
.... 

这里是我的Java代码：

import java.io.File; 
import java.io.IOException; 
import java.util.ArrayList; 
import java.util.List; 
import org.apache.commons.io.FileUtils; 
import org.bson.Document; 
import com.mongodb.DBObject; 
import com.mongodb.MongoClient; 

public class Main { 

    public static void main(String[] args) throws IOException { 

     String jsonString = FileUtils.readFileToString(new File("data/newclicklogs.json"), "UTF-8"); 

     Document doc = Document.parse(jsonString); 
     List<Document> list = new ArrayList<>(); 
     list.add(doc); 

     new MongoClient().getDatabase("test2").getCollection("collection1").insertMany(list); 

    } 
} 

当我询问我的MongoDB集合，只有一个文件获取添加。如何将我的文件中的所有文档添加到mongodb集合中。我是MongoDB的新手。任何帮助表示赞赏。

Answer 1

您应该尝试使用带缓冲读取器的批量写入。

下面的代码将从文件，一行（文档）中读取json数据，然后在将数据写入数据库之前解析json到Document和批量请求。

MongoClient client = new MongoClient("localhost", 27017); 
MongoDatabase database = client.getDatabase("test2"); 
MongoCollection<Document> collection = database.getCollection("collection1"); 

int count = 0; 
int batch = 100; 

List<InsertOneModel<Document>> docs = new ArrayList<>(); 

try (BufferedReader br = new BufferedReader(new FileReader("data/newclicklogs.json"))) { 
     String line; 
     while ((line = br.readLine()) != null) { 
     docs.add(new InsertOneModel<>(Document.parse(line))); 
     count++; 
     if (count == batch) { 
      collection.bulkWrite(docs, new BulkWriteOptions().ordered(false)); 
      docs.clear(); 
      count = 0; 
     } 
    } 
} 

if (count > 0) { 
    collection.bulkWrite(docs, new BulkWriteOptions().ordered(false)); 
} 

当你对整个JSON你基本上是通过覆盖所有以前的的减少了文件最后文件运行Document.parse。

这里更多

http://mongodb.github.io/mongo-java-driver/3.4/driver/tutorials/bulk-writes/