插入从select查询中检索到的MySQL结果集？

Question

我有一个MySQL数据库位于服务器x和另一个在服务器y。

我试图让从表中的记录叫testx这是位于服务器X和INSERT他们到一个表名为暴躁。所以我所做的是执行SELECT声明并将其存储到resultset。然后我试图在resultsetwhile循环内迭代INSERT语句。这是我的示例代码：

private static void cloneTableAndAlter() throws ClassNotFoundException, SQLException, InstantiationException, IllegalAccessException, InterruptedException { 
     Connection connForSource = getConnectionForSource(); 
     Connection connForTarget = getConnectionForTarget(); 

     if (connForSource != null && connForTarget != null) { 
      try { 
       Statement st = connForSource.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY, 
         java.sql.ResultSet.CONCUR_READ_ONLY); 

       String selectStatement = "SELECT field1, field2 FROM dbx.testx where time between ('2016-09-01 00:00:00') and ('2016-09-03 23:59:59');"; 

       String insertStatement = "INSERT INTO testy(" + "field1," + "field2)" + 
         "VALUES(?,?) " + 
         "ON DUPLICATE KEY UPDATE field1 = VALUES(field1);"; <----field1 is a unique key but not primary 

       st.setFetchSize(Integer.MIN_VALUE); 

       ResultSet resultSetForSelect = st.executeQuery(selectStatement); 
       PreparedStatement preparedStatement = connForTarget.prepareStatement(insertStatement); 
       int count = 0; 
       while (resultSetForSelect.next()) { 
        ++count; 
        TableDetails tableDetails = setTableDetails(resultSetForSelect); <--- i'm getting the value from the resultset and setting it to my DTO class. 
        getTableDetails(preparedStatement, tableDetails); <--- setting the values back during the insert from the DTO 
        preparedStatement.executeUpdate(); 
        System.out.println(count + "rows affected"); 
       } 
      } catch (SQLException e) { 
       e.printStackTrace(); 
      } 
     } 
    } 

注意：源表（testx）拥有超过一百万条记录。

我能够做插入，但我觉得插入是有点慢，我得到45-50插入每秒。我要去哪里？

有什么办法可以优化这个操作并增加插入，或者这是插入大型数据集的本质吗？

任何帮助，不胜感激。

Answer 1

，你可以在其他的方式做：

创建选择这个Concats

select GROUP_CONCAT(
    CONCAT(" ('",field1,"','",field2,"')")) as vals 
from my_table; 

的结果是这样的：

结果

mysql> select GROUP_CONCAT(
    -> CONCAT(" ('",field1,"','",field2,"')")) as vals 
    -> from my_table; 
+---------------------------------------------------------------------------------------------------------+ 
| vals                         | 
+---------------------------------------------------------------------------------------------------------+ 
| ('O1','AC'), ('O1','PT'), ('O2','PT'), ('O3','MI'), ('O3','PT'), ('O4','EG'), ('O4','PT'), ('O5','PT') | 
+---------------------------------------------------------------------------------------------------------+ 
1 row in set (0,00 sec) 

mysql> 

，您可以直接在插入语句中对此进行concat，并且只读取一行，并且只写入和执行一条语句。

您的代码

String selectStatement = "SELECT GROUP_CONCAT(CONCAT(" ('",field1,"','",field2,"')")) as vals 
     FROM dbx.testx where time between ('2016-09-01 00:00:00') and ('2016-09-03 23:59:59');"; 

    ResultSet resultSetForSelect = st.executeQuery(selectStatement); 


    String insertStatement = "INSERT INTO testy(" + "field1," + "field2)" + 
          + String from result + 
          "ON DUPLICATE KEY UPDATE field1 = VALUES(field1);"; <----field1 is a unique key but not primary 


# execute one time