如何导入数据库，更新已更改的产品，删除已删除的产品

Question

在基本MySQL中，我是“好的”，但这是“我的头”！

目标：已更改

已被删除

快速删除产品

• 导入数据库
• 更新的产品和高效

数据库表（或多个）巨大的，速度是一个问题。

不需要MyISAM inoDB会更快吗？每个数据库将在一个唯一的表中。

我得到了这是一个开始的地方，我想要做的事：

CREATE TABLE `table` LIKE LiveTable 
LOAD DATA INFILE..... INTO `table` 
UPDATE `table` SET delete=1; -- Set the delete field to true because it will not have been updated 
UPDATE `table` INNER JOIN`table`ON `LiveTable.ID`=`table.ID` 
SET LiveTable.Col1=table.Col1, LiveTable.Col2=table.Col2….. delete=0 
INSERT INTO LiveTable(ID,Col1,Col2,… delete=0) 
SELECT ID,Col1,Col2,...FROM `table` 
LEFT JOIN LiveTable 
ON table.ID = LiveTable.ID 
WHERE LiveTable.ID IS NULL 
DELETE FROM LiveTableWHERE delete = 0 
EMPTY TABLE `table` 

---

> CREATE TABLE `product_table` (
>  `programname` VARCHAR(100) NOT NULL, 
>  `name`  VARCHAR(160) NOT NULL, 
>  `keywords` VARCHAR(300) NOT NULL, 
>  `description` TEXT NOT NULL, 
>  `sku`   VARCHAR(100) NOT NULL, 
>  -- This is the only "unique identifier given, none will be duplicates" 
>  `price`  DECIMAL(10, 2) NOT NULL, 
>  `created`  TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, 
>  `updatedat` TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00', 
>  `delete`  TINYINT(4) NOT NULL DEFAULT '0', 
>  PRIMARY KEY (`sku`) ) ENGINE=myisam DEFAULT CHARSET=latin1; 
> 
> CREATE TABLE IF NOT EXISTS `temptable` LIKE `product_table`; 
> 
> TRUNCATE TABLE `temptable`; -- Remove data from temp table if for some 
> reason it has data in it. 
> 
> LOAD DATA LOW_PRIORITY LOCAL INFILE "catalog.csv" INTO TABLE 
> `temptable` FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY """" 
> LINES TERMINATED BY "\n" IGNORE 1 LINES (`PROGRAMNAME`, `NAME`, 
> `KEYWORDS`, `DESCRIPTION`, `SKU`, `PRICE`); 
> 
> 
> UPDATE `temptable` SET `delete` = 1; -- Set the delete field to 
> true UPDATE `temptable` ttable 
>  INNER JOIN `product_table` mtable 
>   ON (mtable.sku = ttable.sku) SET mtable.programname = ttable.programname, 
>  mtable.name = ttable.name, 
>  mtable.keywords = ttable.keywords, 
>  mtable.description = ttable.description, 
>  mtable.sku = ttable.sku, 
>  mtable.price = ttable.price, 
>  mtable.created = ttable.created, 
>  mtable.updatedat = NOW(),-- Set Last Update 
>  mtable.delete = 0; -- Set Delete to NO 
> 
> -- Not sure what this is for... I'm LOST at this part... 
> INSERT INTO `product_table` VALUES  (`programname`, 
>    `name`, 
>    `keywords`, 
>    `description`, 
>    `sku`, 
>    `price`, 
>    `created`, 
>    `updatedat`, 
>    `delete`); 
> 
> -- This type of join requires alias as far as I know? 
> SELECT `programname`, 
>  `name`, 
>  `keywords`, 
>  `description`, 
>  `sku`, 
>  `price`, 
>  `created`, 
>  `updatedat`, 
>  `delete` FROM `temptable` tmptable 
>  LEFT JOIN `product_table` maintbl 
>   ON tmptable.sku = maintbl.sku WHERE maintbl.sku IS NULL; 
> 
> DELETE FROM `product_table` WHERE `delete` = 0; 
> 
> TRUNCATE TABLE `temptable`; `` remove all the data from temporary 
> table. 

Answer 1

我在这里回答了这个问题自己： https://dba.stackexchange.com/questions/16197/innodb-update-slow-need-a-better-option/16283#16283

使用我从这里得到的消息，在网络和一些网络聊天室，我想出了。网源：http://www.softwareprojects.com/resources/programming/t-how-to-use-mysql-fast-load-data-for-updates-1753.html

[DEMO][1] http://sqlfiddle.com/#!2/4ebe0/1 

的过程是：

Import into a new temp table. 
Update The old table information with information in Temp table. 
Insert new data into the table. (Real world I'm making a new CSV file and using LOAD INTO for the insert) 
delete everything that is no longer in the data feed. 
delete the temp table. 

这似乎是最快的方法为止。

让我知道你的意见是什么。

Answer 2

的InnoDB通常要在表比MyISAM数据更好的是可用而INSERT，UPDATE和DELETE正在发生，因为InnoDB使用行级锁定进行更新，而MyISAM使用表级锁定。

这是第一步。

第二步是在使用ALTER TABLE .. DISABLE KEYS将数据加载到表中之前禁用表中的所有索引，然后在使用ALTER TABLE .. ENABLE KEYS加载后将其重新启用。

以上两项显示了您的性能大幅提升。

作为另一种优化，在执行大规模更新时，将它们分成批（可能基于主键），以便所有行不会同时被锁定。