插入语句的性能改进

在我的ASP.NET MVC应用程序中，我正在运行一些插入，它可以插入10000行或更多行并更新其他几行。这个过程需要很长时间，但我无法逃避插入，因为这正是我被要求做的。现在我正在运行Sql Server Profiler，它需要将近20分钟才能插入这一堆行。我怎么可能改善这一行动的表现？插入语句的性能改进

（我使用LINQ到SQL来将数据插入到数据库。）

这是方法做的代码插入数据：

[AcceptVerbs(HttpVerbs.Post)] 
     public ActionResult SaveEvent(int id) 
     { 
      int eventID= 0; 
      var query = from q in context.InventoryGoods 
         where q.ParentId == id && q.Action.HasValue && q.ActionOn.HasValue == false 
         select q; 

      var stockType = from q in context.Inventory 
          where q.Id == id 
          select q.StockType; 

      if (query.Count() > 0) 
      { 
       foreach (var i in query) 
       { 
         switch (i.Action.Value) 
         { 
          case (int)InventoryGoodsActionEnum.AdjustLocation: 

           Guid guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now, 
             EventOn = DateTime.Now, 
             Type = "LO", 
             Lid = i.LidObtained, 
             Comments = "Inventário "+i.ParentId, 
             UserId = GetUserId(), 
            }; 

            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 



           break; 

          case (int)InventoryGoodsActionEnum.AdjustQuantity: 

           if (!i.QuantityObtained.HasValue) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now, 
              EventOn = DateTime.Now, 
              Type = "AQ", 
              Quantity = (short)(i.QuantityExpected * -1), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 

           } 
           else if ((i.QuantityObtained - (i.QuantityExpected.HasValue ? i.QuantityExpected : 0) != 0)) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now, 
              EventOn = DateTime.Now, 
              Type = "AQ", 
              Quantity = (short)(i.QuantityObtained.Value - (i.QuantityExpected.HasValue ? i.QuantityExpected : 0)), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 
           } 


           break; 

          case (int)InventoryGoodsActionEnum.AdjustQuantityLocation: 

           guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now, 
             EventOn = DateTime.Now, 
             Type = "LO", 
             Lid = i.LidExpected, 
             Comments = "Inventário " + i.ParentId, 
             UserId = GetUserId(), 
            }; 

            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 
           if (!i.QuantityObtained.HasValue) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now.AddSeconds(1), 
              EventOn = DateTime.Now.AddSeconds(1), 
              Type = "AQ", 
              Quantity = (short)(i.QuantityExpected * -1), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 

           } 
           else if ((i.QuantityObtained - (i.QuantityExpected.HasValue ? i.QuantityExpected : 0) != 0)) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now.AddSeconds(1), 
              EventOn = DateTime.Now.AddSeconds(1), 
              Type = "AQ", 
              Quantity = (short)(i.QuantityObtained.Value - (i.QuantityExpected.HasValue ? i.QuantityExpected : 0)), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 
           } 



           break; 

          case (int)InventoryGoodsActionEnum.AdjustStockType: 

           guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now, 
             EventOn = DateTime.Now, 
             Type = "ST", 
             StockType = stockType.First().Value, 
             Comments = "Inventário " + i.ParentId, 
             UserId = GetUserId(), 
            }; 

            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 


           break; 
          case (int)InventoryGoodsActionEnum.AdjustLocationStockType: 

           guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now, 
             EventOn = DateTime.Now, 
             Type = "ST", 
             StockType = stockType.First().Value, 
             Comments = "Inventário " + i.ParentId, 
             UserId = GetUserId(), 
            }; 

            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 

           guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now.AddSeconds(1), 
             EventOn = DateTime.Now.AddSeconds(1), 
             Type = "LO", 
             Lid = i.LidExpected, 
             Comments = "Inventário " + i.ParentId, 
             UserId = GetUserId(), 
            }; 



            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 



           break; 
          case (int)InventoryGoodsActionEnum.AdjustQuantityStockType: 

           guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now, 
             EventOn = DateTime.Now, 
             Type = "ST", 
             StockType = stockType.First().Value, 
             Comments = "Inventário " + i.ParentId, 
             UserId = GetUserId(), 
            }; 

            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 

           if (!i.QuantityObtained.HasValue) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now.AddSeconds(1), 
              EventOn = DateTime.Now.AddSeconds(1), 
              Type = "AQ", 
              Quantity = (short)(i.QuantityExpected * -1), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 

           } 
           else if ((i.QuantityObtained - i.QuantityExpected != 0)) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now.AddSeconds(1), 
              EventOn = DateTime.Now.AddSeconds(1), 
              Type = "AQ", 
              Quantity = (short)(i.QuantityObtained.Value - i.QuantityExpected), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 
           } 


           break; 
          case (int)InventoryGoodsActionEnum.AdjustQuantityLocationStockType: 

           guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now, 
             EventOn = DateTime.Now, 
             Type = "ST", 
             StockType = stockType.First().Value, 
             Comments = "Inventário " + i.ParentId, 
             UserId = GetUserId(), 
            }; 

            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 

           if (!i.QuantityObtained.HasValue) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now.AddSeconds(1), 
              EventOn = DateTime.Now.AddSeconds(1), 
              Type = "AQ", 
              Quantity = (short)(i.QuantityExpected * -1), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 

           } 
           else if ((i.QuantityObtained - i.QuantityExpected != 0)) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now.AddSeconds(1), 
              EventOn = DateTime.Now.AddSeconds(1), 
              Type = "AQ", 
              Quantity = (short)(i.QuantityObtained.Value - i.QuantityExpected), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 
           } 

           guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now.AddSeconds(2), 
             EventOn = DateTime.Now.AddSeconds(2), 
             Type = "LO", 
             Lid = i.LidExpected, 
             Comments = "Inventário " + i.ParentId, 
             UserId = GetUserId(), 
            }; 



            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 




           break; 
        } 
       } 
      } 
      else 
      { 
       var lista = from q in context.InventoryGoods 
          where q.ParentId == id 
          select q; 

       Repository.EvaluateActions(lista.ToList()); 

       SaveEvent(id); 
      } 


      using (var scope = new TransactionScope()) 
      { 
       var thisInventory = from i in context.Inventory 
            where i.Id == id 
            select i; 

       thisInventory.First().State = (int)InventoryStateEnum.Verified; 

       context.SubmitChanges(); 

       scope.Complete(); 
      } 

      Status.Info(string.Format("Acções aplicadas com sucesso.")); 
      return RedirectToAction("Details", new { id }); 
     } 



public void SetActionOn(int id, int eventID) 
     { 
      var InventoryGoods = from i in context.InventoryGoods 
           where i.Id == id 
           select i; 



      using (var scope = new TransactionScope()) 
      { 
       InventoryGoods.First().ActionOn = DateTime.Now; 

       InventoryGoodsEvents ige = new InventoryGoodsEvents 
       { 
        EventId = eventID, 
        InventoryGood = InventoryGoods.First().Id, 
       }; 

       context.InventoryGoodsEvents.InsertOnSubmit(ige); 

       scope.Complete(); 
      } 
     }

来源

2010-07-21 Hallaghan

这种方法是怎么回事：Repository.SetActionOn（i.Id，eventID）; – 2010-07-21 11:42:07

我会在我的主要帖子上张贴它。 – Hallaghan 2010-07-21 11:45:34

停止使用var这么多。

这会运行查询两次（在sqlprofiler中观察）。

if (query.Count() > 0) 
{ 
    foreach (var i in query)

使用此选项可避免多次运行查询。

List<InventoryGoods> rows = query.ToList();

你的代码有很多重复。另外，您在每个提交的上下文中尝试做的太少。您无故控制交易范围。

foreach(InventoryGood i in rows) 
{ 
    InventoryGoodsEvent ige = new InventoryGoodsEvent() 
    //this will attach ige to the object graph tracked by context 
    // which is sufficient to insert ige when submitchanges is called. 
    ige.InventoryGood = i; 

    GoodsEvent ge = GetGoodsEvent(i); //all that conditional logic in there. 
    //this will attach ge to the object graph tracked by context 
    // which will both insert ge and update ige with ge's id when submitchanges is called. 
    ige.GoodsEvent = ge; 

    i.ActionOn = DateTime.Now; 
    //to submit each row, uncomment this. 
    //context.SubmitChanges(); 
} 
//to submit all rows at once, use this. 
context.SubmitChanges();

如果InventoryGoodEgvents没有这些关系属性，请进入设计器并添加关联以创建它们。

一旦你有了这样的代码，那么你就可以决定在一个事务中做什么样的改变。我喜欢为每笔交易插入~100条记录。如果每个事务使用1条记录，则创建每个事务的开销很大。如果每个事务使用1,000,000行，则长时间运行事务的开销很大。

这东西很难学，但要坚持下去。

还有一件事：批量插入不适用于多对多表。

来源

2010-07-21 12:11:52

<主观评论警报>这里使用var的方式没有任何问题。它与Linq查询结合在一起，每个人都知道返回IEnumerable或者你选择的任何东西，并且他还与新关键字结合使用 - 这比冗长的TransactionScope scope = new TransactionScope（）少得多。 Var可以被过度使用，但在这里肯定是适当的。 – mattmc3 2010-07-21 17:02:11

有些特定的声明需要var。使用var之外的是风格问题（如你所说，主观）。您选择的案例仅保存13个字符。也许如果线路没有针对长度进行优化，那么就不会使用这么多次，最终可以节省很多更多的字符。在理解查询语法结果分配的情况下，如你所说的“或其他”。 “或者不管”是问题。 – 2010-07-22 02:18:16

什么DAL你使用EF，L2S，ADO.net还是其他？插入不应该花很多时间来完成。您可以将它们插入本地缓存，稍后再提交更改。

来源

2010-07-21 11:01:04 Wyvern

我们使用的是ADO.NET。 – Hallaghan 2010-07-21 11:05:31

尝试插入8500行后，系统花30分钟完成操作。 – Hallaghan 2010-07-21 11:16:52

好吧，明白了。请看下面的这个模式： context.GoodsEvent.InsertOnSubmit（ge）; context.SubmitChanges（）; 只需移动“context.SubmitChanges（）;”甚至超出了foreach循环的范围。 SubmitChanges是一个耗时的操作，它同步本地缓存和远程Sql DB数据状态。如果你插入8500行，它会同步8500次？但将SubmitChanges（）移出循环后，它只同步一次。 – Wyvern 2010-07-21 11:36:05

Linq-to-sql实际上并没有设计用于在同一批次中将多条记录插入数据库。它会做insert声明insert声明这真的很慢。我建议你在任何你知道需要支持这个插入的地方使用SqlBulkCopy对象来代替你的Linq-to-sql类。如果您需要它们进行对象验证，您甚至可以使用相同的L2S类，但只需将它们转储到1000行块中的DataTable中，并让SqlBulkCopy执行您的实际插入。你甚至可以谷歌L2S和SqlBulkCopy，看看那里有什么扩展方法或其他集成。你不是第一个遇到这个问题的人。

来源

2010-07-21 11:24:40 mattmc3

+1'SqlBulkCopy' – abatishchev 2010-07-21 11:27:16

你看过我上面的方法的代码了吗？你认为我可以改进它以使其更快吗？我想知道我是否可以运行这些“context.SubmitChanges（）;”只是在我的开关结束时，或者如果我现在需要运行他们。你怎么看？ – Hallaghan 2010-07-21 11:30:00

当我发布我的答案时，它并不存在，但乍一看有很多重复的代码。在优化插入性能之前，这对于重构**可能已经成熟。 – mattmc3 2010-07-21 12:10:20

插入语句的性能改进

回答

相关问题