C＃异步Web浏览器运行速度很慢

Question

该程序读取网站列表，然后保存它们。 我发现它运行良好的前2个网址请求。然后变得非常缓慢（每个请求大约5分钟）

第1行和第2行花费的时间只有2秒。 然后所有其他的将是每个约5分钟。

当我调试，我看到它实际上tooks长wb.Navigate(url.ToString());

public static async Task<bool> test() 
    { 

     long totalCnt = rows.Count(); 
     long procCnt = 0; 
     foreach (string url in rows) 
     { 
      procCnt++; 

      string webStr = load_WebStr(url).Result; 
      Console.WriteLine(DateTime.Now+ "["+procCnt + "/" + totalCnt+"] "+url); 
     } 



     return true; 
    } 


public static async Task<string> load_WebStr(string url) 
{ 
    var tcs = new TaskCompletionSource<string>(); 

    var thread = new Thread(() => 
    { 
     EventHandler idleHandler = null; 

     idleHandler = async (s, e) => 
     { 
      // handle Application.Idle just once 
      Application.Idle -= idleHandler; 

      // return to the message loop 
      await Task.Yield(); 

      // and continue asynchronously 
      // propogate the result or exception 
      try 
      { 
       var result = await webBrowser_Async(url); 
       tcs.SetResult(result); 
      } 
      catch (Exception ex) 
      { 
       tcs.SetException(ex); 
      } 

      // signal to exit the message loop 
      // Application.Run will exit at this point 
      Application.ExitThread(); 
     }; 

     // handle Application.Idle just once 
     // to make sure we're inside the message loop 
     // and SynchronizationContext has been correctly installed 
     Application.Idle += idleHandler; 
     Application.Run(); 
    }); 

    // set STA model for the new thread 
    thread.SetApartmentState(ApartmentState.STA); 

    // start the thread and await for the task 
    thread.Start(); 
    try 
    { 
     return await tcs.Task; 
    } 
    finally 
    { 
     thread.Join(); 
    } 

} 


public static async Task<string> webBrowser_Async(string url) 
{ 

    string result = ""; 
    using (var wb = new WebBrowser()) 
    { 
     wb.ScriptErrorsSuppressed = true; 

     TaskCompletionSource<bool> tcs = null; 
     WebBrowserDocumentCompletedEventHandler documentCompletedHandler = (s, e) => 
     tcs.TrySetResult(true); 


     tcs = new TaskCompletionSource<bool>(); 
     wb.DocumentCompleted += documentCompletedHandler; 
     try 
     { 
      wb.Navigate(url.ToString()); 
      // await for DocumentCompleted 
      await tcs.Task; 
     } 
     catch 
     { 
      Console.WriteLine("BUG!"); 

     } 
     finally 
     { 
      wb.DocumentCompleted -= documentCompletedHandler; 
     } 
     // the DOM is ready 

     result = wb.DocumentText; 

    } 


    return result; 
} 

Answer 1

我认识的代码稍加修改的版本，我曾经回答了不少WebBrowser - 相关问题。是this one？包含指向原始来源的链接始终是一个好主意。

无论如何，您在这里如何使用它的主要问题可能是您创建并销毁了列表中每个URL的WebBrowser控件实例。

相反，您应该重新使用WebBrowser（或WebBrowser对象池）的单个实例。您可以找到两个版本here。