代码点火器活动记录 - 帮助精简过程

Question

我目前使用下面的代码来获取uuid的列表，然后将它们分成1000个组，然后将这些组插入到数据库中。

这工作得很好，除了这个已经在时间上下工夫，超过一百万的UUID的 的问题是这种使用的内存的巨量，所以我需要帮助简化此过程中使用较少的内存...

public function send_daily_email($dealId) { 
    set_time_limit(0); 
    $deal = $this->ci->deal->get($dealId); 
    if ($deal == false) 
    throw new exception('Unknown Deal Specified'); 

    $users = $this->db->select('uuid')->from('userRegionLink')->where('regionId', $deal->region)->get(); 

    if ($users->num_rows() == 0) 
    throw new exception('No users in region'); 

    $message = $this->ci->load->view('emails/daily', array('name' => $deal->title, 'content' => $deal->snippet), true); 
    $uuids = array(); 

    foreach ($users->result() as $u) 
    $uuids[] = $u->uuid; 

    $uuids = array_chunk($uuids, 1000); 

    $sendId = 0; 
    foreach ($uuids as $batch) { 
    $count = count($batch); 
    $this->db->set('dealId', $dealId)->set('content', $message)->set('regionId', $deal->region)->set('recipients', $count)->set('created', 'NOW()', false)->set('status', 'Creating'); 
    if ($sendId === 0) { 
     $this->db->insert('dealEmailParent'); 
     $sendId = $this->db->insert_id(); 
     $this->db->set('sendId', $sendId)->where('id', $sendId)->update('dealEmailParent'); 
    } 
    else 
     $this->db->set('sendId', $sendId)->insert('dealEmailParent'); 

    $insert = array(); 

    foreach ($batch as $uuid) 
     $insert[] = array('parentId' => $sendId, 'uuid' => $uuid); 

    $this->db->insert_batch('dealEmailChild', $insert); 
    } 
} 

Answer 1

这返工可以插入1,000,000“用户”一分钟之内没有任何的内存限制:)

public function create_daily_email($dealId) 
    { 
     $time_start = microtime(true); 
     set_time_limit(0); 
     $deal = $this->ci->deal->get($dealId); 
     if ($deal == false) 
      throw new exception('Unknown Deal Specified'); 


     $message = $this->ci->load->view('emails/daily', array('name' => $deal->title, 'content' => $deal->snippet), true); 

     $start = 0; 
     $end = 50000; 

     $q = $this->db->select('uuid')->from('userRegionLink')->where('regionId', $deal->region)->limit($end, $start)->get(); 
     $sendId = 0; 

     while ($q->num_rows() != 0) { 

      //do stuff 

      $uuids = array(); 

      foreach ($q->result() as $u) 
       $uuids[] = $u->uuid; 

      $uuids = array_chunk($uuids, 1000); 

      foreach ($uuids as $batch) { 
       $count = count($batch); 
       $this->db->set('dealId', $dealId)->set('content', $message)->set('regionId', $deal->region)->set('recipients', $count)->set('created', 'NOW()', false)->set('status', 'Creating'); 
       if ($sendId === 0) { 
        $this->db->insert('dealEmailParent'); 
        $sendId = $this->db->insert_id(); 
        $this->db->set('sendId', $sendId)->where('id', $sendId)->update('dealEmailParent'); 
        $parentId = $sendId; 
       } 
       else { 
        $this->db->set('sendId', $sendId)->insert('dealEmailParent'); 
        $parentId = $this->db->insert_id(); 
       } 

       $insert = array(); 

       foreach ($batch as $uuid) { 
        $insert[] = array(
         'parentId' => $parentId, 
         'uuid' => $uuid 
        ); 
       } 

       $this->db->insert_batch('dealEmailChild', $insert); 
      } 

      //stop stuff 

      unset($q); 
      unset($uuids); 
      unset($insert); 
      $start = $start + $end; 
      $q = $this->db->select('uuid')->from('userRegionLink')->where('regionId', $deal->region)->limit($end, $start)->get(); 
     } 

     $this->db->set('status', 'Pending')->where('sendId', $sendId)->update('dealEmailParent'); 

     $time_end = microtime(true); 
     $time = $time_end - $time_start; 

     die("Did nothing in $time seconds"); 
    } 

Answer 2

我讨厌这么说，但是从我对CodeIgniter的了解中，知道获取结果的唯一方法是一次获取整个结果集，即使您只需要一行，或者即使您想要一次取一行并做一些处理。它不像本机mysql（i）/ PDO功能那样使用游标进行操作。

对于这个庞大的数据集，我建议坚持原生的PHP数据库函数和前面的CodeIgniter的活动记录数据库类。