多线程python下载循环

我有一个列表。多线程python下载循环

symbols = ('GGP', 'JPM', 'AIG', 'AMZN','GGP', 'rx', 'jnj', 'osip') 

URL = "http://www.Xxxx_symbol=%s" 

def fetch(symbols): 
    try: 
     url = URL % '+'.join(symbols) 
     fp = urllib2.urlopen(url) 
     try: 
      data = fp.read() 

     finally: 
      fp.close() 
     return data 
    except Exception as e: 
     print "No Internet Access"

我想多线程（与4线程）取数据进程，而不是多进程和不使用扭曲。 Url fetch的输出文件是csv，带有7行标题信息，我想摆脱它。我想循环每个符号在它自己的文件中。我以前使用过这个提取码。我可以得到一个有一个元素的符号列表。

来源

2011-02-01 Merlin

这应该让你开始：

from threading import Thread, Lock 

data = {} 
data_lock = Lock() 

class Fetcher(Thread): 
    def __init__(self, symbol): 
     super(Thread, self).__init__() 
     Thread.__init__(self) 
     self.symbol = symbol 

    def run(self): 
     # put the code from fetch() here 
     # replace 'data = fp.read()' with the following 
     tmp = fp.read() 
     data_lock.acquire() 
     data[self.symbol] = tmp 
     data_lock.release() 

# Start a new Fetcher thread like this: 
fetcher = Fetcher(symbol) 
fetcher.start() 
# To wait for the thread to finish, use Thread.join(): 
fetcher.join()

来源

2011-02-01 01:08:58

多线程python下载循环

回答

相关问题