异步获取和存储图像在Python中

Question

以下代码是非异步代码的示例，有没有什么方法异步获取图像？

​​

我也看到了Grequests库，但我想不出太多，如果这是可能的，或者如何从文档做到这一点。

Answer 1

你不需要任何第三方库。只需为每个请求创建一个线程，启动线程，然后等待所有这些线程在后台完成，或者在下载图像时继续执行应用程序。

import threading 

results = [] 
def getter(url, dest): 
    results.append(urllib.urlretreave(url, dest)) 

threads = [] 
for x in range(0,10): 
    t = threading.Thread(target=getter, args=('http://test.com/file %s.png' % x, 
               'temp/file %s.png' % x)) 
    t.start() 
    threads.append(t) 
# wait for all threads to finish 
# You can continue doing whatever you want and 
# join the threads when you finally need the results. 
# They will fatch your urls in the background without 
# blocking your main application. 
map(lambda t: t.join(), threads) 

您还可以选择创建一个线程池，将让urls和dests从队列中。

如果您使用Python 3，它已经在futures模块中为您实施。

Answer 2

像这样的东西应该帮助你

import grequests 
urls = ['url1', 'url2', ....] # this should be the list of urls 

    requests = (grequests.get(u) for u in urls) 
    responses = grequests.map(requests) 
    for response in responses: 
     if 199 < response.status_code < 400: 
      name = generate_file_name() # generate some name for your image file with extension like example.jpg 
      with open(name, 'wb') as f: # or save to S3 or something like that 
        f.write(response.content) 

这里只图像的下载将是平行的，但每个图像内容写入文件将是连续的，所以你可以创建一个线程或做其他事，使之并行或异步