如何使用gevent执行“读取 - 搜索 - 写入”任务？

Question

我有任务试图交易如下：

1. 读本地文件
2. 线送线到Web服务查询
3. 结果写入到本地文件

`

def read(fin): 
    for query in fin: 
     return query 

def search(query): 
    # Send the query to something like Google Search 
    result = google(query) 
    return result 

def write(fout, result): 
    fout.write(result) 

`

我指的是Gevent教程，但无法弄清楚如何将这三个函数放入gevent中。

Answer 1

我不知道你的例子与gevent有关。无论如何，它仍然可以通过下面的示例代码进行处理。

import gevent 

def read(fin): 
    for query in fin: 
     return query 

def search(query): 
    # Send the query to something like Google Search 
    result = google(query) 
    return result 

def write(fout, result): 
    fout.write(result) 

# read from different files 
input_files = ["input1.json", "input2.json" ... ] 

# write result into output file 
fout = open("output.json", 'wb') 

# get queries from input files 
queries = [] 
for input_file in input_files: 
    with open(input_file, 'rb') as fin: 
     queries.append(gevent.joinall([gevent.spawn(read, fin)])) 

# store results of queries 
results = [] 
for item in [query for query in queries]: 
    results.append(gevent.joinall([gevent.spawn(search, item[0].value)])) 

# save results into output file 
for item in [result for result in results]: 
    gevent.joinall([gevent.spawn(write, fout, item[0].value)]) 

的事情是同时读取到的搜索和搜索到的写有依赖性，这是不能够使用异步的方式来实现这一目标。因此在这个例子中，它可以处理每个函数并一个一个地保存返回值。