并行运行不同函数并按特定顺序获取返回值

Question

我想并行运行不同的函数（foo，bar和baz），然后按照它们的顺序获取每个函数返回的值调用。

我尝试了类似下面的代码，但出于某种原因，它在Windows上永远循环。使用例如multiprocessing可以实现吗？请注意，这些函数预期将参数作为同一事物（即数据）。

注意：下面的代码是针对Python 3的，我想要一个等效的Python 2.7，因为它是我需要使用的。

from concurrent.futures import ProcessPoolExecutor 
from operator import itemgetter 

def foo(data): 
    return "foo" 

def bar(data): 
    return "bar" 

def baz(data): 
    return "baz" 

work = [foo, bar, baz] 
data = [1,2,3,4,5] 

results = [] 

with ProcessPoolExecutor(max_workers=4) as pool: 
    for i, work_item in enumerate(work): 
     future = pool.submit(work_item, data) 
     def callback(ret): 
      results.append((i, ret.result())) 
     future.add_done_callback(callback) 

results.sort(key=itemgetter(0)) 
print(results) 

Answer 1

我改变了你的回调方法，实现了你想要的，我想。检查结果：

... 
with ProcessPoolExecutor(max_workers=4) as pool: 
    for i, work_item in enumerate(work): 
     future = pool.submit(work_item, data) 
     def callback(i): 
      def wrapper(ret): 
       print(ret, i) 
       results.append((i, ret.result())) 
      return wrapper 
     future.add_done_callback(callback(i)) 
... 

，结果如下：

<Future at 0x1057e4d30 state=finished returned str> 0 
<Future at 0x1057e4a58 state=finished returned str> 2 
<Future at 0x105779240 state=finished returned str> 1 
[(0, 'foo'), (1, 'bar'), (2, 'baz')] 

我运行与python3 Mac OS X中的代码。我希望它有帮助。

为蟒蛇2.7，你可以尝试这样的事：

from multiprocessing import Process, Manager 

def foo(data, i, results): 
    results[i] = "foo" 

def bar(data, i, results): 
    results[i] = "bar" 

def baz(data, i, results): 
    results[i] = "baz" 

work = [foo, bar, baz] 
data = [1,2,3,4,5] 

processes = [] 
results = Manager().dict() 
for i, w in enumerate(work): 
    p = Process(target=w, args=(data, i, results)) 
    processes.append(p) 
    p.start() 

for p in processes: 
    p.join() 

print results 

结果是：

{0: 'foo', 1: 'bar', 2: 'baz'} 

How can I recover the return value of a function passed to multiprocessing.Process?