多处理简单功能不起作用，但为什么

Question

我想多处理系统命令，但不能让它与一个简单的程序一起工作。该功能runit（CMD）工作正常，但...

#!/usr/bin/python3 
from subprocess import call, run, PIPE,Popen 
from multiprocessing import Pool 
import os 
pool = Pool() 

def runit(cmd): 
    proc = Popen(cmd, shell=True,stdout=PIPE, stderr=PIPE, universal_newlines=True) 
    return proc.stdout.read() 

#print(runit('ls -l')) 

it = [] 
for i in range(1,3): 
    it.append('ls -l') 

results = pool.map(runit, it) 

它输出：

Process ForkPoolWorker-1: 
Process ForkPoolWorker-2: 
Traceback (most recent call last): 
Traceback (most recent call last): 
    File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap 
    self.run() 
    File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run 
    self._target(*self._args, **self._kwargs) 
    File "/usr/lib/python3.5/multiprocessing/pool.py", line 108, in worker 
    task = get() 
    File "/usr/lib/python3.5/multiprocessing/queues.py", line 345, in get 
    return ForkingPickler.loads(res) 
AttributeError: Can't get attribute 'runit' on <module '__main__' from './syscall.py'> 
    File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap 
    self.run() 
    File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run 
    self._target(*self._args, **self._kwargs) 
    File "/usr/lib/python3.5/multiprocessing/pool.py", line 108, in worker 
    task = get() 
    File "/usr/lib/python3.5/multiprocessing/queues.py", line 345, in get 
    return ForkingPickler.loads(res) 
AttributeError: Can't get attribute 'runit' on <module '__main__' from './syscall.py'> 

然后以某种方式等待和什么也不做，当我按下Ctrl + C几次它吐出来：

^CProcess ForkPoolWorker-4: 
Process ForkPoolWorker-6: 
Traceback (most recent call last): 
    File "./syscall.py", line 17, in <module> 
Process ForkPoolWorker-5: 
    results = pool.map(runit, it) 
    File "/usr/lib/python3.5/multiprocessing/pool.py", line 260, in map 
... 
    buf = self._recv(4) 
    File "/usr/lib/python3.5/multiprocessing/connection.py", line 379, in _recv 
    chunk = read(handle, remaining) 
KeyboardInterrupt 

Answer 1

我不知道，因为我知道这个问题是Windows相关的（和我没有访问Linux中reprocude），但为了可以移植，你必须换你多依赖于的命令或者它的方式蟒蛇冲突滋生的过程：即固定例如运行良好的窗口（并应工作确定在其他平台上也一样）：

from multiprocessing import Pool 
import os 

def runit(cmd): 
    proc = Popen(cmd, shell=True,stdout=PIPE, stderr=PIPE, universal_newlines=True) 
    return proc.stdout.read() 

#print(runit('ls -l')) 

it = [] 
for i in range(1,3): 
    it.append('ls -l') 

if __name__=="__main__": 
    # all calls to multiprocessing module are "protected" by this directive 
    pool = Pool() 

（更紧密地学习中的错误消息，现在我敢肯定，只是移动pool = Pool()后的runit的声明将修复它，以及在Linux上，但包裹在__main__修复+使得便携式）

不过，请注意，您的多只创建一个新的进程，所以你用线程池（Threading pool similar to the multiprocessing Pool?）更好：创建进程的线程SES，像这样：

from multiprocessing.pool import ThreadPool # uses threads, not processes 
import os 

def runit(cmd): 
    proc = Popen(cmd, shell=True,stdout=PIPE, stderr=PIPE, universal_newlines=True) 
    return proc.stdout.read() 

it = [] 
for i in range(1,3): 
    it.append('ls -l') 

if __name__=="__main__": 
    pool = ThreadPool() # ThreadPool instead of Pool 
    results = pool.map(runit, it) 
    print(results) 
     results = pool.map(runit, it) 
     print(results) 

后者溶液是更轻巧，并且不太问题倾向（多是一个微妙的模块来处理）。您将能够使用对象，共享数据等......而无需Manager对象，其他优势