使用多处理模块编写的代码没有给出任何输出
Code written using multiprocessing module is not giving any output
所以我正在尝试学习多处理模块并编写了一段代码(如下),其中生成了 4 个进程 并分配了 8 个作业(在处理器函数中)并且每个作业只包含一个睡眠函数(在示例作业函数中)。现在我写了类似的代码在 多线程模块 中,它工作正常,但在这里它没有输出任何 thing.Please 帮助
from multiprocessing import Process, Lock
import multiprocessing
import time
print_lock = Lock()
def exampleJob(worker): # function simulating some computation
time.sleep(.5)
print_lock.acquire()
print(multiprocessing.current_process.pid,worker)
print_lock.release()
def processor(): #function where process pick up the job
while True:
worker = q.get()
exampleJob(worker)
q.task_done()
q = multiprocessing.JoinableQueue()
process = []
for x in range(4):
p = multiprocessing.Process(target=processor)
process.append(p)
for i in range(0,len(process)):
process[i].start
start = time.time()
for worker in range(8):
q.put(worker)
q.join()
print('Entire job took:',time.time() - start)
第一个问题是start
需要start()
。
另外,不同的进程有不同的全局变量,所以print_lock = Lock()
是每个进程中不同的锁。您必须创建一次锁并将其传递给各个进程。这也适用于队列。
A JoinableQueue
并不是真正需要的。需要的是一个哨兵标志来告诉进程退出并加入进程。
其他修复的工作示例:
import multiprocessing as mp
import time
def exampleJob(print_lock,worker): # function simulating some computation
time.sleep(.5)
with print_lock:
print(mp.current_process().name,worker)
def processor(print_lock,q): # function where process pick up the job
while True:
worker = q.get()
if worker is None: # flag to exit the process
break
exampleJob(print_lock,worker)
# This "if" required for portability in some OSes.
# Windows for example creates new Python processes and imports the original script.
# Without this the below code would run again in each child process.
if __name__ == '__main__':
print_lock = mp.Lock()
q = mp.Queue()
processes = [mp.Process(target=processor,args=(print_lock,q)) for _ in range(4)]
for process in processes:
process.start() # OP code didn't *call* the start method.
start = time.time()
for worker in range(8):
q.put(worker)
for process in processes:
q.put(None) # quit indicator
for process in processes:
process.join()
print('Entire job took:',time.time() - start)
输出:
Process-2 2
Process-1 0
Process-3 1
Process-4 3
Process-3 6
Process-1 5
Process-2 4
Process-4 7
Entire job took: 1.1350018978118896
所以我正在尝试学习多处理模块并编写了一段代码(如下),其中生成了 4 个进程 并分配了 8 个作业(在处理器函数中)并且每个作业只包含一个睡眠函数(在示例作业函数中)。现在我写了类似的代码在 多线程模块 中,它工作正常,但在这里它没有输出任何 thing.Please 帮助
from multiprocessing import Process, Lock
import multiprocessing
import time
print_lock = Lock()
def exampleJob(worker): # function simulating some computation
time.sleep(.5)
print_lock.acquire()
print(multiprocessing.current_process.pid,worker)
print_lock.release()
def processor(): #function where process pick up the job
while True:
worker = q.get()
exampleJob(worker)
q.task_done()
q = multiprocessing.JoinableQueue()
process = []
for x in range(4):
p = multiprocessing.Process(target=processor)
process.append(p)
for i in range(0,len(process)):
process[i].start
start = time.time()
for worker in range(8):
q.put(worker)
q.join()
print('Entire job took:',time.time() - start)
第一个问题是start
需要start()
。
另外,不同的进程有不同的全局变量,所以print_lock = Lock()
是每个进程中不同的锁。您必须创建一次锁并将其传递给各个进程。这也适用于队列。
A JoinableQueue
并不是真正需要的。需要的是一个哨兵标志来告诉进程退出并加入进程。
其他修复的工作示例:
import multiprocessing as mp
import time
def exampleJob(print_lock,worker): # function simulating some computation
time.sleep(.5)
with print_lock:
print(mp.current_process().name,worker)
def processor(print_lock,q): # function where process pick up the job
while True:
worker = q.get()
if worker is None: # flag to exit the process
break
exampleJob(print_lock,worker)
# This "if" required for portability in some OSes.
# Windows for example creates new Python processes and imports the original script.
# Without this the below code would run again in each child process.
if __name__ == '__main__':
print_lock = mp.Lock()
q = mp.Queue()
processes = [mp.Process(target=processor,args=(print_lock,q)) for _ in range(4)]
for process in processes:
process.start() # OP code didn't *call* the start method.
start = time.time()
for worker in range(8):
q.put(worker)
for process in processes:
q.put(None) # quit indicator
for process in processes:
process.join()
print('Entire job took:',time.time() - start)
输出:
Process-2 2
Process-1 0
Process-3 1
Process-4 3
Process-3 6
Process-1 5
Process-2 4
Process-4 7
Entire job took: 1.1350018978118896