多处理 imap 抑制子进程打印

Question

虽然我在 Jupyter notebook (python3.5) 中运行以下代码：

def process(doc):
    print('Process Hey!')
with Pool() as pool:
    results = pool.imap(process, docs)

这不会给笔记本打印，但如果我尝试使用 pool.map 代替，它会吐出文字。所以我真的很好奇这里发生了什么。谢谢！

Answer 1

来自文档 imap A lazier version of map()、imap returns 一个迭代器 map returns 一个列表：

In [24]:  with Pool() as pool:
            results = pool.map(process, docs)
            print(type(results))
   ....:     
Process Hey!
Process Hey!
<class 'list'>


In [25]:  with Pool() as pool:
                results = pool.imap(process, docs)
                print(type(results))
   ....:     
<class 'multiprocessing.pool.IMapIterator'>

In [27]:  with Pool() as pool:
                results = pool.imap(process, docs)
                for _ in results:
                     pass
   ....:     
Process Hey!
Process Hey!
In [28]:  with Pool() as pool:
            results = pool.imap(process, docs)
            list(results)
 ....:     
Process Hey!
Process Hey!

相当于python2中itertools.imap和map的区别，即惰性求值与贪心求值：

In [3]: from itertools import imap

In [4]: r = imap(lambda x: x+1, [1,2,3])
In [4]: r
Out[5]: <itertools.imap at 0x7f09170f1e10>
In [6]: list(r)
Out[6]: [2, 3, 4]

In [7]: r = map(lambda x: x+1, [1,2,3])    
In [8]: r
Out[8]: [2, 3, 4]

多处理 imap 抑制子进程打印

multiprocessing imap suprress child process print

python

multiprocessing