将并行计算的结果移动到 Python 中的最终数组

Question

我有一个并行化的 python 代码，它计算 o[(i)] 的 x[(1,i)] 的值，其中 i 在 (0, npt) 范围内。由于x[(1,i)] = x1[(i)] * x2[(i)]，我分别计算x1[(i)]和x2[(i)]，然后将它们相乘得到x[(1,i)]。

import cmath, csv, sys, math, re
import numpy as np
import multiprocessing as mp

x1 = np.zeros(npt ,dtype=float)
x2 = np.zeros(npt ,dtype=float)

def chi2(i):
    print("\t wavelength", i+1," of ", npt)
    x1[(i)] = (some function of o[(i)]))

    for k in range(nk):
        for n in range(nb):
            for m in range(nb):
                for l in range(nb):
                        x2[(i)] = (x2[(i)] + (another function of o[(i)]))
    print(i,"x1:",x1[(i)])
    print(i,"x2:",x2[(i)])

    np.set_printoptions(precision=3)
    x[(1,i)] = x1[(i)] * x2[(i)]
    print(i,"x:",x[(1,i)])

    return x[(1,i)]

#-----------single process--------------
for i in range (npt):
    chi2(i)

#------------parallel processes-------------
#pool = mp.Pool(mp.cpu_count())
#pool.map(chi2,[i for i in range (npt)])
#pool.close() 
#-------------------------------------------

print("x:",x)

“并行处理”代码的输出如下：

wavelength 1  of  6
wavelength 2  of  6
wavelength 3  of  6
wavelength 4  of  6
wavelength 6  of  6
wavelength 5  of  6
1 x1: 13.064431907056434
1 x2: -1.9906250877567282
1 x: -26.006385911476013
5 x1: 32.428696460232054
5 x2: -7.62814423558251
5 x: -247.37077397057408
2 x1: 16.848860458915905
2 x2: -2.8743277048490476
2 x: -48.429146412197625
3 x1: 21.301496841785333
0 x1: 9.893885346287407
3 x2: -4.053590423587782
0 x2: -1.339636506591729
3 x: -86.34754360594641
0 x: -13.254210001919562
4 x1: 26.47666689558421
4 x2: -5.606053928481043
4 x: -148.42962246307385
x: [[1030. 1130. 1230. 1330. 1430. 1530.]
 [   0.    0.    0.    0.    0.    0.]]

并行代码能够获取 x1[(i)]、x1[(i)] 和 x[(1,i)] 的各个值，但无法将这些值放入完整数组 print("x:",x) .

因此我尝试使用单一进程，结果是我想要的：

         wavelength 1  of  6
0 x1: 9.893885346287407
0 x2: -1.339636506591729
0 x: -13.254210001919562
         wavelength 2  of  6
1 x1: 13.064431907056434
1 x2: -1.9906250877567282
1 x: -26.006385911476013
         wavelength 3  of  6
2 x1: 16.848860458915905
2 x2: -2.8743277048490476
2 x: -48.429146412197625
         wavelength 4  of  6
3 x1: 21.301496841785333
3 x2: -4.053590423587782
3 x: -86.34754360594641
         wavelength 5  of  6
4 x1: 26.47666689558421
4 x2: -5.606053928481043
4 x: -148.42962246307385
         wavelength 6  of  6
5 x1: 32.428696460232054
5 x2: -7.62814423558251
5 x: -247.37077397057408
x: [[1030.    1130.    1230.    1330.    1430.    1530.   ]
 [ -13.254  -26.006  -48.429  -86.348 -148.43  -247.371]]

谁能告诉我是什么阻止了并行代码将值输入 x[(1,i)]？

Answer 1

当运行多处理时，全局变量不会以这种方式在进程之间共享。每个进程都有自己的变量副本。

Pool.map return 是给定输入列表的每个函数的 return 值的列表。你可以做类似的事情（而不是你使用的 pool.map 行）：

x[1] = pool.map(chi2,[i for i in range (npt)])

使用chi2 的return 值并将其插入到x 中的正确位置。请注意，这使用了这样一个事实，即您可以将值列表分配到 numpy 数组的列或行中。

将并行计算的结果移动到 Python 中的最终数组

Moving results from parallel calculation to a final array in Python

python

parallel-processing

python-multiprocessing

python-pool