如何将 3D Python/NumPy 数组保存为文本文件？

Question

我要进行大量的计算，每次都要保存一个2D文件文本，所以我想将结果存储在"real-time"中作为一个3D文本文件，每个切片对应一个计算结果。

第一次计算没问题，但是当我进行第二次计算时，在"np.loadtxt"这一步中，数组维度变成了二维...所以我达不到我的目标...但我可以当我开始调整尺寸 (... , ... , 1)

时进行重塑

#MY FIRST RESULTS
test1 = open("C:/test.txt", "r")
test_r = np.genfromtxt(test, skip_header=1)
test_r = np.expand_dims(test_r, axis=2) #I create a new axis to save in 3D
test1.close()

#I test if the "Store" file to keep all my results is created.
try:
    Store= np.loadtxt('C:/Store.txt')
except:
    test=1

#If "Store" is not created, I do it or I concatenate in my file.
if test ==1:
    Store= test_r
    np.savetxt('C:/Store.txt', Store)
    test=2
else:
    Store = np.concatenate((Store,test_r), axis=2)
    np.savetxt('C:/Store.txt', Store)


#MY SECOND RESULTS
test2 = open("C:/test.txt", "r")
test_r = np.genfromtxt(test, skip_header=1)
test_r = np.expand_dims(test_r, axis=2)
test2.close()

#I launch the procedure again to "save" the results BUT DURING THE LOADTXT STEP THE ARRAY DIMENSIONS CHANGE TO BECOME A 2D ARRAY...
try:
    Store= np.loadtxt('C:/Store.txt')
except:
    test=1


if test ==1:
    Store= test_r
    np.savetxt('C:/Store.txt', Store)
    test=2
else:
    Store = np.concatenate((Store,test_r), axis=2)
    np.savetxt('C:/Store.txt', Store)

Answer 1

这里有一个 cPickle 的例子：

import cPickle

# write to cPickle
cPickle.dump( thing_to_save, open( "filename.pkl", "wb" ) )

# read from cPickle
thing_to_save = cPickle.load( open( "filename.pkl", "rb" ) )

open() 函数的 "wb" 和 "rb" 参数很重要。 CPickle 以二进制格式写入对象，因此仅使用 "w" 和 "r" 是行不通的。

Answer 2

如果保存文件需要是'csv'风格的文本，你可以使用多个savetxt和loadtxt。关键是知道这两者都可以将打开的文件作为输入。

写作范例：

In [31]: A=np.arange(3*2*4).reshape(3,2,4)    
In [32]: A    # normal display as 3 blocks of 2d array
Out[32]: 
array([[[ 0,  1,  2,  3],
        [ 4,  5,  6,  7]],

       [[ 8,  9, 10, 11],
        [12, 13, 14, 15]],

       [[16, 17, 18, 19],
        [20, 21, 22, 23]]])

In [36]: for a in A:print a, '\n'   # or iterate on the 1st dimension
[[0 1 2 3]
 [4 5 6 7]] 

[[ 8  9 10 11]
 [12 13 14 15]] 

[[16 17 18 19]
 [20 21 22 23]]

按照该示例，我可以迭代文件，对每个子数组使用 savetxt：

In [37]: with open('3dcsv.txt','wb') as f:
    for a in A:
        np.savetxt(f, a, fmt='%10d')
        f.write('\n')
   ....:

使用系统cat确认文件写入（通过ipython）：

In [38]: cat 3dcsv.txt
         0          1          2          3
         4          5          6          7

         8          9         10         11
        12         13         14         15

        16         17         18         19
        20         21         22         23

为了简单阅读，loadtxt 显然忽略了空行，返回一个 6 x 4 数组。所以我知道它应该是 (2,3,4) 我可以轻松地重塑结果。

In [39]: np.loadtxt('3dcsv.txt')
Out[39]: 
array([[  0.,   1.,   2.,   3.],
       [  4.,   5.,   6.,   7.],
       [  8.,   9.,  10.,  11.],
       [ 12.,  13.,  14.,  15.],
       [ 16.,  17.,  18.,  19.],
       [ 20.,  21.,  22.,  23.]])

经过一些调试后，我让这个多重 loadtxt 开始工作。 loadtxt（和 genfromtxt）处理行列表。

In [53]: A1=[]     # list to collect blocks

In [54]: with open('3dcsv.txt') as f:
    lines = []     # list to collect lines
    while 1:
        aline = f.readline()
        if aline.strip():
            lines.append(aline)     # nonempty line
        else:              # empty line
            if len(lines)==0: break
            A1.append(np.loadtxt(lines, dtype=int))
            lines = []
   ....:             

In [55]: A1 = np.array(A1)

In [56]: A1
Out[56]: 
array([[[ 0,  1,  2,  3],
        [ 4,  5,  6,  7]],

       [[ 8,  9, 10, 11],
        [12, 13, 14, 15]],

       [[16, 17, 18, 19],
        [20, 21, 22, 23]]])

这可能不是最强大的组合 save/load，但它提供了构建更好的东西的框架。

但是如果不需要是文本那么pickle就可以了，原生的numpy也是'save/load'

In [57]: np.save('3dsave.npy',A)

In [58]: np.load('3dsave.npy')
Out[58]: 
array([[[ 0,  1,  2,  3],
        [ 4,  5,  6,  7]],

       [[ 8,  9, 10, 11],
        [12, 13, 14, 15]],

       [[16, 17, 18, 19],
        [20, 21, 22, 23]]])

如何将 3D Python/NumPy 数组保存为文本文件？

How do I save a 3D Python/NumPy array as a text file?

python

arrays

text

numpy

concatenation