如何将返回值（从上一个函数）读入pandas、python？

Question

在下面的程序中

我想access/pipe下游函数中一个函数的数据。

使用 python 代码如下：

def main():
data1, data2, data3 = read_file()
do_calc(data1, data2, data3)   

def read_file():
    data1 = ""
    data2 = ""
    data3 = ""

    file1 = open('file1.txt', 'r+').read()
    for line in file1
        do something....
        data1 += calculated_values

    file2 = open('file2.txt', 'r+').read()
    for line in file1
        do something...
        data2 += calculated_values    

    file1 = open('file1.txt', 'r+').read()
    for line in file1
        do something...
        data3 += calculated_values

    return data1, data2, data3

def do_calc(data1, data2, data3):
    d1_frame = pd.read_table(data1, sep='\t')
    d2_frame = pd.read_table(data2, sep='\t')
    d3_frame = pd.read_table(data3, sep='\t')

    all_data = [d1_frame, d2_frame, d3_frame]

main()

给定的代码有什么问题？熊猫似乎无法正确读取输入文件，但正在将数据 1、2 和 3 的值打印到屏幕上。

read_hdf 似乎读取了文件但不正确。有没有一种方法可以将函数返回的数据直接读入 pandas（无需将 writing/reading 读入文件）。

错误信息：

Traceback (most recent call last):

  File "calc.py", line 757, in <module>

    main()

  File "calc.py", line 137, in main

    merge_tables(pop1_freq_table, pop2_freq_table, f1_freq_table)

  File "calc.py", line 373, in merge_tables

    df1 = pd.read_table(pop1_freq_table, sep='\t')

  File "/home/everestial007/.local/lib/python3.5/site-packages/pandas/io/parsers.py", line 645, in parser_f

    return _read(filepath_or_buffer, kwds)

  File "/home/everestial007/.local/lib/python3.5/site-packages/pandas/io/parsers.py", line 388, in _read

    parser = TextFileReader(filepath_or_buffer, **kwds)

  File "/home/everestial007/.local/lib/python3.5/site-packages/pandas/io/parsers.py", line 729, in __init__

    self._make_engine(self.engine)

  File "/home/everestial007/.local/lib/python3.5/site-packages/pandas/io/parsers.py", line 922, in _make_engine

    self._engine = CParserWrapper(self.f, **self.options)

  File "/home/everestial007/.local/lib/python3.5/site-packages/pandas/io/parsers.py", line 1389, in __init__

    self._reader = _parser.TextReader(src, **kwds)

  File "pandas/parser.pyx", line 373, in pandas.parser.TextReader.__cinit__ (pandas/parser.c:4019)

  File "pandas/parser.pyx", line 665, in pandas.parser.TextReader._setup_parser_source (pandas/parser.c:7967)

FileNotFoundError: File b'0.667,0.333\n2\t15800126\tT\tT,A\t0.667,0.333\n2\t15800193\tC\tC,T\t0.667,0.333\n2\t15800244\tT\tT,C\......

如有任何解释，我将不胜感激。

Answer 1

pd.read_table(data1, sep='\t') 将 data1 视为文件路径，因为它没有 read 方法。您可以在堆栈跟踪中看到它尝试打开一个名为 csv 文件内容的文件。

来自read_table帮助：

Parameters
--------
filepath_or_buffer : str, pathlib.Path, py._path.local.LocalPath or any
object with a read() method (such as a file handle or StringIO)

您应该将其转换为 io.StringIO 对象以便可以读取

快速修复：

pd.read_table(io.StringIO(data1), sep='\t')

但这会创建数据的副本。最好的解决方法是直接创建 io.StringIO 缓冲区：

def read_file():
    data1 = io.StringIO()


    file1 = open('file1.txt', 'r+').read()
    for line in file1
        do something....
        data1.write(calculated_values)

    # in the end
    data1.seek(0)  # reset to start of "file"

如何将返回值（从上一个函数）读入pandas、python？

How to read the returned value (from previous function) into pandas, python?

python

stdin

main

function

pandas