如何在matlab中缓冲CSV文件中的数据？

Question

我有 CSV 文件 " 3000 raw" 需要将其分成块但需要先缓冲

详细步骤

在buffer中逐个读取raw直到达到1000 raw然后将是chunk1
在 chunk1 上进行处理
从缓冲区中删除 1000

我不知道如何将数据 "row by row" 放入 matlab 的缓冲区

谢谢

Answer 1

版本 1

正在读取文件line-by-line。

buffer_size = 1000;
buffer = strings([buffer_size,1]);

filename = fullfile(pwd, 'table.csv');
file_id = fopen(filename);

buffer_index = 1;
next_line = fgetl(file_id);

while(ischar(next_line))

    % Put next line into buffer.
    buffer(buffer_index) = next_line;

    % Is the buffer full?
    if buffer_index == buffer_size
        % Process buffer
        disp(buffer);

        % Clear the buffer.
        buffer = strings([buffer_size,1]);
    end

    % Advance to next line.
    buffer_index = buffer_index + 1;

    % Wrap around to start of buffer if necessary and adjust for one-based indexing.
    buffer_index = mod((buffer_index - 1), buffer_size) + 1; 

    % Read next line from file.
    next_line = fgetl(file_id);
end

fclose(file_id);

版本 2

将 CSV 文件转换为 XLSB（快速二进制格式 XLSX）以利用 readtable，然后您可以这样做：

chunks = 3;
chunk_size = 1000;
first_column = 'A';
last_column = 'Z';

my_filename = fullfile(pwd, 'MyCsvTable.xlsb');

for chunk = 1:chunks
    chunk_begin = (chunk - 1) * chunk_size + 1;
    chunk_end = chunk_begin + chunk_size - 1;
    range = sprintf("%s%d:%s%d", first_column, chunk_begin, last_column, chunk_end);
    my_table_chunk = readtable(my_filename, 'ReadVariableNames', false, 'Range', range)

    % Process my_table_chunk

end

如何在matlab中缓冲CSV文件中的数据？

how to buffer data from CSV file in matlab?

csv

matlab

buffer

版本 1

版本 2