如何将 bash 命令的输出写入 csv 文件？

Question

我目前在 bash 中使用以下循环跟踪我机器的内存使用情况：

echo "      date     time $(free -m | grep total | sed -E 's/^    (.*)//g')" | tee -a log20201113.txt
while true; do
    echo "$(date '+%Y-%m-%d %H:%M:%S') $(free -m | grep Mem: | sed 's/Mem://g')" | tee -a log20201113.txt
    sleep 1
done

这给我一个 txt 文件，里面有类似的东西（修改后的值）：

      date     time          total       used       free     shared    buffers     cached
2020-11-13 22:55:56           8333        627       5705          0       1638        685
2020-11-13 22:55:57           8333        677       5656          0       1638        685
2020-11-13 22:55:58           8333        725       5607          0       1638        685
2020-11-13 22:55:59           8333        773       5560          0       1638        685

我正在尝试将其输出到 csv 文件。放置 csv 扩展名似乎不起作用，因为它不包含分隔符。我如何包含分隔符并将其放入 csv 文件中？

Answer 1

在 sed 中尝试管道：

echo "      date     time $(free -m | grep total | sed -E 's/^    (.*)//g')" | tee -a log20201113.txt
while true; do
    echo "$(date '+%Y-%m-%d %H:%M:%S') $(free -m | grep Mem: | sed 's/Mem://g')" | tee -a log20201113.txt
    sleep 1
done | sed "s/\s\+/,/g"

sed 将用逗号替换空格，然后你就可以得到 csv。

您可以直接从文本文件中执行以下操作：

sed -i "s/\s\+/,/g" yourfile.txt, 它将用逗号替换所有空格。

然后你得到

2020-11-14,17:42:42,7748,2190,729,601,4828,4490
2020-11-14,17:42:43,7748,2189,733,598,4825,4494

Answer 2

在 tee 之前通过管道传输到 awk '{=}1' OFS="," 应该足以转换为 csv。例如：

echo "      date     time $(free -m | grep total | sed -E 's/^    (.*)//g')" | awk '{=}1' OFS="," | tee -a log20201113.csv
while true; do
    echo "$(date '+%Y-%m-%d %H:%M:%S') $(free -m | grep Mem: | sed 's/Mem://g')" | awk '{=}1' OFS="," | tee -a log20201113.csv
    sleep 1
done

产生：

$ head -n 5 log20201113.csv
date,time,total,used,free,shared,buff/cache,available
2020-11-14,17:30:04,3638,1663,765,177,1208,1561
2020-11-14,17:30:05,3638,1663,765,177,1208,1561
2020-11-14,17:30:06,3638,1662,766,177,1208,1562
2020-11-14,17:30:07,3638,1662,767,177,1208,1562

当 awk 分配给字段 (=) 时，它将

causes [=27=] to be reconstructed by concatenating the $i's separated by OFS

其中 [=18=] 是输入行，$i 是字段，OFS 在我们的例子中是 ,。终止 1 只是一个“始终匹配”模式，它总是使用默认操作 (print) 执行。

但这也将以 csv 格式打印到 stdout。如果这是一个问题，并且您想在终端上保留格式良好的输出并在文件中保留 csv，您可以使用此 "trick":

echo "      date     time $(free -m | grep total | sed -E 's/^    (.*)//g')" | tee /dev/tty | awk '{=}1' OFS="," >> log20201113.csv
while true; do
    echo "$(date '+%Y-%m-%d %H:%M:%S') $(free -m | grep Mem: | sed 's/Mem://g')" | tee /dev/tty | awk '{=}1' OFS="," >> log20201113.csv
    sleep 1
done

Answer 3

这是 GNU awk 中的一个，它从 /proc/meminfo:

读取内存信息

$ gawk '
@load "time"                                  # sleep
BEGIN {
    OFS=","                                   # field separator
    printf "%s%s%s%s%s%s%s",                  # header
        "date time" OFS,
        "total" OFS,
        "used" OFS,
        "free" OFS,
        "shared" OFS,
        "buffers" OFS,
        "cached" ORS
    while(1) {                                # keep looping
        while((getline < "/proc/meminfo")>0)
            a[]=
        printf "%s%s%s%s%s%s%s",
            strftime("%F%T") OFS,
            a["MemTotal:"] OFS,
            a["MemTotal:"]-a["MemFree:"]-a["Buffers:"]-a["Cached:"]-a["SReclaimable"] OFS,
            a["MemFree:"] OFS,
            a["ShmemHugePages:"] OFS,
            a["Buffers:"] OFS,
            a["Buffers:"]+a["SReclaimable:"] ORS
        sleep(1)
    }
}'

输出：

date time,total,used,free,shared,buffers,cached
2020-11-1419:12:42,16135228,6586304,6713024,0,213652,362516
2020-11-1419:12:43,16135228,6586304,6713024,0,213652,362516
...

如何将 bash 命令的输出写入 csv 文件？

How to write the output of a bash command to a csv file?

memory

bash

memory-management