一段时间后NodeJS exec停止而没有错误

Question

我正在使用 child_process 的 exec。

函数运行没问题，但是 4-5 分钟后，它就停止了，没有报告任何错误，但是脚本应该运行至少 24 小时...

这是代码：

import { exec } from 'child_process';

function searchDirectory(dirPath) {
    let lineBuffer = '';

    const cmd = `find ${dirPath} -type f -name "*.txt" | pv -l -L 10 -q`;
    const findData = exec(cmd);

    findData.on('error', err => log.error(err));
    findData.stdout.on('data', data => {
        lineBuffer += data;
        let lines = lineBuffer.split('\n');
        for (var i = 0; i < lines.length - 1; i++) {
            let filepath = lines[i];

            processfile(filepath);
        }
        lineBuffer = lines[lines.length - 1];
    });
    findData.stdout.on('end', () => console.log('finished finding...'));
}

pv 命令会减慢输出速度，我需要这个，因为我找到的路径是在网络上并且非常慢 (60mb/s)。

当我直接在终端中运行命令时，它工作正常（我没有等 24 小时，但我让它等了半小时，它仍然是运行ning）。

processfile 函数实际上使用 axios 进行异步调用以将一些数据发送到服务器：

    let data = readFileSync(file);
    ...
    axios.post(API_URL, { obj: data }, { proxy: false })
        .then(res => {
            log.info('Successfully saved object : ' + res.data._id);
        })
        .catch(err => {
            log.error(err.response ? err.response.data : err);
        });

什么会导致脚本停止？有什么想法吗？

谢谢

Answer 1

我发现了问题，不建议将 exec 用于大量输出，因为它使用的缓冲区大小有限。使用 spawn 代替：

The most significant difference between child_process.spawn and child_process.exec is in what they return - spawn returns a stream and exec returns a buffer.

child_process.spawn returns an object with stdout and stderr streams. You can tap on the stdout stream to read data that the child process sends back to Node. stdout being a stream has the "data", "end", and other events that streams have. spawn is best used to when you want the child process to return a large amount of data to Node - image processing, reading binary data etc.

child_process.exec returns the whole buffer output from the child process. By default the buffer size is set at 200k. If the child process returns anything more than that, you program will crash with the error message "Error: maxBuffer exceeded". You can fix that problem by setting a bigger buffer size in the exec options. But you should not do it because exec is not meant for processes that return HUGE buffers to Node. You should use spawn for that. So what do you use exec for? Use it to run programs that return result statuses, instead of data.

来自 : https://www.hacksparrow.com/difference-between-spawn-and-exec-of-node-js-child_process.html

一段时间后NodeJS exec停止而没有错误

NodeJS exec stop after some time without error

javascript

bash

file

exec

node.js