使用管道在同一进程上执行多个 shell 命令时读取死锁
Read deadlock when using pipes to execute multiple shell commands on the same process
我正在制作一个 C++ 程序，需要在同一个 bash shell 实例中 运行 多个命令。 我需要这个，因为一些命令正在设置需要由后续命令读取的 bash 变量。
我正在使用 pipes to make file descriptors that are then read and written to using read and write, the other ends of these pipes are connected to a child that is made using fork。
当命令没有 return 输出时会出现问题，例如设置 bash 变量。 在下面的代码中，读取将永远挂在命令号 2 上。我已经搜索了几天，似乎没有办法在不关闭某处管道的情况下检测命令何时完成 运行ning。我相信如果我关闭管道，我将无法重新打开它，这意味着我需要制作一个没有加载变量的新 bash shell。
此外，我无法确定哪些命令不会 return 输出，因为此代码将从网络服务器 运行 获取它需要的命令，并希望避免串联带有“&&”的命令用于精细的错误报告。
#include <unistd.h>
#include <fcntl.h>
#include <cstdlib>
#include <string>
#include <iostream>
using namespace std;
int main(int argc, char *argv[])
{
int inPipeFD[2];
int outPipeFD[2];
// Create a read and write pipe for communication with the child process
pipe(inPipeFD);
pipe(outPipeFD);
// Set the read pipe to be blocking
fcntl(inPipeFD[0], F_SETFL, fcntl(inPipeFD[0], F_GETFL) & ~O_NONBLOCK);
fcntl(inPipeFD[1], F_SETFL, fcntl(inPipeFD[1], F_GETFL) & ~O_NONBLOCK);
// Create a child to run the job commands in
int pid = fork();
if(pid == 0) // Child
{
// Close STDIN and replace it with outPipeFD read end
dup2(outPipeFD[0], STDIN_FILENO);
// Close STDOUT and replace it with inPipe read end
dup2(inPipeFD[1], STDOUT_FILENO);
system("/bin/bash");
}
else // Parent
{
// Close the read end of the write pipe
close(outPipeFD[0]);
// Close the write end of the read pipe
close(inPipeFD[1]);
}
// Command 1
char buf[256];
string command = "echo test\n";
write(outPipeFD[1], command.c_str(), command.length());
read(inPipeFD[0], buf, sizeof(buf));
cout << buf << endl;
// Command 2
char buf2[256];
command = "var=worked\n";
write(outPipeFD[1], command.c_str(), command.length());
read(inPipeFD[0], buf2, sizeof(buf2));
cout << buf2 << endl;
// Command 3
char buf3[256];
command = "echo $var\n";
write(outPipeFD[1], command.c_str(), command.length());
read(inPipeFD[0], buf3, sizeof(buf3));
cout << buf3 << endl;
}
有没有办法在不关闭管道的情况下检测到子命令已经完成？
一种解决方案是将 bash 设置为交互模式，方法是使用 system("/bin/bash -i"); 启动它，并将提示设置为最后一个命令的退出代码。
首先，一个让书写和阅读更简单的便捷功能：
std::string command(int write_fd, int read_fd, std::string cmd) {
write(write_fd, cmd.c_str(), cmd.size());
cmd.resize(1024); // turn cmd into a buffer
auto len = read(read_fd, cmd.data(), cmd.size());
if(len == -1) len = 0;
cmd.resize(static_cast<std::size_t>(len));
return cmd;
}
那么在你的父进程中：
sleep(1); // ugly way to make reasonably sure the child has started bash
int& out = outPipeFD[1]; // for convenience
int& in = inPipeFD[0]; // for convenience
// first, set the prompt
std::cout << command(out, in, "export PS1='$?\n'\n") << '\n';
// then all these will print something
std::cout << command(out, in, "echo test\n") << '\n';
std::cout << command(out, in, "var=worked\n") << '\n';
std::cout << command(out, in, "echo $var\n") << '\n';
这样你总能有东西可读 - 你也可以用它来验证命令是否正确执行。
如果您的 bash 需要一个处于 -i(nteractive) 模式的真实终端，我们必须在没有它的情况下进行。想法：
- 为每个发送的命令添加
echo $? + 分隔符
- 将管道设置为非阻塞模式以便能够捕获杂项。糟糕的情况，例如发送命令
exit。
- 读取直到找到分隔符或发生错误。
为了使定界符难以猜测（不能轻易强制读取不同步），我会为每个命令生成一个新的定界符。
下面是一个示例，说明将这些想法与内联评论结合起来会是什么样子：
#include <fcntl.h>
#include <unistd.h>
#include <sys/select.h>
#include <algorithm>
#include <cstdlib>
#include <iostream>
#include <random>
#include <sstream>
#include <utility>
#include <vector>
// a function to generate a random string to use as a delimiter
std::string generate_delimiter() {
thread_local std::mt19937 prng(std::random_device{}());
thread_local std::uniform_int_distribution dist('a', 'z');
thread_local auto gen = [&]() { return dist(prng); };
std::string delimiter(128, 0);
std::generate(delimiter.begin(), delimiter.end(), gen);
return delimiter;
}
// custom exit codes for the command function
enum exit_status_t {
ES_WRITE_FAILED = 256,
ES_READ_FAILED,
ES_EXIT_STATUS_NOT_FOUND
};
// a function executing a command and returning the output and exit code
std::pair<std::vector<std::string>, exit_status_t> command(int write_fd,
int read_fd,
std::string cmd) {
constexpr size_t BufSize = 1024;
// a string that is unlikely to show up in the output:
const std::string delim = generate_delimiter() + "\n";
cmd += "\necho -e $?\"\n\"" + delim; // add echoing of status code
auto len = write(write_fd, cmd.c_str(), cmd.size()); // send the commands
if(len <= 0) return {{}, ES_WRITE_FAILED}; // couldn't write, return
cmd.resize(0); // use cmd to collect all read data
std::string buffer(BufSize, 0);
// a loop to extract all data until the delimiter is found
fd_set read_set{};
FD_SET(read_fd, &read_set);
while(true) {
// wait until something happens on the pipe
select(read_fd + 1, &read_set, nullptr, nullptr, nullptr);
if((len = read(read_fd, buffer.data(), buffer.size())) <= 0) {
// Failed reading - pipe probably closed on the other side.
// Add a custom exit code and the delimiter and break out.
cmd += "\n" + std::to_string(ES_READ_FAILED) + "\n" + delim;
break;
}
// append what was read to cmd
cmd.append(buffer.begin(), buffer.begin() + len);
// break out of the loop if we got the delimiter
if(cmd.size() >= delim.size() &&
cmd.substr(cmd.size() - delim.size()) == delim)
{
break;
}
}
cmd.resize(cmd.size() - delim.size()); // remove the delimiter
// put what was read in an istringstream for parsing
std::istringstream is(cmd);
// extract line by line
std::vector<std::string> output;
while(std::getline(is, cmd)) {
output.push_back(cmd);
}
// extract the exit code at the last line
exit_status_t retval = ES_EXIT_STATUS_NOT_FOUND;
if(not output.empty()) { // should never be empty but ...
retval = static_cast<exit_status_t>(std::stoi(output.back(), nullptr));
output.resize(output.size() - 1);
}
return {output, retval}; // return the pair
}
试驾：
int main() {
int inPipeFD[2];
int outPipeFD[2];
// Create a read and write pipe for communication with the child process
pipe(inPipeFD);
pipe(outPipeFD);
// Set the read pipe to be non-blocking
fcntl(inPipeFD[0], F_SETFL, fcntl(inPipeFD[0], F_GETFL) | O_NONBLOCK);
fcntl(inPipeFD[1], F_SETFL, fcntl(inPipeFD[1], F_GETFL) | O_NONBLOCK);
// Create a child to run the job commands in
int pid = fork();
if(pid == 0) // Child
{
// Close STDIN and replace it with outPipeFD read end
dup2(outPipeFD[0], STDIN_FILENO);
close(outPipeFD[0]); // not needed anymore
// Close STDOUT and replace it with inPipe read end
dup2(inPipeFD[1], STDOUT_FILENO);
close(inPipeFD[1]); // not needed anymore
// execl() is cleaner than system() since it replaces the process
// completely. Use /bin/sh instead if you'd like.
execl("/bin/bash", "bash", nullptr);
return 1; // to not run the parent code in case execl fails
}
// Parent
// Close the read end of the write pipe
close(outPipeFD[0]);
// Close the write end of the read pipe
close(inPipeFD[1]);
sleep(1);
int& out = outPipeFD[1]; // for convenience
int& in = inPipeFD[0]; // for convenience
// a list of commands, including an erroneous command(foobar) + exit
for(std::string cmd : {"echo test", "var=worked", "echo $var", "foobar", "exit"})
{
std::cout << "EXECUTING COMMAND: " << cmd << '\n';
auto [output, exit_status] = command(out, in, cmd);
// print what was returned
for(auto str : output) std::cout << str << '\n';
std::cout << "(exit status=" << exit_status << ")\n";
}
}
可能的输出：
EXECUTING COMMAND: echo test
test
(exit status=0)
EXECUTING COMMAND: var=worked
(exit status=0)
EXECUTING COMMAND: echo $var
worked
(exit status=0)
EXECUTING COMMAND: foobar
bash: line 7: foobar: command not found
(exit status=127)
EXECUTING COMMAND: exit
(exit status=257)
我正在制作一个 C++ 程序，需要在同一个 bash shell 实例中 运行 多个命令。 我需要这个，因为一些命令正在设置需要由后续命令读取的 bash 变量。
我正在使用 pipes to make file descriptors that are then read and written to using read and write, the other ends of these pipes are connected to a child that is made using fork。
当命令没有 return 输出时会出现问题，例如设置 bash 变量。 在下面的代码中，读取将永远挂在命令号 2 上。我已经搜索了几天，似乎没有办法在不关闭某处管道的情况下检测命令何时完成 运行ning。我相信如果我关闭管道，我将无法重新打开它，这意味着我需要制作一个没有加载变量的新 bash shell。
此外，我无法确定哪些命令不会 return 输出，因为此代码将从网络服务器 运行 获取它需要的命令，并希望避免串联带有“&&”的命令用于精细的错误报告。
#include <unistd.h>
#include <fcntl.h>
#include <cstdlib>
#include <string>
#include <iostream>
using namespace std;
int main(int argc, char *argv[])
{
int inPipeFD[2];
int outPipeFD[2];
// Create a read and write pipe for communication with the child process
pipe(inPipeFD);
pipe(outPipeFD);
// Set the read pipe to be blocking
fcntl(inPipeFD[0], F_SETFL, fcntl(inPipeFD[0], F_GETFL) & ~O_NONBLOCK);
fcntl(inPipeFD[1], F_SETFL, fcntl(inPipeFD[1], F_GETFL) & ~O_NONBLOCK);
// Create a child to run the job commands in
int pid = fork();
if(pid == 0) // Child
{
// Close STDIN and replace it with outPipeFD read end
dup2(outPipeFD[0], STDIN_FILENO);
// Close STDOUT and replace it with inPipe read end
dup2(inPipeFD[1], STDOUT_FILENO);
system("/bin/bash");
}
else // Parent
{
// Close the read end of the write pipe
close(outPipeFD[0]);
// Close the write end of the read pipe
close(inPipeFD[1]);
}
// Command 1
char buf[256];
string command = "echo test\n";
write(outPipeFD[1], command.c_str(), command.length());
read(inPipeFD[0], buf, sizeof(buf));
cout << buf << endl;
// Command 2
char buf2[256];
command = "var=worked\n";
write(outPipeFD[1], command.c_str(), command.length());
read(inPipeFD[0], buf2, sizeof(buf2));
cout << buf2 << endl;
// Command 3
char buf3[256];
command = "echo $var\n";
write(outPipeFD[1], command.c_str(), command.length());
read(inPipeFD[0], buf3, sizeof(buf3));
cout << buf3 << endl;
}
有没有办法在不关闭管道的情况下检测到子命令已经完成？
一种解决方案是将 bash 设置为交互模式，方法是使用 system("/bin/bash -i"); 启动它，并将提示设置为最后一个命令的退出代码。
首先，一个让书写和阅读更简单的便捷功能：
std::string command(int write_fd, int read_fd, std::string cmd) {
write(write_fd, cmd.c_str(), cmd.size());
cmd.resize(1024); // turn cmd into a buffer
auto len = read(read_fd, cmd.data(), cmd.size());
if(len == -1) len = 0;
cmd.resize(static_cast<std::size_t>(len));
return cmd;
}
那么在你的父进程中：
sleep(1); // ugly way to make reasonably sure the child has started bash
int& out = outPipeFD[1]; // for convenience
int& in = inPipeFD[0]; // for convenience
// first, set the prompt
std::cout << command(out, in, "export PS1='$?\n'\n") << '\n';
// then all these will print something
std::cout << command(out, in, "echo test\n") << '\n';
std::cout << command(out, in, "var=worked\n") << '\n';
std::cout << command(out, in, "echo $var\n") << '\n';
这样你总能有东西可读 - 你也可以用它来验证命令是否正确执行。
如果您的 bash 需要一个处于 -i(nteractive) 模式的真实终端，我们必须在没有它的情况下进行。想法：
- 为每个发送的命令添加
echo $?+ 分隔符 - 将管道设置为非阻塞模式以便能够捕获杂项。糟糕的情况，例如发送命令
exit。 - 读取直到找到分隔符或发生错误。
为了使定界符难以猜测（不能轻易强制读取不同步），我会为每个命令生成一个新的定界符。
下面是一个示例，说明将这些想法与内联评论结合起来会是什么样子：
#include <fcntl.h>
#include <unistd.h>
#include <sys/select.h>
#include <algorithm>
#include <cstdlib>
#include <iostream>
#include <random>
#include <sstream>
#include <utility>
#include <vector>
// a function to generate a random string to use as a delimiter
std::string generate_delimiter() {
thread_local std::mt19937 prng(std::random_device{}());
thread_local std::uniform_int_distribution dist('a', 'z');
thread_local auto gen = [&]() { return dist(prng); };
std::string delimiter(128, 0);
std::generate(delimiter.begin(), delimiter.end(), gen);
return delimiter;
}
// custom exit codes for the command function
enum exit_status_t {
ES_WRITE_FAILED = 256,
ES_READ_FAILED,
ES_EXIT_STATUS_NOT_FOUND
};
// a function executing a command and returning the output and exit code
std::pair<std::vector<std::string>, exit_status_t> command(int write_fd,
int read_fd,
std::string cmd) {
constexpr size_t BufSize = 1024;
// a string that is unlikely to show up in the output:
const std::string delim = generate_delimiter() + "\n";
cmd += "\necho -e $?\"\n\"" + delim; // add echoing of status code
auto len = write(write_fd, cmd.c_str(), cmd.size()); // send the commands
if(len <= 0) return {{}, ES_WRITE_FAILED}; // couldn't write, return
cmd.resize(0); // use cmd to collect all read data
std::string buffer(BufSize, 0);
// a loop to extract all data until the delimiter is found
fd_set read_set{};
FD_SET(read_fd, &read_set);
while(true) {
// wait until something happens on the pipe
select(read_fd + 1, &read_set, nullptr, nullptr, nullptr);
if((len = read(read_fd, buffer.data(), buffer.size())) <= 0) {
// Failed reading - pipe probably closed on the other side.
// Add a custom exit code and the delimiter and break out.
cmd += "\n" + std::to_string(ES_READ_FAILED) + "\n" + delim;
break;
}
// append what was read to cmd
cmd.append(buffer.begin(), buffer.begin() + len);
// break out of the loop if we got the delimiter
if(cmd.size() >= delim.size() &&
cmd.substr(cmd.size() - delim.size()) == delim)
{
break;
}
}
cmd.resize(cmd.size() - delim.size()); // remove the delimiter
// put what was read in an istringstream for parsing
std::istringstream is(cmd);
// extract line by line
std::vector<std::string> output;
while(std::getline(is, cmd)) {
output.push_back(cmd);
}
// extract the exit code at the last line
exit_status_t retval = ES_EXIT_STATUS_NOT_FOUND;
if(not output.empty()) { // should never be empty but ...
retval = static_cast<exit_status_t>(std::stoi(output.back(), nullptr));
output.resize(output.size() - 1);
}
return {output, retval}; // return the pair
}
试驾：
int main() {
int inPipeFD[2];
int outPipeFD[2];
// Create a read and write pipe for communication with the child process
pipe(inPipeFD);
pipe(outPipeFD);
// Set the read pipe to be non-blocking
fcntl(inPipeFD[0], F_SETFL, fcntl(inPipeFD[0], F_GETFL) | O_NONBLOCK);
fcntl(inPipeFD[1], F_SETFL, fcntl(inPipeFD[1], F_GETFL) | O_NONBLOCK);
// Create a child to run the job commands in
int pid = fork();
if(pid == 0) // Child
{
// Close STDIN and replace it with outPipeFD read end
dup2(outPipeFD[0], STDIN_FILENO);
close(outPipeFD[0]); // not needed anymore
// Close STDOUT and replace it with inPipe read end
dup2(inPipeFD[1], STDOUT_FILENO);
close(inPipeFD[1]); // not needed anymore
// execl() is cleaner than system() since it replaces the process
// completely. Use /bin/sh instead if you'd like.
execl("/bin/bash", "bash", nullptr);
return 1; // to not run the parent code in case execl fails
}
// Parent
// Close the read end of the write pipe
close(outPipeFD[0]);
// Close the write end of the read pipe
close(inPipeFD[1]);
sleep(1);
int& out = outPipeFD[1]; // for convenience
int& in = inPipeFD[0]; // for convenience
// a list of commands, including an erroneous command(foobar) + exit
for(std::string cmd : {"echo test", "var=worked", "echo $var", "foobar", "exit"})
{
std::cout << "EXECUTING COMMAND: " << cmd << '\n';
auto [output, exit_status] = command(out, in, cmd);
// print what was returned
for(auto str : output) std::cout << str << '\n';
std::cout << "(exit status=" << exit_status << ")\n";
}
}
可能的输出：
EXECUTING COMMAND: echo test
test
(exit status=0)
EXECUTING COMMAND: var=worked
(exit status=0)
EXECUTING COMMAND: echo $var
worked
(exit status=0)
EXECUTING COMMAND: foobar
bash: line 7: foobar: command not found
(exit status=127)
EXECUTING COMMAND: exit
(exit status=257)