可以使用 subprocess.Popen 为 python3.7 处理命令列表

Question

我想将应用于多个 url 的 curl 命令的内容写入一个文件，例如我有

 days = [f'from-{x+1}d-to-{x}'for x in range(5, 0, -1)]

 urls = [f'https:\example.com?{day}/data' for day in days]

 command = [f'curl {url}' for url in urls]

 command

['curl https:\example.com?from-6d-to-5/data', 'curl https:\example.com?from-5d-to-4/data', 'curl https:\example.com?from-4d-to-3/data', 'curl https:\example.com?from-3d-to-2/data', 'curl https:\example.com?from-2d-to-1/data']

我正在尝试将其全部写入一个文件：

content = subprocess.Popen(([x for x in command]), shell = True, text = True, stdout = subprocess.PIPE).communicate()
 file_name = open('file_1', 'a') 
 file_name.write(str(content))

但看起来 subprocess.Popen 只执行第一个 curl 命令，因为我在控制台中只能看到一个输出：

% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 12591 0 12591 0 0 20931 0 --:--:-- --:--:-- --:--:-- 20915

有没有办法用subprocess.Popen执行多个命令？我想控制台输出的数量应该与要卷曲的网址数量相同

Answer 1

可以在Python内直接使用urllib，一般不需要使用subprocess/curl:

import urllib.request

days = [f'from-{x+1}d-to-{x}'for x in range(5, 0, -1)]
urls = [f'https://example.com?{day}/data' for day in days]

for url in urls:
    with urllib.request.urlopen(url) as response:
       print(response.read())

Answer 2

来自官方文档：

subprocess.Popen(args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=True, shell=False, cwd=None, env=None, universal_newlines=None, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None, text=None)

Execute a child program in a new process. On POSIX, the class uses os.execvp()-like behavior to execute the child program. On Windows, the class uses the Windows CreateProcess() function. The arguments to Popen are as follows.

args should be a sequence of program arguments or else a single string or path-like object. By default, the program to execute is the first item in args if args is a sequence. If args is a string, the interpretation is platform-dependent and described below. See the shell and executable arguments for additional differences from the default behavior. Unless otherwise stated, it is recommended to pass args as a sequence.

Popen() 期望只创建一个子进程。因此，考虑第一个命令，其他命令可能被视为额外参数。

正如@Maurice 所回答的那样，您可以使用 urllib 来获得 URL 响应。如果您仍想为此目的使用子流程，则可能需要进行更改，例如

responses = [str(subprocess.Popen(x.split(" "), shell = True, text = True, stdout = subprocess.PIPE).communicate()) for x in commands]

file_name = open('file_1', 'a')
file_name.writelines(responses)

如果您有很多 URL 要处理，这可能不是一个好的选择。

可以使用 subprocess.Popen 为 python3.7 处理命令列表

Possibility to process a list of commands with subprocess.Popen for python3.7

python

curl

subprocess

popen