执行脚本的多个实例 - GNU Parallel
Executing multiple instances of script - GNU Parallel
我目前正在尝试使用 GNU Parallel, however I am not being successful at it. Installation was pretty easy. My goal is to run two parallel instances of python script youtube-dl 及其相应的参数。使用并行执行脚本的正确方法是什么？
平行
parallel 'youtube-dl -w --no-warnings -o "/media/video1.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/stun-gun-used-on-man-in-police-custody-dies-lawsuit-dnt-brown-tsr.cnn"' 'youtube-dl -w --no-warnings -o "/media/video2.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/5-year-old-saves-family-from-house-fire.ktla"'
如果 运行 没有并行的单独命令，它会起作用：
youtube-dl -w --no-warnings -o "/media/video1.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/stun-gun-used-on-man-in-police-custody-dies-lawsuit-dnt-brown-tsr.cnn"
youtube-dl -w --no-warnings -o "/media/video2.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/5-year-old-saves-family-from-house-fire.ktla"
对于只有两个流，在第一个命令的末尾放置一个与号 (&) 会更容易，您就完成了。像这样：
youtube-dl -w --no-warnings -o "/media/video1.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/stun-gun-used-on-man-in-police-custody-dies-lawsuit-dnt-brown-tsr.cnn" &
youtube-dl -w --no-warnings -o "/media/video2.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/5-year-old-saves-family-from-house-fire.ktla" &
# Wait for both downloads to finish
wait
如果你想使用 GNU Parallel...像这样 - 虽然我通常会被告知更好的方法....:-)
创建一个名为 work.txt 的文件，其中包含以下内容：
/media/video1.%(ext)s http://www.cnn.com/videos/us/2015/11/11/stun-gun-used-on-man-in-police-custody-dies-lawsuit-dnt-brown-tsr.cnn
/media/video2.%(ext)s http://www.cnn.com/videos/us/2015/11/11/5-year-old-saves-family-from-house-fire.ktla
那么命令将是：
parallel --colsep " " youtube-dl -w --no-warnings -o {1} {2} < work.txt
另一种选择是将您的两个命令简单地放在一个文件中，然后将其发送到 GNU Parallel 的标准输入，如下所示：
将其存储在 work.txt
youtube-dl -w --no-warnings -o "/media/video1.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/stun-gun-used-on-man-in-police-custody-dies-lawsuit-dnt-brown-tsr.cnn"
youtube-dl -w --no-warnings -o "/media/video2.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/5-year-old-saves-family-from-house-fire.ktla"
然后 运行
parallel < work.txt
我目前正在尝试使用 GNU Parallel, however I am not being successful at it. Installation was pretty easy. My goal is to run two parallel instances of python script youtube-dl 及其相应的参数。使用并行执行脚本的正确方法是什么？
平行
parallel 'youtube-dl -w --no-warnings -o "/media/video1.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/stun-gun-used-on-man-in-police-custody-dies-lawsuit-dnt-brown-tsr.cnn"' 'youtube-dl -w --no-warnings -o "/media/video2.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/5-year-old-saves-family-from-house-fire.ktla"'
如果 运行 没有并行的单独命令，它会起作用：
youtube-dl -w --no-warnings -o "/media/video1.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/stun-gun-used-on-man-in-police-custody-dies-lawsuit-dnt-brown-tsr.cnn"
youtube-dl -w --no-warnings -o "/media/video2.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/5-year-old-saves-family-from-house-fire.ktla"
对于只有两个流，在第一个命令的末尾放置一个与号 (&) 会更容易，您就完成了。像这样：
youtube-dl -w --no-warnings -o "/media/video1.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/stun-gun-used-on-man-in-police-custody-dies-lawsuit-dnt-brown-tsr.cnn" &
youtube-dl -w --no-warnings -o "/media/video2.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/5-year-old-saves-family-from-house-fire.ktla" &
# Wait for both downloads to finish
wait
如果你想使用 GNU Parallel...像这样 - 虽然我通常会被告知更好的方法....:-)
创建一个名为 work.txt 的文件，其中包含以下内容：
/media/video1.%(ext)s http://www.cnn.com/videos/us/2015/11/11/stun-gun-used-on-man-in-police-custody-dies-lawsuit-dnt-brown-tsr.cnn
/media/video2.%(ext)s http://www.cnn.com/videos/us/2015/11/11/5-year-old-saves-family-from-house-fire.ktla
那么命令将是：
parallel --colsep " " youtube-dl -w --no-warnings -o {1} {2} < work.txt
另一种选择是将您的两个命令简单地放在一个文件中，然后将其发送到 GNU Parallel 的标准输入，如下所示：
将其存储在 work.txt
youtube-dl -w --no-warnings -o "/media/video1.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/stun-gun-used-on-man-in-police-custody-dies-lawsuit-dnt-brown-tsr.cnn"
youtube-dl -w --no-warnings -o "/media/video2.%(ext)s" "http://www.cnn.com/videos/us/2015/11/11/5-year-old-saves-family-from-house-fire.ktla"
然后 运行
parallel < work.txt