Shell 下载脚本

Question

我应该编写一个 sh 脚本来下载一个文件（大约 100MB - 100GB）并每 10 秒在 mysql 数据库中写入进度（百分比、下载速度、剩余时间）。有人可以帮我吗？

谢谢！

祝你有愉快的一天:)

Answer 1

wget 有 --progress 选项，但它是为正在观看进度的人准备的。 curl 也有一个进度表，但同样，它是为人类设计的。

您可以使用 wget -o 将进度信息发送到日志文件并对其进行解析。

--2017-03-01 13:13:21--  http://download.thinkbroadband.com/1GB.zip
Resolving download.thinkbroadband.com (download.thinkbroadband.com)... 80.249.99.148
Connecting to download.thinkbroadband.com (download.thinkbroadband.com)|80.249.99.148|:80... connect
HTTP request sent, awaiting response... 200 OK
Length: 1073741824 (1.0G) [application/zip]
Saving to: ‘1GB.zip’

     0K .......... .......... .......... .......... ..........  0%  150K 1h56m
    50K .......... .......... .......... .......... ..........  0%  308K 86m34s
   100K .......... .......... .......... .......... ..........  0% 2.31M 60m10s
   150K .......... .......... .......... .......... ..........  0%  348K 57m41s
   200K .......... .......... .......... .......... ..........  0% 3.34M 47m10s

你必须编写一个程序来解析它。

使用提供进度回调的 HTTP 客户端库编写一个小程序更简单、更灵活。这是 Perl 中使用 HTTP::Tiny.

的示例

#!/usr/bin/perl

use strict;
use warnings;
use v5.10;
use HTTP::Tiny;

my $url = "http://download.thinkbroadband.com/100MB.zip";

# A tiny HTTP client.
my $http = HTTP::Tiny->new;

# Track how many bytes have been received.
my $completed = 0;
$http->request("GET", $url, {
    # Add a callback when the next data hunk is received.
    data_callback => sub {
        # The data hunk, and info about the request.
        my($chunk, $response) = @_;

        # Add to the total received.
        $completed += length $chunk;

        # Get the total length (this won't always be available)
        my $length = $response->{'headers'}{'content-length'} || '?';

        # Calculate the percent received.
        my $percent = $completed / $length * 100;

        # Print it, making sure not to print too many decimal places.
        printf "%d of %d - %.2f%%\n", $completed, $length, $percent;

        return;
    }
});

大多数语言都有类似的东西。现在你可以随心所欲记录进度了。

Shell 下载脚本

Shell download script

mysql

shell

file

wget

download