如何使用 luacurl / libcurl / curl 和 Lua 获取 HTML 代码

Question

我的代码中缺少什么才能获取网站的 html 源代码（感谢@Michal Kottman）？与右键单击并单击 chrome 中的 "view page source" 相同。

local curl = require "luacurl"
local c = curl.new()

function GET(url)
    c:setopt(curl.OPT_URL, url)
    c:setopt(curl.OPT_PROXY, "http://myproxy.bla.com:8080")
    c:setopt(curl.OPT_HTTPHEADER, "Connection: Keep-Alive", "Accept-Language: en-us")
    c:setopt(curl.OPT_CONNECTTIMEOUT, 30 )
    local t = {} -- this will collect resulting chunks
    c:setopt(curl.OPT_WRITEFUNCTION, function (param, buf)
        table.insert(t, buf) -- store a chunk of data received
        return #buf
    end)
    c:setopt(curl.OPT_PROGRESSFUNCTION, function(param, dltotal, dlnow)
        print('%', url, dltotal, dlnow) -- do your fancy reporting here
    end)
    c:setopt(curl.OPT_NOPROGRESS, false) -- use this to activate progress
    assert(c:perform())
    return table.concat(t) -- return the whole data as a string
end

--local s = GET 'http://www.lua.org/'
local s = GET 'https://www.youtube.com/watch?v=dT_fkwX4fRM'
print(s)
file = io.open("text.html", "wb")
file:write(s)
file:close()

不幸的是，它必须使用 Lua 并为 libcurl 使用 luacurl 绑定，因为 luasocket 在提供代理时无法正常工作（至少对我而言）。我下载的文件是空的。使用 cmd 我可以毫无问题地获取页面源 curl http://mypage.com

它非常适合 lua.org 但对于 youtube 链接却不行。我错过了什么？

Answer 1

local curl = require "luacurl"
local c = curl.new()

function GET(url)
    c:setopt(curl.OPT_URL, url)
    c:setopt(curl.OPT_PROXY, "http://myproxy.com:8080")
    c:setopt(curl.OPT_HTTPHEADER, "Connection: Keep-Alive", "Accept-Language: en-us")
    c:setopt(curl.OPT_CONNECTTIMEOUT, 30 )
    c:setopt(curl.OPT_FOLLOWLOCATION, true) -- REALLY IMPORTANT ELSE FAIL
    c:setopt(curl.OPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36")
    c:setopt(curl.OPT_SSL_VERIFYPEER, false) -- REALLY IMPORTANT ELSE NOTHING HAPPENS -.-
    c:setopt(curl.OPT_ENCODING, "utf8") -- could be important
    local t = {} -- this will collect resulting chunks
    c:setopt(curl.OPT_WRITEFUNCTION, function (param, buf)
        table.insert(t, buf) -- store a chunk of data received
        return #buf
    end)
    c:setopt(curl.OPT_PROGRESSFUNCTION, function(param, dltotal, dlnow)
        print('%', url, dltotal, dlnow) -- do your fancy reporting here
    end)
    c:setopt(curl.OPT_NOPROGRESS, false) -- use this to activate progress
    assert(c:perform())
    return table.concat(t) -- return the whole data as a string
end

如何使用 luacurl / libcurl / curl 和 Lua 获取 HTML 代码

How do I get the HTML code using luacurl / libcurl / curl and Lua

lua

curl