inets httpd：无法让简单的 esi 脚本工作

Question

这是我用来配置 httpd 服务器的proplist_file：

[
  {modules, [
    mod_alias,
    mod_actions,
    mod_cgi,
    mod_get,
    mod_esi,
    mod_log
  ]},
  {bind_address, "localhost"}, 
  {port,0},
  {server_name,"httpd_test"},
  {server_root,"/Users/7stud/erlang_programs/inets_proj"},
  {document_root,"./htdocs"},
  {script_alias, {"/cgi-bin/", "/Users/7stud/erlang_programs/inets_proj/cgi-bin/"} },
  {erl_script_alias, {"/cgi-bin/example", [httpd_example]} },
  {erl_script_nocache, true},
  {error_log, "./errors.log"},
  {transfer_log, "./requests.log"}
].

对于 esi 脚本，关键属性是：

  {erl_script_alias, {"/cgi-bin/example", [httpd_example]} }

根据 inets docs:

ESI Properties - Requires mod_esi

{erl_script_alias, {URLPath, [AllowedModule]}}
URLPath = string() and AllowedModule = atom(). erl_script_alias marks all URLs matching url-path as erl scheme scripts. A matching URL is mapped into a specific module and function, for example:
 {erl_script_alias, {"/cgi-bin/example", [httpd_example]}}
A request to http://your.server.org/cgi-bin/example/httpd_example:yahoo would refer to httpd_example:yahoo/3 or, if that does not exist, httpd_example:yahoo/2 and http://your.server.org/cgi-bin/example/other:yahoo would not be allowed to execute.

我的目录结构是：

~/erlang_programs$ tree inets_proj/
inets_proj/
├── cgi-bin
│   ├── 1.pl
│   ├── example
│   │   ├── httpd_example.beam
│   │   └── httpd_example.erl
│   ├── httpd_example.beam
│   └── httpd_example.erl
├── cl.beam
├── cl.erl
├── errors.log
├── htdocs
│   └── file1.txt
├── httpd_example.beam
├── httpd_example.erl
├── mylog.log
├── requests.log
├── s.beam
├── s.erl
└── server.conf

我不确定 httpd_example 模块放在哪里，所以我把它放在几个地方。

esi_mod.erl:

-module(esi_mod).
-compile(export_all).

log(Data) ->
    {ok, IoDevice} = file:open(
        "/Users/7stud/erlang_programs/inets_proj/mylog.log",
        [append]
    ),

    ok = file:write(IoDevice, Data),
    file:close(IoDevice).

get_data(SessionID, _Env, _Input) ->
    Headers = "Content-Type: text/html\r\n\r\n",
    Data = "Hello, esi",

    log(["--Inside esi_mod:get_data() ~n"]),

    ok = mod_esi:deliver(SessionID, Headers),  %Headers must be a string.
    ok = mod_esi:deliver(SessionID, Data).     %Data can be an iolist.

这是 shell 中报告的服务器信息：

$ erl
Erlang/OTP 20 [erts-9.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V9.2  (abort with ^G)

1> c(s).         
s.erl:2: Warning: export_all flag enabled - all functions will be exported
{ok,s}
2> S = s:start().
<0.86.0>
3> httpd:info(S).
[{mime_types,[{"htm","text/html"},{"html","text/html"}]},
 {server_name,"httpd_test"},
 {erl_script_nocache,true},
 {script_alias,{"/cgi-bin/",
                "/Users/7stud/erlang_programs/inets_proj/cgi-bin/"}},
 {bind_address,{127,0,0,1}},
 {modules,[mod_alias,mod_actions,mod_cgi,mod_get,mod_esi,
           mod_log]},
 {server_root,"/Users/7stud/erlang_programs/inets_proj"},
 {erl_script_alias,{"/cgi-bin/example",[httpd_example]}},
 {port,64470},
 {transfer_log,<0.93.0>},
 {error_log,<0.92.0>},
 {document_root,"./htdocs"}]
4>

这是我的请求：

~$ curl -vv "http://localhost:64470/cgi-bin/example/httpd_example:get_data"
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 64470 (#0)
> GET /cgi-bin/example/httpd_example:get_data HTTP/1.1
> Host: localhost:64470
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 404 Object Not Found
< Date: Wed, 28 Feb 2018 10:22:38 GMT
< Server: inets/6.4.5
< Content-Type: text/html
< Content-Length: 245
< 
<HTML>
       <HEAD>
           <TITLE>Object Not Found</TITLE>
      </HEAD>
      <BODY>
      <H1>Object Not Found</H1>
The requested URL &#47;cgi-bin&#47;example&#47;httpd_example:get_data was not found on this server.
</BODY>
      </HTML>
* Connection #0 to host localhost left intact
~$

errors.log:

[Timestamp] access to /cgi-bin/example/httpd_example:get_data failed for 127.0.0.1, reason: "httpd_file: Can't open ./htdocs/cgi-bin/example/httpd_example:get_data: File not found"

根据报错信息，请求路径：

/cgi-bin/example/httpd_example:get_data

已转换为：

./htdocs/cgi-bin/example/httpd_example:get_data

这意味着请求路径被附加到 ./htdocs。嗯？？

Answer 1

好的，我开始工作了。首先，当我四处寻找时，我偶然发现了：

10 Essential Erlang Tools for Erlang Developers

其中包含我迄今为止使用过的最棒的 erlang "tool"：

Command history that isn't wiped out when you exit the erlang shell.

您可以 install/enable 按照此处的说明进行操作：https://github.com/ferd/erlang-history

回到手头的问题：

1) 我发现 httpd 模块的顺序很重要。 mod_esi 需要在 mod_get:

之前

server.conf:

[
  {modules, [
    mod_alias,
    mod_actions,
    mod_esi,
    mod_cgi,
    mod_get,
    mod_log
  ]},
  {bind_address, "localhost"}, 
  {port,0},
  {server_name,"httpd_test"},
  {server_root,"/Users/7stud/erlang_programs/inets_proj"},
  {document_root,"./htdocs"},
  {script_alias, {"/cgi-bin/", "/Users/7stud/erlang_programs/inets_proj/cgi-bin/"} },
  {erl_script_alias, {"/erl", [mymod]} },
  {erl_script_nocache, true},
  {error_log, "./errors.log"},
  {transfer_log, "./requests.log"}
].

获得正确的模块顺序后，由于奇怪的转换路径，我不再收到 "File not found" 错误。

2) esi 模块的代码必须在 server_root 目录中。我的 esi 模块名称是 mymod.erl:

~/erlang_programs$ tree inets_proj/
inets_proj/
├── cgi-bin
│   ├── 1.pl
│   ├── example
│   │   ├── httpd_example.beam
│   │   └── httpd_example.erl
│   ├── httpd_example.beam
│   └── httpd_example.erl
├── cl.beam
├── cl.erl
├── errors.log
├── htdocs
│   └── file1.txt
├── mylog.log
├── mymod.beam
├── mymod.erl
├── requests.log
├── s.beam
├── s.erl
├── server.conf
├── xhttpd_example.beam
└── xhttpd_example.erl

3) 因为我指定了：

{erl_script_alias, {"/erl", [mymod]} }

我需要使用的url是：

http://localhost:57867/erl/mymod:get_data

端口必须与服务器的端口匹配。正确的路径是您在 erl_script_alias 属性加上 /modulename:funcname 或 /modulename/funcname.

中指定的任何路径

这是mymod.erl：

-module(mymod).
-export([get_data/3]).

log(Data) ->
    {ok, IoDevice} = file:open(
        "/Users/7stud/erlang_programs/inets_proj/mylog.log",
        [append]
    ),

    file:write(IoDevice, Data),
    file:close(IoDevice).

get_data(SessionID, Env, Input) ->
    Headers = "Content-Type: text/html\r\n\r\n",
    Data = [
        <<"Hello, ">>, 
        "esi!\n"
    ],

    log(io_lib:format(
        "Inside mymod:get_data()\nSessionId=~p\nEnv=~p\nInput=~p\n", 
        [SessionID, Env, Input]
    )),

    mod_esi:deliver(SessionID, Headers),  %Headers must be a string.
    mod_esi:deliver(SessionID, Data).     %Data can be an iolist.

根据 mod_esi docs:

mod_esi:deliver/2 shall be used to generate the response to the client and SessionID is an identifier that shall by used when calling this function, do not assume anything about the datatype. This function may be called several times to chunk the response data. Notice that the first chunk of data sent to the client must at least contain all HTTP header fields that the response will generate. If the first chunk does not contain the end of HTTP header, that is, "\r\n\r\n", the server assumes that no HTTP header fields will be generated.

4) 编译 mymod.erl:

~/erlang_programs/inets_proj$ erlc mymod.erl 
~/erlang_programs/inets_proj$

每次对 mymod.erl 进行更改后都必须重新编译，然后重新启动服务器。如果这行得通，那就更简单了：

5> httpd:reload_config("server.conf", disturbing).
{error,{missing_property,server_name}}

但即使我的配置文件确实指定了 server_name 属性，我还是收到了那个错误。

5) 我建议您通过在模块列表中包含 mod_log 并指定属性:

来记录错误

{error_log, "./errors.log"}

然后检查该文件以获取有关请求失败时发生的情况的任何反馈。

6) 当我调用我的自定义 log() 方法（为了将一些信息写入文件）时，我不知道 log() 方法导致异常，导致服务器拒绝请求并输入error.log 中的 module traverse failed 消息：

[Timestamp], module traverse failed: mod_esi:do => 
   Error Type:  exit
   Error:       {mod_esi_linked_process_died,<0.97.0>,normal}
   Stack trace: [{mod_esi,receive_headers,1,[{file,"mod_esi.erl"},{line,428}]},
                 {mod_esi,deliver_webpage_chunk,3,
                     [{file,"mod_esi.erl"},{line,389}]},
                 {mod_esi,erl_scheme_webpage_chunk,5,
                     [{file,"mod_esi.erl"},{line,380}]},
                 {mod_esi,generate_webpage,7,
                     [{file,"mod_esi.erl"},{line,314}]},
                 {httpd_response,traverse_modules,2,
                     [{file,"httpd_response.erl"},{line,77}]},
                 {httpd_response,generate_and_send_response,1,
                     [{file,"httpd_response.erl"},{line,44}]},
                 {httpd_request_handler,handle_response,1,
                     [{file,"httpd_request_handler.erl"},{line,655}]},
                 {gen_server,try_dispatch,4,
                     [{file,"gen_server.erl"},{line,616}]}]

7) 这是我用来启动服务器的模块：

-module(s).
-compile(export_all).

%Need to look up port with httpd:info(Server)

ensure_inets_start() ->
    case inets:start() of
        ok -> ok;
        {error,{already_started,inets}} -> ok
    end.


start() ->
    ok = s:ensure_inets_start(),

    {ok, Server} = inets:start(httpd, 
        [{proplist_file, "./server.conf"}]
    ),
    Server.


stop(Server) ->
    ok = inets:stop(httpd, Server).

8) 以下是一些使用 curl...

的示例请求

shell中的服务器信息：

~/erlang_programs/inets_proj$ erl
Erlang/OTP 20 [erts-9.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V9.2  (abort with ^G)
1> c(s).         
s.erl:2: Warning: export_all flag enabled - all functions will be exported
{ok,s}
2> S = s:start().
<0.86.0>
3> httpd:info(S).
[{mime_types,[{"htm","text/html"},{"html","text/html"}]},
 {server_name,"httpd_test"},
 {erl_script_nocache,true},
 {script_alias,{"/cgi-bin/",
                "/Users/7stud/erlang_programs/inets_proj/cgi-bin/"}},
 {bind_address,{127,0,0,1}},
 {modules,[mod_alias,mod_actions,mod_esi,mod_cgi,mod_get,
           mod_log]},
 {server_root,"/Users/7stud/erlang_programs/inets_proj"},
 {erl_script_alias,{"/erl",[mymod]}},
 {port,59202},
 {transfer_log,<0.93.0>},
 {error_log,<0.92.0>},
 {document_root,"./htdocs"}]
4>

请求 1（esi 获取请求）：

~$  curl -v "http://localhost:57867/erl/mymod:get_data"
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 57867 (#0)
> GET /erl/mymod:get_data HTTP/1.1
> Host: localhost:57867
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Wed, 28 Feb 2018 13:28:09 GMT
< Server: inets/6.4.5
< Cache-Control: no-cache
< Pragma: no-cache
< Expires: Wed, 28 Feb 2018 13:28:09 GMT
< Transfer-Encoding: chunked
< Content-Type: text/html
< 
Hello, esi!
* Connection #0 to host localhost left intact
~$

mylog.log:

~/erlang_programs/inets_proj$ cat mylog.log
...
...
Inside mymod:get_data()
SessionId=<0.99.0>
Env=[{server_software,"inets/6.4.5"},
     {server_name,"httpd_test"},
     {host_name,"ChristophersMBP"},
     {gateway_interface,"CGI/1.1"},
     {server_protocol,"HTTP/1.1"},
     {server_port,59202},
     {request_method,"GET"},
     {remote_addr,"127.0.0.1"},
     {peer_cert,undefined},
     {script_name,"/erl/mymod:get_data"},
     {http_host,"localhost:59202"},
     {http_user_agent,"curl/7.58.0"},
     {http_accept,"*/*"}]
Input=[]
-------

请求 2（esi 获取带有查询字符串的请求）：

~$  curl -v "http://localhost:59202/erl/mymod:get_data?a=1&b=2"
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 59202 (#0)
> GET /erl/mymod:get_data?a=1&b=2 HTTP/1.1
> Host: localhost:59202
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Wed, 28 Feb 2018 13:47:41 GMT
< Server: inets/6.4.5
< Cache-Control: no-cache
< Pragma: no-cache
< Expires: Wed, 28 Feb 2018 13:47:41 GMT
< Transfer-Encoding: chunked
< Content-Type: text/html
< 
Hello, esi!
* Connection #0 to host localhost left intact

mylog.log:

~/erlang_programs/inets_proj$ cat mylog.log
...
...
Inside mymod:get_data()
SessionId=<0.105.0>
Env=[{server_software,"inets/6.4.5"},
     {server_name,"httpd_test"},
     {host_name,"ChristophersMBP"},
     {gateway_interface,"CGI/1.1"},
     {server_protocol,"HTTP/1.1"},
     {server_port,59202},
     {request_method,"GET"},
     {remote_addr,"127.0.0.1"},
     {peer_cert,undefined},
     {script_name,"/erl/mymod:get_data?a=1&b=2"},
     {http_host,"localhost:59202"},
     {http_user_agent,"curl/7.58.0"},
     {http_accept,"*/*"},
     {query_string,"a=1&b=2"}]
Input="a=1&b=2"

请求 3（esi post 请求）：

~$  curl -v --data "a=1&b=2" "http://localhost:59202/erl/mymod:get_data"
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 59202 (#0)
> POST /erl/mymod:get_data HTTP/1.1
> Host: localhost:59202
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Length: 7
> Content-Type: application/x-www-form-urlencoded
> 
* upload completely sent off: 7 out of 7 bytes
< HTTP/1.1 200 OK
< Date: Wed, 28 Feb 2018 13:51:44 GMT
< Server: inets/6.4.5
< Cache-Control: no-cache
< Pragma: no-cache
< Expires: Wed, 28 Feb 2018 13:51:44 GMT
< Transfer-Encoding: chunked
< Content-Type: text/html
< 
Hello, esi!
* Connection #0 to host localhost left intact

mylog.log:

Inside mymod:get_data()
SessionId=<0.108.0>
Env=[{server_software,"inets/6.4.5"},
     {server_name,"httpd_test"},
     {host_name,"ChristophersMBP"},
     {gateway_interface,"CGI/1.1"},
     {server_protocol,"HTTP/1.1"},
     {server_port,59202},
     {request_method,"POST"},
     {remote_addr,"127.0.0.1"},
     {peer_cert,undefined},
     {script_name,"/erl/mymod:get_data"},
     {http_host,"localhost:59202"},
     {http_user_agent,"curl/7.58.0"},
     {http_accept,"*/*"},
     {http_content_length,"7"},
     {http_content_type,"application/x-www-form-urlencoded"}]
Input="a=1&b=2"
-------

请求 4（cgi 获取请求）：

~$  curl -v "http://localhost:59202/cgi-bin/1.pl"
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 59202 (#0)
> GET /cgi-bin/1.pl HTTP/1.1
> Host: localhost:59202
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Wed, 28 Feb 2018 13:41:43 GMT
< Server: inets/6.4.5
< Transfer-Encoding: chunked
< Content-Type: text/html
< 
Hello, Perl.
* Connection #0 to host localhost left intact

请求 5（从 document_root 目录获取常规文件的请求）：

~$  curl -v "http://localhost:59202/file1.txt"
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 59202 (#0)
> GET /file1.txt HTTP/1.1
> Host: localhost:59202
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Wed, 28 Feb 2018 13:42:15 GMT
< Server: inets/6.4.5
< Content-Type: text/plain
< Etag: nCZT0114
< Content-Length: 14
< Last-Modified: Mon, 26 Feb 2018 02:51:52 GMT
< 
Hello, world!
* Connection #0 to host localhost left intact

inets httpd：无法让简单的 esi 脚本工作

inets httpd: Can't get simple esi script to work

erlang

inets