如何将脚本用作命令行应用程序或模块

Question

我觉得这应该是显而易见的，但我已经在谷歌上搜索了很多，但我还没有找到解决方案，所以如果你能提供帮助，我将不胜感激：

我写了一个 python 脚本，我希望它既可以用作命令行应用程序，也可以用作我可以在其他应用程序中加载的模块。到目前为止，我只为命令行选项编写了代码。以下是我的文件 etl.py 结构的摘要：

import os
import re
import sys
import shlex
import argparse
import itertools
import subprocess
from sqlalchemy import create_engine, text
# other imports here...

def etl(argv):
    """
    ETL function for rasters ...
    # more doc here
    """
    args = parser.parse_args(argv)
    d = vars(args)

    os.chdir(d.get("root_dir"))
    files = [f for f in os.listdir(".") for p in d.get("products") if p in f]

    # a) Reprojection (option "r")
    if d.get("which") == "r":
        for f in files:
            ofile = os.path.abspath(os.path.join(d.get("reproj_dir"), f))
            if os.path.exists(ofile):
                if d.get("overwrite") is True:
                    gdutil.reproject(os.path.abspath(f), ofile,    .get("proj"))
                else:
                    print("{}: File already exists. skipping".format(ofile))
            else:
                gdutil.reproject(os.path.abspath(f), ofile, d.get("proj"))
          print("All files reprojected into EPSG {}".format(d.get("proj")))

    # b) Merge (option "m")
    if d.get("which") == "m":
        # more operations here until...

if __name__ == '__main__':
    parser = argparse.ArgumentParser(
        description="""
        'Extract, Transform, Load' script for rasters ...""",
        formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument("-dir", help="""
        directory containing the input rasters to process

        default value is the current directory""",
        dest="root_dir", default=os.getcwd(), const=os.getcwd(), nargs="?", metavar="DIR")
    gen_opts = argparse.ArgumentParser(add_help=False)
    gen_opts.add_argument("-p", help="""
        product type(s) to process

        default is to process all products in DIR.           

        """.format(os.path.basename(sys.argv[0])),
        dest="products", nargs="*", metavar="PROD")

    # more gen_opts arguments here

    subparsers = parser.add_subparsers()
    parser_r = subparsers.add_parser("r", help="""
        reproject rasters into target reprojection

        if rasters corresponding to the passed arguments are found, they will be
        reprojected into the target spatial reference system as defined by
        the passed EPSG code

        reprojected files will have the same pixel resolution as the input
        rasters

        resampling method is nearest neighbor     

        reprojected files are saved to the folder specified with option -r_dir
        or to its default value if missing
        """.format(os.path.basename(sys.argv[0])),
        parents=[gen_opts, overw_opts])
    parser_r.add_argument("proj", help="""
        EPSG code of the destination reprojection""",
        type=int, metavar="EPSG_CODE")

    # more parser_r arguments here

    parser_r.set_defaults(which="r")
    parser_m = subparsers.add_parser("m", help="""
        merge input rasters into a mosaic

        rasters are merged based on the supplied level

        all files that are included in a merge must have the same projection,
        pixel size, no-data value, and be unique in space. otherwise, results
        are not guaranteed""", parents=[gen_opts, group_opt, overw_opts])
    parser_m.add_argument("-m_dir", help="""
        if supplied, the merged rasters will be saved in directory M_DIR

        if not supplied they will be saved to a subfolder named 'merged'
        located at in DIR, or in R_DIR if a reprojection was made in the same
        call""",
        dest="merge_dir", type=str, nargs=1, metavar="M_DIR")
    parser_m.set_defaults(which="m")

    # more parsers and arguments here

    # etl(sys.argv)
    etl(["-dir", "E:\Data\Spatial\soilgrids.org", "r", "3175", "-c", "M"])

到目前为止，我只能运行最后一行取消注释并从终端不带参数调用脚本，但这不是现在的主要问题。

我的问题是：如何将脚本用作我可以在其他脚本中导入的模块，例如从 etl 导入重新项目？

我认为一件事是将我的部分代码（即每个部分以评论开头，例如 a) Reprojection、b) Merge、c) ...）放在它们自己的函数中：

def reproject():
    # add code here

def merge():
    # add code here

然后向每个解析器添加默认函数（例如 parser_r.set_defaults(func=reproject)）但是，如果我还要从另一个应用程序中使用它们，我将如何定义每个函数定义的参数我将导入模块 etl，例如：

from etl import reproject
reproject() #  arguments?

我必须添加可选参数或关键字吗？我是否必须测试是否可以使用 parser.parse_args 解析参数？我该怎么做？

非常感谢您的帮助！

Answer 1

您必须更改函数的调用签名（和主体）

def etl(argv):
    ...

这样它就不会对 argv 做任何事情。相反，它看起来像：

def etl(products, reproj_dir, ...):
    ...

这称为为您的函数定义接口。

您可以将所有参数解析内容留在 if __name__ == '__main__': 块内，但是调用 etl 的时刻应该使用 sys.argv 的内容。 您也应该将调用 parser.parse_args 的行也移到该块内。

看函数体，好像是踢进了两种模式，所以确实最好是def reproject和def merge，然后把它们的接口固定下来，as你在想。

一旦你有了合适的接口，只需导入模块并直接调用这些函数。实际功能根本不需要了解命令行界面。

如何将脚本用作命令行应用程序或模块

How can a script be used either as a command line application or as a module

python

command-line

argparse