python setup.py sdist 和自定义设置关键字不能一起玩

python setup.py sdist and custom setup keywords don't play together

副标题:不仅是sdist

我正在尝试获取我正在处理的包的 setup.py 文件,以便与 sdist 很好地配合使用。 setup.py 文件的相关部分是:

from setuptools.command.test import test
[...]
class Tox(test):
   "as described in 
    http://tox.readthedocs.org/en/latest/example/basic.html?highlight=setuptools#integration-with-setuptools-distribute-test-commands"
   [...]
def run_tests(self):
    if self.distribution.install_requires:
        self.distribution.fetch_build_eggs(
            self.distribution.install_requires)
    if self.distribution.tox_requires:
        self.distribution.fetch_build_eggs(self.distribution.tox_requires)
    # import here, cause outside the eggs aren't loaded
    import tox
    import shlex
    args = self.tox_args
    if args:
        args = shlex.split(self.tox_args)
    else:
        args = ""
    errno = tox.cmdline(args=args)
    sys.exit(errno)


entry_points ={}
distutils_ext = {'distutils.setup_keywords': [
                    "tox_requires = setuptools.dist:check_requirements", ]
                 }
entry_points.update(distutils_ext)

setup(
      install_requires=['six', 'numpy', 'matplotlib', 'scipy', 'astropy>=1',
                  'Pillow', ],

    cmdclass={
        'test': PyTest,  # this is to run python setup.py test
        'tox': Tox,
    },

    # list of packages and data
    packages=find_packages(),

    # tests
    tests_require=['pytest', 'pytest-cov'],
    tox_requires=['tox'],
    # other keywords, mostly metadata
)

如果我运行python setup.py sdist,我一开始就得到一个警告:

/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'tox_requires'
  warnings.warn(msg)

但是 sdist 工作正常,它创建了一个 tar.gz 文件,我可以用它来安装我的包。

但是如果我 运行 第二次,它开始于(这是枕头建筑的开始):

warning: no previously-included files found matching '.editorconfig'
Building using 4 processes
_imaging.c: In function ‘getink’:

并开始将所有必需的包构建到 .eggs 目录中。

如果删除 *egg-info 目录,我可以重新运行 命令。 如果我注释掉 tox_requires=[...] 行,我可以根据需要多次构建 sdist。

现在根据 setuptools documentation 上面的命令应该是 运行 向设置函数添加新参数的正确方法。


根据副标题,问题不仅在于 sdist,还可能是由于我 non-understanding 关于设置工具和要求的工作方式。

如果我 运行 python setup.py tox 在一个没有安装 tox 的地方我得到,在安装一些测试包后它不应该安装(即 pytestpytest-cov):

回溯(最后一次调用): [...] 文件“/usr/lib/python2.7/dist-packages/setuptools/command/test.py”,第 127 行,在 with_project_on_sys_path 中 功能() 文件 "setup.py",第 65 行,在 run_tests 中 如果 self.distribution.tox_requires: AttributeError:分发实例没有属性 'tox_requires'


[更新] tox_requires 在安装过程中 pip 也很混乱。如果它被注释掉,我可以毫无问题地安装该软件包;否则它开始编译包的源代码并且它系统地失败,因为它在构建诸如 scipy

之类的东西时找不到 numpy

如何让设置工具识别并正确使用 tox_requires

一旦这个问题得到解决,我认为我可以摆脱这里的虚假安装,从而更好地实现 Tox class,也许可以覆盖 [=29= 中的更多内容] 或直接从 Command

派生

下面描述的完整(有效)解决方案包含 8 个文件(包括简短的 README.rst) 总共有 43 行代码。这比你的代码少 原始问题。

虽然很短,但支持很多开发测试场景 非常方便。

无论如何,它并没有完全回答你的问题,但我相信,它满足了 它背后的要求。

三行长setup.py

从技术上讲,可以将包含 tox 自动化的 test 命令放入您的 setup.py,但是,结果可能会非常混乱且难以理解。

可以用更简单的方法实现相同的结果:

  • 开发人员假设:

    • 使用 git
    • 已将 tox 安装到系统中
  • 包用户:

    • 安装生成的包没有特殊要求
  • (可选)如果您希望您的用户通过单个命令测试包并将测试报告收集在中央服务器中:

    • 安装 devpi-server 并授予您的用户访问权限
    • 请您的用户安装 $ pip install devpi

该解决方案基于以下工具和包:

  • pbr:简化包创建,包括。通过 git 标签和 从 git 提交消息创建 AUTHORSChangeLog
  • pytest:优秀的测试框架,但任何其他框架都可以 被用来代替它。
  • tox:出色的构建和测试自动化工具。
  • coverage:测量测试覆盖率的工具(工作更简单 pytest-cov)

您也可以选择使用:

  • devpi-server:受密码保护的私有 PyPi 服务器 使用权。允许简单测试并提供测试报告收集。
  • devpi:类似pip的工具。除了安装还支持 运行宁 tox 定义的测试(安装,运行 测试,在步骤中发布报告)。

正在编写包

新建工程目录并初始化git:

$ mkdir francesco
$ cd francesco
$ git init

创建包或模块

这里我们创建了单个模块francesco,但同样适用于更多 模块或包。

francesco.py

def main():
    print("Hi, it is me, Francesco, keeping things simple.")

requirements.txt

为包的实际安装创建包列表:

six

test_requirements.txt

定义测试所需的包:

pytest
coverage

tests/test_it.py

启动测试套件:

from francesco import main


def test_this():
    main()
    print("All seems fine to me")
    assert True

setup.py

你有没有想过傻傻的setup.py?开始了:

from setuptools import setup

setup(setup_requires=["pbr"], pbr=True)

setup.cfg

元数据属于配置文件:

[metadata]
name = francesco
author = Francesco Montesano
author-email = fm@acme.com
summary = Nice and simply installed python module supporting testing in different pythons
description-file = README.rst
[files]
modules=francesco
[entry_points]
console_scripts =
    francesco = francesco:main

tox.ini

配置 tox 自动构建和测试:

[tox]
envlist = py27, py34

[testenv]
commands =
    coverage run --source francesco -m pytest -sv tests
    coverage report
    coverage html
deps =
    -rtest_requirements.txt

README.rst

永远不会忘记README.rst:

===========================================
Complex package with 3 line long `setup.py`
===========================================

Can we keep`setup.py` simple and still support automated testing?

...

tox:在所有支持的 python 版本中构建 sdist 和 运行 测试

在项目根目录下,只需运行单条命令tox:

$ tox
GLOB sdist-make: /home/javl/sandbox/setuppy/setup.py
py27 inst-nodeps: /home/javl/sandbox/setuppy/.tox/dist/francesco-0.0.0.zip
py27 runtests: PYTHONHASHSEED='2409409075'
py27 runtests: commands[0] | coverage run --source francesco -m pytest -sv tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.9, pytest-2.8.7, py-1.4.31, pluggy-0.3.1 -- /home/javl/sandbox/setuppy/.tox/py27/bin/python2.7
cachedir: .cache
rootdir: /home/javl/sandbox/setuppy, inifile: 
collecting ... collected 1 items

tests/test_it.py::test_this Hi, it is me, Francesco, keeping things simple.
All seems fine to me
PASSED

=========================== 1 passed in 0.01 seconds ===========================
py27 runtests: commands[1] | coverage report
Name           Stmts   Miss  Cover
----------------------------------
francesco.py       2      0   100%
py27 runtests: commands[2] | coverage html
py34 inst-nodeps: /home/javl/sandbox/setuppy/.tox/dist/francesco-0.0.0.zip
py34 runtests: PYTHONHASHSEED='2409409075'
py34 runtests: commands[0] | coverage run --source francesco -m pytest -sv tests
============================= test session starts ==============================
platform linux -- Python 3.4.2, pytest-2.8.7, py-1.4.31, pluggy-0.3.1 -- /home/javl/sandbox/setuppy/.tox/py34/bin/python3.4
cachedir: .cache
rootdir: /home/javl/sandbox/setuppy, inifile: 
collecting ... collected 1 items

tests/test_it.py::test_this Hi, it is me, Francesco, keeping things simple.
All seems fine to me
PASSED

=========================== 1 passed in 0.01 seconds ===========================
py34 runtests: commands[1] | coverage report
Name           Stmts   Miss  Cover
----------------------------------
francesco.py       2      0   100%
py34 runtests: commands[2] | coverage html
___________________________________ summary ____________________________________
  py27: commands succeeded
  py34: commands succeeded
  congratulations :)

获取 sdist

ls .tox/dist
francesco-0.0.0.zip

在 Python 2.7 virtualenv

中开发

激活Python 2.7 virtualenv

$ source .tox/py27/bin/activate

运行 测试

(py27) $ py.test -sv tests

==============================================================================================
test session starts
===============================================================================================
platform linux2 -- Python 2.7.9, pytest-2.8.7, py-1.4.31, pluggy-0.3.1
-- /home/javl/sandbox/setuppy/.tox/py27/bin/python2.7 cachedir: .cache
rootdir: /home/javl/sandbox/setuppy, inifile: collected 1 items

tests/test_it.py::test_this Hi, it is me, Francesco, keeping things
simple. All seems fine to me PASSED

============================================================================================
1 passed in 0.01 seconds
============================================================================================

测量测试覆盖率

(py27)$ coverage run --source francesco -m pytest -sv tests
.....
(py27)$ coverage report
Name           Stmts   Miss  Cover
----------------------------------
francesco.py       2      0   100%

在网络浏览器中查看覆盖率报告

(py27)$ coverage html
(py27)$ firefox htmlcov/index.html

发布新的软件包版本

(可选)安装本地 devpi-server

devpi-server的安装这里就不说了,但是非常 简单,特别是如果您只为您的本地机器安装 个人测试。

提交源代码,分配版本标签

确保所有源代码都已提交。

正在评估版本标签:

$ git tag -a 0.1

重新运行 tox 和 build sdist 的测试

确保你已经停用了virtualenv(否则会与 毒素):

(py27)$ deactivate

运行 tox:

$ tox
.....
...it builds as usual, may fail, if you have forgotten to commit some changes or files...

找到新版本包的 sdist:

$ ls .tox/dist/francesco-0.1.0.
.tox/dist/francesco-0.1.0.zip

大功告成。您可以分发您的包的新测试版本 像往常一样给用户。

(可选)将sdist上传到devpi-server并在本地测试

以下步骤假定您已安装 devpi-server 并 运行ning.

$ devpi login javl
...enter your password...
$ devpi upload .tox/dist/francesco-0.1.0.zip

在干净的环境中测试包

(如果激活则停用 virtualenv):

$ cd /tmp
$ mkdir testing
$ cd testing
$ devpi test francesco
received http://localhost:3141/javl/dev/+f/4f7/c13fee84bb7c8/francesco-0.1.0.zip
unpacking /tmp/devpi-test6/downloads/francesco-0.1.0.zip to /tmp/devpi-test6/zip
/tmp/devpi-test6/zip/francesco-0.1.0$ tox --installpkg /tmp/devpi-test6/downloads/francesco-0.1.0.zip -i ALL=http://localhost:3141/javl/dev/+simple/ --recreate --result-json /tmp/devpi-test6/zip/toxreport.json
-c /tmp/devpi-test6/zip/francesco-0.1.0/tox.ini
py27 create: /tmp/devpi-test6/zip/francesco-0.1.0/.tox/py27
py27 installdeps: -rtest_requirements.txt
py27 inst: /tmp/devpi-test6/downloads/francesco-0.1.0.zip
py27 installed: coverage==4.0.3,francesco==0.1.0,py==1.4.31,pytest==2.8.7,six==1.10.0,wheel==0.24.0
py27 runtests: PYTHONHASHSEED='3916044270'
py27 runtests: commands[0] | coverage run --source francesco -m pytest -sv tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.9, pytest-2.8.7, py-1.4.31, pluggy-0.3.1 -- /tmp/devpi-test6/zip/francesco-0.1.0/.tox/py27/bin/python2.7
cachedir: .cache
rootdir: /tmp/devpi-test6/zip/francesco-0.1.0, inifile:
collecting ... collected 1 items

tests/test_it.py::test_this Hi, it is me, Francesco, keeping things simple.
All seems fine to me
PASSED

=========================== 1 passed in 0.01 seconds ===========================
py27 runtests: commands[1] | coverage report
Name           Stmts   Miss  Cover
----------------------------------
francesco.py       2      0   100%
py27 runtests: commands[2] | coverage html
py34 create: /tmp/devpi-test6/zip/francesco-0.1.0/.tox/py34
py34 installdeps: -rtest_requirements.txt
py34 inst: /tmp/devpi-test6/downloads/francesco-0.1.0.zip
py34 installed: coverage==4.0.3,francesco==0.1.0,py==1.4.31,pytest==2.8.7,six==1.10.0,wheel==0.24.0
py34 runtests: PYTHONHASHSEED='3916044270'
py34 runtests: commands[0] | coverage run --source francesco -m pytest -sv tests
============================= test session starts ==============================
platform linux -- Python 3.4.2, pytest-2.8.7, py-1.4.31, pluggy-0.3.1 -- /tmp/devpi-test6/zip/francesco-0.1.0/.tox/py34/bin/python3.4
cachedir: .cache
rootdir: /tmp/devpi-test6/zip/francesco-0.1.0, inifile:
collecting ... collected 1 items

tests/test_it.py::test_this Hi, it is me, Francesco, keeping things simple.
All seems fine to me
PASSED

=========================== 1 passed in 0.01 seconds ===========================
py34 runtests: commands[1] | coverage report
Name           Stmts   Miss  Cover
----------------------------------
francesco.py       2      0   100%
py34 runtests: commands[2] | coverage html
____________________________________________________________________________________________________ summary _____________________________________________________________________________________________________
  py27: commands succeeded
  py34: commands succeeded
  congratulations :)
wrote json report at: /tmp/devpi-test6/zip/toxreport.json
posting tox result data to http://localhost:3141/javl/dev/+f/4f7/c13fee84bb7c8/francesco-0.1.0.zip
successfully posted tox result data

您可以在网络浏览器中查看测试结果:

$ firefox http://localhost:3141

然后搜索"francesco"包,点击包名,在table中找到 名为 "tox results" 的列,单击此处以显示环境设置和测试 结果。

让您的用户测试包

假设您的 devpi-server 是 运行ning 并且您的用户可以访问它。

用户应安装 devpi 命令:

$ pip install devpi

(注意,此工具未安装 devpi-server 中的任何内容)

帮助您的用户获得对 devpi-server 的访问权限(此处不涉及)。

那么用户只是运行测试:

$ devpi test francesco

测试后是运行(自动使用tox,但用户没有 关心那个),你会在 devpi web 的同一个地方找到测试结果 与您之前找到的界面一样。