如何使用 Bazel 的 py_library 导入参数

Question

我正在尝试在 Bazel 构建的项目中使用 Boto3，但似乎无法正确导入库。由于 Boto git 存储库，所有源都位于存储库根目录中名为 botocore 和 boto3 的文件夹中。导入都是 boto3.boto3，第一个对应外部依赖的名称，第二个是所在的根文件夹。如何使用 py_binary 和 py_library 规则的 imports 属性从内部 boto3 而不是另一个规则导入？

这是我的工作区的样子：

//WORKSPACE

BOTOCORE_BUILD_FILE = """

py_library(
    name = "botocore",
    srcs = glob([ "botocore/**/*.py" ]),
    imports = [ "botocore" ],
    visibility = [ "//visibility:public" ],
)

"""

_BOTO3_BUILD_FILE = """

py_library(
    name = "boto3",
    srcs = glob([ "boto3/**/*.py" ]),
    imports = [ "boto3" ],
    deps = [ "@botocore//:botocore" ],
    visibility = [ "//visibility:public" ],
)

"""

new_git_repository(
    name = "botocore",
    commit = "cc3da098d06392c332a60427ff434aa51ba31699",
    remote = "https://github.com/boto/botocore.git",
    build_file_content = _BOTOCORE_BUILD_FILE,
)

new_git_repository(
    name = "boto3",
    commit = "8227503d7b1322b45052a16b197ac41fedd634e9", # 1.4.4
    remote = "https://github.com/boto/boto3.git",
    build_file_content = _BOTO3_BUILD_FILE,
)

//BUILD

py_binary(
    name = "example",
    srcs = [ "example.py" ],
    deps = [
        "@boto3//:boto3",
    ],
)

//example.py

import boto3

boto3.client('')

正在检查构建文件夹的内容

$ ls bazel-bin/example.runfiles/*
bazel-bin/example.runfiles/__init__.py bazel-bin/example.runfiles/MANIFEST

bazel-bin/example.runfiles/boto3:
boto3  __init__.py

bazel-bin/example.runfiles/botocore:
botocore  __init__.py

当我尝试运行示例脚本时，我得到 AttributeError: 'module' object has no attribute 'client' 我可以 import boto3.boto3 但随后使用其中的任何内容都会导致缺少依赖项，例如 boto3.sessions 因为一切嵌套在 <target-name>.boto3

Answer 1

我认为您的方向是正确的，但是由于 python sys.path.[=19= 的顺序，您运行陷入了一个微妙的问题]

如果我运行你的例子并在 example.py 中打印出 sys.path，我看到路径包含顺序：

bazel-out/local-fastbuild/bin/example.runfiles
bazel-out/local-fastbuild/bin/example.runfiles/boto3/boto3
bazel-out/local-fastbuild/bin/example.runfiles/boto3

第二行是由于您的 WORKSPACE 文件中的 imports = ['boto3']。

我认为您希望第三行是您从中获取 import boto3 的地方，因为您希望 python 看到 bazel-out/local-fastbuild/bin/example.runfiles/boto3/boto3/__init__.py.

因此，当 python 计算 import boto3 时，它从第一个条目中看到 bazel-out/local-fastbuild/bin/example.runfiles/boto3/__init__.py 并使用它，而不是从第三个条目中看到 bazel-out/local-fastbuild/bin/example.runfiles/boto3/boto3/__init__.py。

我认为这里的答案是将您的 "workspace" 命名为不同于它包含的目录的名称。例如：

# WORKSPACE
new_git_repository(
  name = "boto3_archive",
  commit = "8227503d7b1322b45052a16b197ac41fedd634e9", # 1.4.4
  remote = "https://github.com/boto/boto3.git",
  build_file_content = _BOTO3_BUILD_FILE,
)

# BUILD
py_binary(
  name = "example",
  srcs = [ "example.py" ],
  deps = [
    "@boto3_archive//:boto3",
  ],
)

当我在您的示例中执行此操作时，出现以下错误：ImportError: No module named dateutil.parser，我认为这是进步。

Answer 2

文件

$ tree
.
├── BUILD
├── WORKSPACE
├── requirements.txt
└── src
    └── main.py

requirements.txt

$ cat requirements.txt 
boto3==1.13.4

工作空间

$ cat WORKSPACE 
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
    name = "rules_python",
    url = "https://github.com/bazelbuild/rules_python/releases/download/0.0.2/rules_python-0.0.2.tar.gz",
    strip_prefix = "rules_python-0.0.2",
    sha256 = "b5668cde8bb6e3515057ef465a35ad712214962f0b3a314e551204266c7be90c",
)
load("@rules_python//python:repositories.bzl", "py_repositories")

py_repositories()

# Only needed if using the packaging rules.
load("@rules_python//python:pip.bzl", "pip_repositories")

pip_repositories()


# Python external packages installation
load(
    "@rules_python//python:pip.bzl", "pip3_import"
)

pip3_import(
    name = "lambda_deps",
    requirements = "//:requirements.txt", # Top level requirements.txt file
)

load("@lambda_deps//:requirements.bzl", "pip_install")
pip_install()

建造

$ cat BUILD
load(
    "@lambda_deps//:requirements.bzl",
    "requirement"
)

py_binary(
    name = 's3_test',
    main = 'main.py',
    srcs = ['src/main.py'],
    deps = [
        requirement('boto3')
    ]
)

src/main.py

$ cat src/main.py
import boto3


def hello_boto3():
    print('hello', boto3.client('s3'))


if __name__ == '__main__':
    hello_boto3()

构建并运行

$ bazel clean && bazel build //:s3_test
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.
INFO: Analyzed target //:s3_test (24 packages loaded, 1234 targets configured).
INFO: Found 1 target...
Target //:s3_test up-to-date:
  bazel-bin/s3_test
INFO: Elapsed time: 11.666s, Critical Path: 0.25s
INFO: 0 processes.
INFO: Build completed successfully, 5 total actions

$ ./bazel-bin/s3_test
hello <botocore.client.S3 object at 0x7ff1b6686b38>

如何使用 Bazel 的 py_library 导入参数

How to use Bazel's py_library imports argument

python

build

boto3

bazel

文件

requirements.txt

工作空间

建造

src/main.py

构建并运行