bazel:如果已知构建图,为什么目标会不断增加
bazel: why do targets keep increasing if build graph is known
根据 this doc,分析和执行阶段分别处理建立依赖关系树(以及其他事情)以及在需要时进行工作。如果那是真的,我很好奇为什么目标总数会随着构建的进行而不断增加(即,当我开始一个大型构建时,bazel 可能会报告它构建了 100 个目标中的 5 个,但稍后会说它构建了 20 个300 个目标,依此类推,分母会增加一段时间,直到趋于平稳)。
我听说加载和分析阶段可以混合。我可能不完整或不正确的理解是,当 bazel 解析 BUILD 文件时,调用分析以确定命令行上请求的目标需要哪些依赖项,然后我猜这会以某种方式传回加载程序以拉入任何其他这些依赖项引用的 BUILD 文件,如果依赖项(以及 BUILD 文件)不在本地存储库中,这可能会导致加载程序退出并获取远程存储库。
然而,我的理解是,虽然依赖图的动态构建是 bazel 未来的一个潜在方向,但目前,执行不会与分析混合,因此当执行开始时,完整的依赖树应该可用于 bazel(以及已知的目标总数)? bazel 是否有完整的树,但只是不想遍历树以获取计数以防它很大,还是这里发生了其他事情?
注意:我找到了对这种现象的简短提及here,但没有解释为什么会发生。
您在进度条中看到的数字是指操作(命令行..ish)而不是目标(例如 //my:target
)。 I wrote a blog post关于动作图,这里是相关的描述:
The action graph contains a different set of information: file-level
dependencies, full command lines, and other information Bazel needs to
execute the build. If you are familiar with Bazel’s build phases, the
action graph is the output of the loading and analysis phase and used
during the execution phase.
However, Bazel does not necessarily execute every action in the graph.
It only executes if it has to, that is, the action graph is the super
set of what is actually executed.
至于为什么分母越来越大,是因为动作图中的动作到执行发现是惰性的。这是来自 Bazel TL Ulf Adams 的更好解释:
The problem is that Skyframe does not eagerly walk the action graph,
but it does it lazily. The reason for that is performance, since the
action graph can be rather large and this was previously a blocking
operation (where Bazel would just hang for some time). The downside is
that all threads that walk the action graph block on actions that they
execute, which delays discovery of remaining actions. That's why the
number keeps going up during the build.
来源:https://github.com/bazelbuild/bazel/issues/3582#issuecomment-329405311
根据 this doc,分析和执行阶段分别处理建立依赖关系树(以及其他事情)以及在需要时进行工作。如果那是真的,我很好奇为什么目标总数会随着构建的进行而不断增加(即,当我开始一个大型构建时,bazel 可能会报告它构建了 100 个目标中的 5 个,但稍后会说它构建了 20 个300 个目标,依此类推,分母会增加一段时间,直到趋于平稳)。
我听说加载和分析阶段可以混合。我可能不完整或不正确的理解是,当 bazel 解析 BUILD 文件时,调用分析以确定命令行上请求的目标需要哪些依赖项,然后我猜这会以某种方式传回加载程序以拉入任何其他这些依赖项引用的 BUILD 文件,如果依赖项(以及 BUILD 文件)不在本地存储库中,这可能会导致加载程序退出并获取远程存储库。
然而,我的理解是,虽然依赖图的动态构建是 bazel 未来的一个潜在方向,但目前,执行不会与分析混合,因此当执行开始时,完整的依赖树应该可用于 bazel(以及已知的目标总数)? bazel 是否有完整的树,但只是不想遍历树以获取计数以防它很大,还是这里发生了其他事情?
注意:我找到了对这种现象的简短提及here,但没有解释为什么会发生。
您在进度条中看到的数字是指操作(命令行..ish)而不是目标(例如 //my:target
)。 I wrote a blog post关于动作图,这里是相关的描述:
The action graph contains a different set of information: file-level dependencies, full command lines, and other information Bazel needs to execute the build. If you are familiar with Bazel’s build phases, the action graph is the output of the loading and analysis phase and used during the execution phase.
However, Bazel does not necessarily execute every action in the graph. It only executes if it has to, that is, the action graph is the super set of what is actually executed.
至于为什么分母越来越大,是因为动作图中的动作到执行发现是惰性的。这是来自 Bazel TL Ulf Adams 的更好解释:
The problem is that Skyframe does not eagerly walk the action graph, but it does it lazily. The reason for that is performance, since the action graph can be rather large and this was previously a blocking operation (where Bazel would just hang for some time). The downside is that all threads that walk the action graph block on actions that they execute, which delays discovery of remaining actions. That's why the number keeps going up during the build.
来源:https://github.com/bazelbuild/bazel/issues/3582#issuecomment-329405311