列出 SRC_URI 构建 Yocto 映像所需的全部 packages/files
Listing SRC_URI of all packages/files needed to build Yocto image
我想列出烘焙图像时 bitbake 将获取的所有文件。
目前，我能够通过执行 bitbake core-image-minimal -c fetchall 然后解析日志文件来获取烘焙 Yocto 图像所需的所有文件的 SRC_URI。
是否有更简单的方法无需下载文件即可获得相同的结果？
我不确定 bitbake 是否支持这样的功能。理想情况下，我正在寻找一个命令，该命令打印出包名称并列出具有相应 URL
的所有文件
> bitbake core-image-minimal -c fetchall --print-only
我需要这样的东西，并且在此之前完成了部分工作。
我可以通过执行以下命令生成一个混乱的 URI 列表：
bitbake -g zlib && cat recipe-depends.dot | \
grep -v -e '-native' | grep -v digraph | \
grep -v -e '-image' | awk '{print }' | \
sort | uniq | xargs -I {} -t bitbake -e {} | grep SRC_URI=
这会为您提供食谱中使用的所有 URI 和文件以及一些注释。
这不是一个完美的解决方案，但我会看看是否可以改进它。
通常 bitbake 不提供此类功能。
但我能够创建一个简单的解决方案 创建简单的 .bbclass 文件，将其添加到所有食谱中local.conf 文件，请查看我的步骤以便存档：
步骤：
我们来创建一个classprint-src.bbclass文件用来获取和打印SRC_URI 变量（记得将此 class 文件存储在 conf/bblayers.conf 中可用的层中）：
$ cat print-src.bbclass
python do_print_src () {
# should probably be indented
srcuri = d.getVar('SRC_URI', True).split()
bb.warn("SRC_URI look like: %s" % srcuri)
}
addtask do_print_src before do_fetch
将 INHERIT += "print-src" 添加到您的 conf/local.conf 文件中
编辑： 重要的是使用 bitbake --运行only 选项，允许 运行 指定目标的taskgraph的特定任务（需要使用--运行only选项do_print_src作为 print_src),
编辑：请注意--运行all=RUNALL和--运行only=RUNONLY 是用 Yocto Sumo release 2.5,
引入的
$ bitbake core-image-minimal --runonly print_src
Loaded 1236 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Build Configuration:
BB_VERSION = "1.37.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "universal-4.8"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.5"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-poky
meta-yocto-bsp = "master:13cc30cd7de4841990b600e83e1249c81a5171dd"
Initialising tasks: 100% |##########################################################################################################################################################################| Time: 0:00:00
NOTE: Executing RunQueue Tasks
WARNING: ptest-runner-2.2+gitAUTOINC+49956f65bb-r0 do_print_src: SRC_URI look like: ['git://git.yoctoproject.org/ptest-runner2']
WARNING: grep-3.1-r0 do_print_src: SRC_URI look like: ['http://ftp.gnu.org/gnu/grep/grep-3.1.tar.xz', 'file://0001-Unset-need_charset_alias-when-building-for-musl.patch']
...
...
NOTE: Tasks Summary: Attempted 201 tasks of which 0 didn't need to be rerun and all succeeded.
Summary: There were 202 WARNING messages shown.
请查看示例警告输出日志行：
警告： ptest-运行ner-2.2+gitAUTOINC+49956f65bb-r0 do_print_src：SRC_URI 看起来像：['git://git.yoctoproject.org/ptest-runner2'].
我修补了 poky 以在 downloads 中创建 *.src 个文件，其中包含包的有效提取 URL。
bitbake/lib/bb/fetch2/__init__.py | 19 ++++++++++++++-----
1 file changed, 14 insertions(+), 5 deletions(-)
diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index b853da30bd..03e84e0919 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -1257,16 +1257,16 @@ class FetchData(object):
# Note: .done and .lock files should always be in DL_DIR whereas localpath may not be.
if self.localpath and self.localpath.startswith(dldir):
- basepath = self.localpath
+ self.basepath = self.localpath
elif self.localpath:
- basepath = dldir + os.sep + os.path.basename(self.localpath)
+ self.basepath = dldir + os.sep + os.path.basename(self.localpath)
elif self.basepath or self.basename:
- basepath = dldir + os.sep + (self.basepath or self.basename)
+ self.basepath = dldir + os.sep + (self.basepath or self.basename)
else:
bb.fatal("Can't determine lock path for url %s" % url)
- self.donestamp = basepath + '.done'
- self.lockfile = basepath + '.lock'
+ self.donestamp = self.basepath + '.done'
+ self.lockfile = self.basepath + '.lock'
def setup_revisions(self, d):
self.revisions = {}
@@ -1607,6 +1607,15 @@ class Fetch(object):
m = ud.method
localpath = ""
+ p = "%s.src"%ud.basepath
+ d = os.path.dirname(p)
+ if d != '':
+ bb.utils.mkdirhier(d)
+ with open(p, 'wb') as f:
+ data = "%s" % ud.url
+ f.write(bytes(data, 'ASCII'))
+ return True
+
if ud.lockfile:
lf = bb.utils.lockfile(ud.lockfile)
运行 bitbake core-image-minimal -c fetchall 结果：
$> find downloads/ -name '*.src' | head -n 5
downloads/lzop-1.03.tar.gz.src
downloads/libtheora-1.1.1.tar.bz2.src
downloads/mpfr-3.1.5.tar.xz.src
downloads/makedevs.c.src
downloads/expat-2.2.0.tar.bz2.src
这不是最佳解决方案，但我希望这样的功能能够进入主流。
我根据题中的解写了脚本。
基本上我需要制作一个包含所有源的 cscope 项目，因此我需要列出所有 URI 并克隆所有 repos。
我写了 2 个脚本 1) 列出所有 SRC_URI 和分支
2) 在单个目录中安装（克隆）所有代码
listURI.sh
#!/bin/bash
TMP_FILE="___x__2242230p.txt"
SRCH_FILE="log.do_fetch"
TIME=$(date +%Y-%m-%d-%H-%M)
OUT_FILE="Project-$TIME.txt"
DEF_BRANCH="master"
BRANCH=""
SRC_URI=""
function usage {
echo "Usage : [=10=] < -f | -g > <Component-List-file>"
echo "Options :"
echo " -f : Run bitbake fetch and collect URI"
echo " -g : Get URI from Already fetched Project"
}
if [ $# -eq 0 ]
then
usage
exit
fi
if [ "" == "-f" ] || [ "" == "-g" ];then
if [ "" == "-f" ] && [ -z "" ];then
echo "Should pass Project-Name after -f Option"
exit
fi
else
echo "Unknown Option"
usage
exit
fi
POKYROOTDIR=`basename "$PWD"`
#echo $POKYROOTDIR
if [[ $POKYROOTDIR != "build" ]];then
echo "You are not on poky/build (Sourced Poky) Directory"
exit 0
fi
POKYROOTDIR=$PWD
if [ "" == "-f" ];then
echo "Running === bitbake -c fetchall -f --no-setscene =="
bitbake -c fetchall -f --no-setscene
fi
if [ "" == "-g" ];then
if [ ! -d tmp/work ];then
echo " Please run == bitbake -c fetchall <image> -f --no-setscene == before this"
usage
exit
fi
fi
echo "Looking for URIs --- It may take couple of minuites!"
rm -rf $OUT_FILE
cd tmp/work
find . -name $SRCH_FILE -exec grep -i 'DEBUG: For url git' {} \; -print > $POKYROOTDIR/$TMP_FILE
cd $POKYROOTDIR
while IFS= read -r line
do
#echo "$line"
BRANCH=$(echo $line | awk -F"branch=" '/branch=/{print }' | sed -e 's/;/ /' | awk '{print }')
SRC_URI=$(echo $line | cut -d';' -f1-1 | awk -F"url" '/url/{print }' | awk '{print }' | sed -e 's/git:/ssh:/')
if [ ! -z "$BRANCH" ] && [ ! -z "$SRC_URI" ];then
echo "$BRANCH $SRC_URI" >> $OUT_FILE
elif [ ! -z "$SRC_URI" ];then
echo "$DEF_BRANCH $SRC_URI" >> $OUT_FILE
fi
if [ ! -z "$BRANCH" ];then
echo "Found URI : BRANCH:$BRANCH URI:$SRC_URI"
elif [ ! -z "$SRC_URI" ];then
echo "Found URI : BRANCH:Default URI:$SRC_URI"
fi
#echo "$BRANCH $SRC_URI" >> $OUT_FILE
done < "$TMP_FILE"
#echo "=================== OUT PUT ==========================="
#cat $OUT_FILE
rm -rf $TMP_FILE
echo "----------------INFO-----------------"
echo "Project Creation: Success"
echo "Project Path : $POKYROOTDIR"
echo "Project file : $OUT_FILE"
echo "-------------------------------------"
installURI.sh
#!/bin/bash
function usage {
echo "Usage : [=11=] <List-file>"
}
if [ $# -eq 0 ]
then
usage
exit
fi
if [ -z "" ];then
usage
exit
fi
if [ ! -z "$(ls -A $PWD)" ]; then
echo "Directory ($PWD) must be Empty.... else it will mess up project creation"
exit
fi
if [ ! -f ];then
echo "list file () not found"
usage
exit
fi
filename=
while read -r line; do
name="$line"
echo "Fetching Projects"
git clone -b $line
done < "$filename"
echo "----------------INFO-----------------"
echo "Project Creation: Success"
echo "Project Path : $PWD"
echo "-------------------------------------"
有点手动和直接的方法是从头开始构建您的图像，复制所有提取日志，然后通过它。
$ mkdir fetch_logs
$ find tmp/work/ -type f -name log.do_fetch* | cpio -pdm fetch_logs
现在 grep "Fetch(ing/er)" 在此 fetch_logs 中，它将提供有关 URL 您的食谱用于获取源的信息。这将有助于查看用于在基于 PREMIRROR 的构建中获取的最终 URI。
注意：如果您启用了 sstate-cache 并正常运行，您可能根本看不到 do_fetch。
我想列出烘焙图像时 bitbake 将获取的所有文件。
目前，我能够通过执行 bitbake core-image-minimal -c fetchall 然后解析日志文件来获取烘焙 Yocto 图像所需的所有文件的 SRC_URI。
是否有更简单的方法无需下载文件即可获得相同的结果？
我不确定 bitbake 是否支持这样的功能。理想情况下，我正在寻找一个命令，该命令打印出包名称并列出具有相应 URL
的所有文件> bitbake core-image-minimal -c fetchall --print-only
我需要这样的东西，并且在此之前完成了部分工作。
我可以通过执行以下命令生成一个混乱的 URI 列表：
bitbake -g zlib && cat recipe-depends.dot | \
grep -v -e '-native' | grep -v digraph | \
grep -v -e '-image' | awk '{print }' | \
sort | uniq | xargs -I {} -t bitbake -e {} | grep SRC_URI=
这会为您提供食谱中使用的所有 URI 和文件以及一些注释。
这不是一个完美的解决方案，但我会看看是否可以改进它。
通常 bitbake 不提供此类功能。
但我能够创建一个简单的解决方案 创建简单的 .bbclass 文件，将其添加到所有食谱中local.conf 文件，请查看我的步骤以便存档：
步骤：
我们来创建一个classprint-src.bbclass文件用来获取和打印SRC_URI 变量（记得将此 class 文件存储在 conf/bblayers.conf 中可用的层中）：
$ cat print-src.bbclass python do_print_src () { # should probably be indented srcuri = d.getVar('SRC_URI', True).split() bb.warn("SRC_URI look like: %s" % srcuri) } addtask do_print_src before do_fetch将 INHERIT += "print-src" 添加到您的 conf/local.conf 文件中
编辑： 重要的是使用 bitbake --运行only 选项，允许 运行 指定目标的taskgraph的特定任务（需要使用--运行only选项do_print_src作为 print_src),
编辑：请注意--运行all=RUNALL和--运行only=RUNONLY 是用 Yocto Sumo release 2.5,
引入的$ bitbake core-image-minimal --runonly print_src
Loaded 1236 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Build Configuration:
BB_VERSION = "1.37.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "universal-4.8"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.5"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-poky
meta-yocto-bsp = "master:13cc30cd7de4841990b600e83e1249c81a5171dd"
Initialising tasks: 100% |##########################################################################################################################################################################| Time: 0:00:00
NOTE: Executing RunQueue Tasks
WARNING: ptest-runner-2.2+gitAUTOINC+49956f65bb-r0 do_print_src: SRC_URI look like: ['git://git.yoctoproject.org/ptest-runner2']
WARNING: grep-3.1-r0 do_print_src: SRC_URI look like: ['http://ftp.gnu.org/gnu/grep/grep-3.1.tar.xz', 'file://0001-Unset-need_charset_alias-when-building-for-musl.patch']
...
...
NOTE: Tasks Summary: Attempted 201 tasks of which 0 didn't need to be rerun and all succeeded.
Summary: There were 202 WARNING messages shown.
请查看示例警告输出日志行：
警告： ptest-运行ner-2.2+gitAUTOINC+49956f65bb-r0 do_print_src：SRC_URI 看起来像：['git://git.yoctoproject.org/ptest-runner2'].
我修补了 poky 以在 downloads 中创建 *.src 个文件，其中包含包的有效提取 URL。
bitbake/lib/bb/fetch2/__init__.py | 19 ++++++++++++++-----
1 file changed, 14 insertions(+), 5 deletions(-)
diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index b853da30bd..03e84e0919 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -1257,16 +1257,16 @@ class FetchData(object):
# Note: .done and .lock files should always be in DL_DIR whereas localpath may not be.
if self.localpath and self.localpath.startswith(dldir):
- basepath = self.localpath
+ self.basepath = self.localpath
elif self.localpath:
- basepath = dldir + os.sep + os.path.basename(self.localpath)
+ self.basepath = dldir + os.sep + os.path.basename(self.localpath)
elif self.basepath or self.basename:
- basepath = dldir + os.sep + (self.basepath or self.basename)
+ self.basepath = dldir + os.sep + (self.basepath or self.basename)
else:
bb.fatal("Can't determine lock path for url %s" % url)
- self.donestamp = basepath + '.done'
- self.lockfile = basepath + '.lock'
+ self.donestamp = self.basepath + '.done'
+ self.lockfile = self.basepath + '.lock'
def setup_revisions(self, d):
self.revisions = {}
@@ -1607,6 +1607,15 @@ class Fetch(object):
m = ud.method
localpath = ""
+ p = "%s.src"%ud.basepath
+ d = os.path.dirname(p)
+ if d != '':
+ bb.utils.mkdirhier(d)
+ with open(p, 'wb') as f:
+ data = "%s" % ud.url
+ f.write(bytes(data, 'ASCII'))
+ return True
+
if ud.lockfile:
lf = bb.utils.lockfile(ud.lockfile)
运行 bitbake core-image-minimal -c fetchall 结果：
$> find downloads/ -name '*.src' | head -n 5
downloads/lzop-1.03.tar.gz.src
downloads/libtheora-1.1.1.tar.bz2.src
downloads/mpfr-3.1.5.tar.xz.src
downloads/makedevs.c.src
downloads/expat-2.2.0.tar.bz2.src
这不是最佳解决方案，但我希望这样的功能能够进入主流。
我根据题中的解写了脚本。 基本上我需要制作一个包含所有源的 cscope 项目，因此我需要列出所有 URI 并克隆所有 repos。 我写了 2 个脚本 1) 列出所有 SRC_URI 和分支 2) 在单个目录中安装（克隆）所有代码
listURI.sh
#!/bin/bash
TMP_FILE="___x__2242230p.txt"
SRCH_FILE="log.do_fetch"
TIME=$(date +%Y-%m-%d-%H-%M)
OUT_FILE="Project-$TIME.txt"
DEF_BRANCH="master"
BRANCH=""
SRC_URI=""
function usage {
echo "Usage : [=10=] < -f | -g > <Component-List-file>"
echo "Options :"
echo " -f : Run bitbake fetch and collect URI"
echo " -g : Get URI from Already fetched Project"
}
if [ $# -eq 0 ]
then
usage
exit
fi
if [ "" == "-f" ] || [ "" == "-g" ];then
if [ "" == "-f" ] && [ -z "" ];then
echo "Should pass Project-Name after -f Option"
exit
fi
else
echo "Unknown Option"
usage
exit
fi
POKYROOTDIR=`basename "$PWD"`
#echo $POKYROOTDIR
if [[ $POKYROOTDIR != "build" ]];then
echo "You are not on poky/build (Sourced Poky) Directory"
exit 0
fi
POKYROOTDIR=$PWD
if [ "" == "-f" ];then
echo "Running === bitbake -c fetchall -f --no-setscene =="
bitbake -c fetchall -f --no-setscene
fi
if [ "" == "-g" ];then
if [ ! -d tmp/work ];then
echo " Please run == bitbake -c fetchall <image> -f --no-setscene == before this"
usage
exit
fi
fi
echo "Looking for URIs --- It may take couple of minuites!"
rm -rf $OUT_FILE
cd tmp/work
find . -name $SRCH_FILE -exec grep -i 'DEBUG: For url git' {} \; -print > $POKYROOTDIR/$TMP_FILE
cd $POKYROOTDIR
while IFS= read -r line
do
#echo "$line"
BRANCH=$(echo $line | awk -F"branch=" '/branch=/{print }' | sed -e 's/;/ /' | awk '{print }')
SRC_URI=$(echo $line | cut -d';' -f1-1 | awk -F"url" '/url/{print }' | awk '{print }' | sed -e 's/git:/ssh:/')
if [ ! -z "$BRANCH" ] && [ ! -z "$SRC_URI" ];then
echo "$BRANCH $SRC_URI" >> $OUT_FILE
elif [ ! -z "$SRC_URI" ];then
echo "$DEF_BRANCH $SRC_URI" >> $OUT_FILE
fi
if [ ! -z "$BRANCH" ];then
echo "Found URI : BRANCH:$BRANCH URI:$SRC_URI"
elif [ ! -z "$SRC_URI" ];then
echo "Found URI : BRANCH:Default URI:$SRC_URI"
fi
#echo "$BRANCH $SRC_URI" >> $OUT_FILE
done < "$TMP_FILE"
#echo "=================== OUT PUT ==========================="
#cat $OUT_FILE
rm -rf $TMP_FILE
echo "----------------INFO-----------------"
echo "Project Creation: Success"
echo "Project Path : $POKYROOTDIR"
echo "Project file : $OUT_FILE"
echo "-------------------------------------"
installURI.sh
#!/bin/bash
function usage {
echo "Usage : [=11=] <List-file>"
}
if [ $# -eq 0 ]
then
usage
exit
fi
if [ -z "" ];then
usage
exit
fi
if [ ! -z "$(ls -A $PWD)" ]; then
echo "Directory ($PWD) must be Empty.... else it will mess up project creation"
exit
fi
if [ ! -f ];then
echo "list file () not found"
usage
exit
fi
filename=
while read -r line; do
name="$line"
echo "Fetching Projects"
git clone -b $line
done < "$filename"
echo "----------------INFO-----------------"
echo "Project Creation: Success"
echo "Project Path : $PWD"
echo "-------------------------------------"
有点手动和直接的方法是从头开始构建您的图像，复制所有提取日志，然后通过它。
$ mkdir fetch_logs
$ find tmp/work/ -type f -name log.do_fetch* | cpio -pdm fetch_logs
现在 grep "Fetch(ing/er)" 在此 fetch_logs 中，它将提供有关 URL 您的食谱用于获取源的信息。这将有助于查看用于在基于 PREMIRROR 的构建中获取的最终 URI。
注意：如果您启用了 sstate-cache 并正常运行，您可能根本看不到 do_fetch。