当 运行 业力测试在 Jenkins 代理 docker 图像中时，ChromeHeadless 挂起（使用 Kubernetes 插件）
ChromeHeadless hangs when running karma tests inside Jenkins agent docker image (with Kubernetes plugin)
我有一个 angular 应用程序，它使用默认测试套件运行（使用 karma 启动无头 chrome）。 Jenkins 与 kubernetes 插件一起运行，以在每次提交作业时启动新的 jenkins 代理实例。 （如果这个错误得到修复，我将在 Whosebug 中创建一篇关于 运行 headless chrome tests in docker 容器的文章，因为这似乎是一个常见问题）。
我基于 this 构建（减去量角器部分，因为没有使用量角器）。
在浏览器测试完成之前，一切都会正常构建和运行。
+ npm run test:vt --watch=false --progress=false --browsers=ChromeHeadless --headless --disable-gpu --window-size=800x600 --disable-dev-shm-usage --no-sandbox
> {my_company}/{my_project}docs@0.0.0-PLACEHOLDER test:vt /home/jenkins/agent/workspace/UI_{my_project}_PR-192
> ng test {my_project}
[33m04 09 2019 16:00:58.826:WARN [karma]: [39mNo captured browser, open http://localhost:9876/
[32m04 09 2019 16:00:58.832:INFO [karma-server]: [39mKarma v3.1.4 server started at http://0.0.0.0:9876/
[32m04 09 2019 16:00:58.832:INFO [launcher]: [39mLaunching browsers ChromeHeadlessNoSandbox with concurrency unlimited
[32m04 09 2019 16:00:58.839:INFO [launcher]: [39mStarting browser ChromeHeadless
[33m04 09 2019 16:01:16.236:WARN [karma]: [39mNo captured browser, open http://localhost:9876/
[32m04 09 2019 16:01:16.432:INFO [HeadlessChrome 76.0.3809 (Linux 0.0.0)]: [39mConnected on socket FcDILSSqmJFawfnrAAAA with id 26860248
HeadlessChrome 76.0.3809 (Linux 0.0.0): Executed 0 of 863 SUCCESS (0 secs / 0 secs)
...{bunches 'o lines}...
[1A[2KHeadlessChrome 76.0.3809 (Linux 0.0.0): Executed 796 of 863 (skipped 67) SUCCESS (0 secs / 32.499 secs)
[1A[2KHeadlessChrome 76.0.3809 (Linux 0.0.0): Executed 796 of 863 (skipped 67) SUCCESS (33.59 secs / 32.499 secs)
TOTAL: 796 SUCCESS
TOTAL: 796 SUCCESS
然后终端似乎就挂在这里了。我让它静置 24 小时，没有任何变化。
我对内存和 cpu 使用情况进行了监控。 cpu 下降到噪音水平 (<10mcpu)，就好像它在等待输入一样。所以，我添加了标志“--password-store=basic”，以防止 ChromeHeadless 在等待输入时挂起（我试图找到推荐这个的业力 github 错误报告，但找不到重新找到它）。不幸的是，这没有成功。
关于如何排除或解决此问题的任何想法？
docker 映像是这样构建的（最后两个 运行 命令用于设置 Chrome 二进制文件）：
FROM jenkins/jnlp-slave:3.29-1
ARG DOCKER_VERSION=18.06.1~ce~3-0~debian
ARG DC_VERSION=1.24.1
USER root
RUN apt-get update && \
apt-get install -qq -y --no-install-recommends \
apt-transport-https \
ca-certificates \
curl \
gnupg2 \
software-properties-common && \
curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add - && \
apt-key fingerprint 0EBFCD88 && \
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian stretch stable" && \
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian stretch stable" && \
apt-get update && \
apt-get install -qq -y --no-install-recommends docker-ce=${DOCKER_VERSION} && \
curl -L https://github.com/docker/compose/releases/download/${DC_VERSION}/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose && \
chmod +x /usr/local/bin/docker-compose && \
curl -sL https://deb.nodesource.com/setup_12.x | bash - && \
apt-get install nodejs && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
RUN apt-get install -qq -y --no-install-recommends nodejs npm
# Set the Chrome repo.
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list
# Install Chrome.
RUN apt-get update && apt-get -y install google-chrome-stable
ENTRYPOINT ["jenkins-slave"]
我的 karama.conf 是这样设置的：
module.exports = function (config) {
config.set({
basePath: '',
frameworks: ['jasmine', '@angular-devkit/build-angular'],
plugins: [
require('karma-jasmine'),
require('karma-chrome-launcher'),
require('karma-jasmine-html-reporter'),
require('karma-coverage-istanbul-reporter'),
require('@angular-devkit/build-angular/plugins/karma')
],
client: {
clearContext: false // leave Jasmine Spec Runner output visible in browser
},
coverageIstanbulReporter: {
dir: require('path').join(__dirname, '../../coverage'),
reports: ['html', 'lcovonly'],
fixWebpackSourcePaths: true
},
reporters: ['progress', 'kjhtml'],
port: 9876,
colors: true,
logLevel: config.LOG_INFO,
autoWatch: true,
browsers: ['ChromeHeadlessNoSandbox'],
customLaunchers: {
ChromeHeadlessNoSandbox: {
base: 'ChromeHeadless',
flags: [
"--no-sandbox",
// required to run without privileges in Docker
"--disable-web-security",
"--disable-gpu",
"--remote-debugging-port=9222"
]
}
},
singleRun: false,
junitReporter:{
outputDir:'test-reports',
// results will be saved as $outputDir/$browserName.xml
outputFile:'junit-report.xml',
// if included, results will be saved as $outputDir/$browserName/$outputFile
suite:'',
// suite will become the package name attribute in xml testsuite element
useBrowserName:false,
// add browser name to report and classes names
nameFormatter:undefined,
// function (browser, result) to customize the name attribute in xml testcase element
classNameFormatter:undefined,
// function (browser, result) to customize the classname attribute in xml testcase element
properties:{
} // key value pair of properties to add to the section of the report
}
});
};
运行测试的 Jenkinsfile 部分是：
stage('Test') {
echo "Running tests"
sh 'npm run test:vt --watch=false --progress=false --browsers=ChromeHeadless --headless --disable-gpu --window-size=800x600 --disable-dev-shm-usage --no-sandbox --password-store=basic'
}
我使用与 Jenkins 安装在同一台机器上的 karma 和 CromeHeadless（小型项目和站点）。有了这个配置，我从来没有看到一个 Chrome 实例在测试完成后继续工作。您可以尝试将此添加到您的 karma.conf:
browsers: ['ChromeHeadless'],
singleRun: true,
restartOnFileChange: true
例如，我的 karma.conf 看起来像这样（只发送重要行）：
customLaunchers: {
ChromeHeadless: {
base: 'Chrome',
flags: [
'--headless',
'--disable-gpu',
'--no-sandbox',
'--remote-debugging-port=9222',
]
}
},
browsers: ['ChromeHeadless'],
singleRun: true,
restartOnFileChange: true
单个运行选项意味着"If true, Karma will start and capture all configured browsers, run tests and then exit with an exit code of 0 or 1 depending on whether all tests passed or any tests failed."（来自业力配置文件site）
我有一个 angular 应用程序，它使用默认测试套件运行（使用 karma 启动无头 chrome）。 Jenkins 与 kubernetes 插件一起运行，以在每次提交作业时启动新的 jenkins 代理实例。 （如果这个错误得到修复，我将在 Whosebug 中创建一篇关于 运行 headless chrome tests in docker 容器的文章，因为这似乎是一个常见问题）。
我基于 this 构建（减去量角器部分，因为没有使用量角器）。
在浏览器测试完成之前，一切都会正常构建和运行。
+ npm run test:vt --watch=false --progress=false --browsers=ChromeHeadless --headless --disable-gpu --window-size=800x600 --disable-dev-shm-usage --no-sandbox
> {my_company}/{my_project}docs@0.0.0-PLACEHOLDER test:vt /home/jenkins/agent/workspace/UI_{my_project}_PR-192
> ng test {my_project}
[33m04 09 2019 16:00:58.826:WARN [karma]: [39mNo captured browser, open http://localhost:9876/
[32m04 09 2019 16:00:58.832:INFO [karma-server]: [39mKarma v3.1.4 server started at http://0.0.0.0:9876/
[32m04 09 2019 16:00:58.832:INFO [launcher]: [39mLaunching browsers ChromeHeadlessNoSandbox with concurrency unlimited
[32m04 09 2019 16:00:58.839:INFO [launcher]: [39mStarting browser ChromeHeadless
[33m04 09 2019 16:01:16.236:WARN [karma]: [39mNo captured browser, open http://localhost:9876/
[32m04 09 2019 16:01:16.432:INFO [HeadlessChrome 76.0.3809 (Linux 0.0.0)]: [39mConnected on socket FcDILSSqmJFawfnrAAAA with id 26860248
HeadlessChrome 76.0.3809 (Linux 0.0.0): Executed 0 of 863 SUCCESS (0 secs / 0 secs)
...{bunches 'o lines}...
[1A[2KHeadlessChrome 76.0.3809 (Linux 0.0.0): Executed 796 of 863 (skipped 67) SUCCESS (0 secs / 32.499 secs)
[1A[2KHeadlessChrome 76.0.3809 (Linux 0.0.0): Executed 796 of 863 (skipped 67) SUCCESS (33.59 secs / 32.499 secs)
TOTAL: 796 SUCCESS
TOTAL: 796 SUCCESS
然后终端似乎就挂在这里了。我让它静置 24 小时，没有任何变化。
我对内存和 cpu 使用情况进行了监控。 cpu 下降到噪音水平 (<10mcpu)，就好像它在等待输入一样。所以，我添加了标志“--password-store=basic”，以防止 ChromeHeadless 在等待输入时挂起（我试图找到推荐这个的业力 github 错误报告，但找不到重新找到它）。不幸的是，这没有成功。
关于如何排除或解决此问题的任何想法？
docker 映像是这样构建的（最后两个 运行 命令用于设置 Chrome 二进制文件）：
FROM jenkins/jnlp-slave:3.29-1
ARG DOCKER_VERSION=18.06.1~ce~3-0~debian
ARG DC_VERSION=1.24.1
USER root
RUN apt-get update && \
apt-get install -qq -y --no-install-recommends \
apt-transport-https \
ca-certificates \
curl \
gnupg2 \
software-properties-common && \
curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add - && \
apt-key fingerprint 0EBFCD88 && \
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian stretch stable" && \
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian stretch stable" && \
apt-get update && \
apt-get install -qq -y --no-install-recommends docker-ce=${DOCKER_VERSION} && \
curl -L https://github.com/docker/compose/releases/download/${DC_VERSION}/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose && \
chmod +x /usr/local/bin/docker-compose && \
curl -sL https://deb.nodesource.com/setup_12.x | bash - && \
apt-get install nodejs && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
RUN apt-get install -qq -y --no-install-recommends nodejs npm
# Set the Chrome repo.
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list
# Install Chrome.
RUN apt-get update && apt-get -y install google-chrome-stable
ENTRYPOINT ["jenkins-slave"]
我的 karama.conf 是这样设置的：
module.exports = function (config) {
config.set({
basePath: '',
frameworks: ['jasmine', '@angular-devkit/build-angular'],
plugins: [
require('karma-jasmine'),
require('karma-chrome-launcher'),
require('karma-jasmine-html-reporter'),
require('karma-coverage-istanbul-reporter'),
require('@angular-devkit/build-angular/plugins/karma')
],
client: {
clearContext: false // leave Jasmine Spec Runner output visible in browser
},
coverageIstanbulReporter: {
dir: require('path').join(__dirname, '../../coverage'),
reports: ['html', 'lcovonly'],
fixWebpackSourcePaths: true
},
reporters: ['progress', 'kjhtml'],
port: 9876,
colors: true,
logLevel: config.LOG_INFO,
autoWatch: true,
browsers: ['ChromeHeadlessNoSandbox'],
customLaunchers: {
ChromeHeadlessNoSandbox: {
base: 'ChromeHeadless',
flags: [
"--no-sandbox",
// required to run without privileges in Docker
"--disable-web-security",
"--disable-gpu",
"--remote-debugging-port=9222"
]
}
},
singleRun: false,
junitReporter:{
outputDir:'test-reports',
// results will be saved as $outputDir/$browserName.xml
outputFile:'junit-report.xml',
// if included, results will be saved as $outputDir/$browserName/$outputFile
suite:'',
// suite will become the package name attribute in xml testsuite element
useBrowserName:false,
// add browser name to report and classes names
nameFormatter:undefined,
// function (browser, result) to customize the name attribute in xml testcase element
classNameFormatter:undefined,
// function (browser, result) to customize the classname attribute in xml testcase element
properties:{
} // key value pair of properties to add to the section of the report
}
});
};
运行测试的 Jenkinsfile 部分是：
stage('Test') {
echo "Running tests"
sh 'npm run test:vt --watch=false --progress=false --browsers=ChromeHeadless --headless --disable-gpu --window-size=800x600 --disable-dev-shm-usage --no-sandbox --password-store=basic'
}
我使用与 Jenkins 安装在同一台机器上的 karma 和 CromeHeadless（小型项目和站点）。有了这个配置，我从来没有看到一个 Chrome 实例在测试完成后继续工作。您可以尝试将此添加到您的 karma.conf:
browsers: ['ChromeHeadless'],
singleRun: true,
restartOnFileChange: true
例如，我的 karma.conf 看起来像这样（只发送重要行）：
customLaunchers: {
ChromeHeadless: {
base: 'Chrome',
flags: [
'--headless',
'--disable-gpu',
'--no-sandbox',
'--remote-debugging-port=9222',
]
}
},
browsers: ['ChromeHeadless'],
singleRun: true,
restartOnFileChange: true
单个运行选项意味着"If true, Karma will start and capture all configured browsers, run tests and then exit with an exit code of 0 or 1 depending on whether all tests passed or any tests failed."（来自业力配置文件site）