Puppeteer 脚本在本地工作,但在 EC2 AWS 上不工作
Puppeteer Script Works Local But Not On EC2 AWS
我 运行 EC2 AWS 上的以下脚本与 Amazon Linux AMI
const puppeteer = require('puppeteer');
url_ = 'https://www.barchart.com/futures/quotes/ES*0/interactive-chart/fullscreen';
func()
async function func() {
console.log(0)
const browser = await puppeteer.launch();
console.log(1)
const page = await browser.newPage();
console.log(2)
await page.goto(url_);
console.log(page)
return page
}
从今天开始,由于没有在 EC2 上进行任何更改或新安装,该脚本直到昨天才停止工作。
本地计算机上的相同脚本仍然有效。
而不是在 EC2 上给出以下错误:
(node:12636) UnhandledPromiseRejectionWarning: Error: Navigation failed because browser has disconnected!
at CDPSession.LifecycleWatcher._eventListeners.helper.addEventListener (/home/ec2-user/hd/node_modules/puppeteer/lib/LifecycleWatcher.js:47:107)
at CDPSession.emit (events.js:189:13)
at CDPSession._onClosed (/home/ec2-user/hd/node_modules/puppeteer/lib/Connection.js:215:10)
at Connection._onClose (/home/ec2-user/hd/node_modules/puppeteer/lib/Connection.js:138:15)
at WebSocketTransport._ws.addEventListener.event (/home/ec2-user/hd/node_modules/puppeteer/lib/WebSocketTransport.js:45:22)
at WebSocket.onClose (/home/ec2-user/hd/node_modules/ws/lib/event-target.js:124:16)
at WebSocket.emit (events.js:189:13)
at WebSocket.emitClose (/home/ec2-user/hd/node_modules/ws/lib/websocket.js:191:10)
at Socket.socketOnClose (/home/ec2-user/hd/node_modules/ws/lib/websocket.js:850:15)
at Socket.emit (events.js:189:13)
-- ASYNC --
at Frame. (/home/ec2-user/hd/node_modules/puppeteer/lib/helper.js:110:27)
at Page.goto (/home/ec2-user/hd/node_modules/puppeteer/lib/Page.js:656:49)
at Page. (/home/ec2-user/hd/node_modules/puppeteer/lib/helper.js:111:23)
at func (/home/ec2-user/hd/scrape_bk.js:19:13)
at process._tickCallback (internal/process/next_tick.js:68:7)
(node:12636) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:12636) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
如果我给这个 URL (http://www.google.com) 我在 EC2 上得到这个错误(虽然它仍然在本地机器上工作):
(node:12938) UnhandledPromiseRejectionWarning: Error: Page crashed!
at Page._onTargetCrashed (/home/ec2-user/hd/node_modules/puppeteer/lib/Page.js:185:24)
at CDPSession.Page.client.on.event (/home/ec2-user/hd/node_modules/puppeteer/lib/Page.js:140:56)
at CDPSession.emit (events.js:189:13)
at CDPSession._onMessage (/home/ec2-user/hd/node_modules/puppeteer/lib/Connection.js:200:12)
at Connection._onMessage (/home/ec2-user/hd/node_modules/puppeteer/lib/Connection.js:112:17)
at WebSocketTransport._ws.addEventListener.event (/home/ec2-user/hd/node_modules/puppeteer/lib/WebSocketTransport.js:41:24)
at WebSocket.onMessage (/home/ec2-user/hd/node_modules/ws/lib/event-target.js:120:16)
at WebSocket.emit (events.js:189:13)
at Receiver.receiverOnMessage (/home/ec2-user/hd/node_modules/ws/lib/websocket.js:789:20)
at Receiver.emit (events.js:189:13)
(node:12938) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:12938) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
(node:12938) UnhandledPromiseRejectionWarning: Error: Navigation failed because browser has disconnected!
at CDPSession.LifecycleWatcher._eventListeners.helper.addEventListener (/home/ec2-user/hd/node_modules/puppeteer/lib/LifecycleWatcher.js:47:107)
at CDPSession.emit (events.js:189:13)
at CDPSession._onClosed (/home/ec2-user/hd/node_modules/puppeteer/lib/Connection.js:215:10)
at Connection._onClose (/home/ec2-user/hd/node_modules/puppeteer/lib/Connection.js:138:15)
at WebSocketTransport._ws.addEventListener.event (/home/ec2-user/hd/node_modules/puppeteer/lib/WebSocketTransport.js:45:22)
at WebSocket.onClose (/home/ec2-user/hd/node_modules/ws/lib/event-target.js:124:16)
at WebSocket.emit (events.js:189:13)
at WebSocket.emitClose (/home/ec2-user/hd/node_modules/ws/lib/websocket.js:191:10)
at Socket.socketOnClose (/home/ec2-user/hd/node_modules/ws/lib/websocket.js:850:15)
at Socket.emit (events.js:189:13)
-- ASYNC --
at Frame. (/home/ec2-user/hd/node_modules/puppeteer/lib/helper.js:110:27)
at Page.goto (/home/ec2-user/hd/node_modules/puppeteer/lib/Page.js:656:49)
at Page. (/home/ec2-user/hd/node_modules/puppeteer/lib/helper.js:111:23)
at func (/home/ec2-user/hd/scrape_bk.js:13:13)
at process._tickCallback (internal/process/next_tick.js:68:7)
(node:12938) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
我不知道如何确定问题的根源,因为它突然出现,没有对机器或代码进行任何更改,并且在我的本地机器上它工作正常。
puppeteer 似乎能够启动浏览器,但它随后随机崩溃。这不应该发生在普通机器上,并且可能由于资源限制而发生。
您应该检查您的系统 运行 是否正确。特别是,您可能想要检查:
- 内存使用
- CPU 用法
- 磁盘使用情况
内存太少或电量不足CPU可能会导致随机崩溃以及硬盘不足space。
我 运行 EC2 AWS 上的以下脚本与 Amazon Linux AMI
const puppeteer = require('puppeteer');
url_ = 'https://www.barchart.com/futures/quotes/ES*0/interactive-chart/fullscreen';
func()
async function func() {
console.log(0)
const browser = await puppeteer.launch();
console.log(1)
const page = await browser.newPage();
console.log(2)
await page.goto(url_);
console.log(page)
return page
}
从今天开始,由于没有在 EC2 上进行任何更改或新安装,该脚本直到昨天才停止工作。
本地计算机上的相同脚本仍然有效。
而不是在 EC2 上给出以下错误:
(node:12636) UnhandledPromiseRejectionWarning: Error: Navigation failed because browser has disconnected! at CDPSession.LifecycleWatcher._eventListeners.helper.addEventListener (/home/ec2-user/hd/node_modules/puppeteer/lib/LifecycleWatcher.js:47:107) at CDPSession.emit (events.js:189:13) at CDPSession._onClosed (/home/ec2-user/hd/node_modules/puppeteer/lib/Connection.js:215:10) at Connection._onClose (/home/ec2-user/hd/node_modules/puppeteer/lib/Connection.js:138:15) at WebSocketTransport._ws.addEventListener.event (/home/ec2-user/hd/node_modules/puppeteer/lib/WebSocketTransport.js:45:22) at WebSocket.onClose (/home/ec2-user/hd/node_modules/ws/lib/event-target.js:124:16) at WebSocket.emit (events.js:189:13) at WebSocket.emitClose (/home/ec2-user/hd/node_modules/ws/lib/websocket.js:191:10) at Socket.socketOnClose (/home/ec2-user/hd/node_modules/ws/lib/websocket.js:850:15) at Socket.emit (events.js:189:13) -- ASYNC -- at Frame. (/home/ec2-user/hd/node_modules/puppeteer/lib/helper.js:110:27) at Page.goto (/home/ec2-user/hd/node_modules/puppeteer/lib/Page.js:656:49) at Page. (/home/ec2-user/hd/node_modules/puppeteer/lib/helper.js:111:23) at func (/home/ec2-user/hd/scrape_bk.js:19:13) at process._tickCallback (internal/process/next_tick.js:68:7) (node:12636) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1) (node:12636) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
如果我给这个 URL (http://www.google.com) 我在 EC2 上得到这个错误(虽然它仍然在本地机器上工作):
(node:12938) UnhandledPromiseRejectionWarning: Error: Page crashed! at Page._onTargetCrashed (/home/ec2-user/hd/node_modules/puppeteer/lib/Page.js:185:24) at CDPSession.Page.client.on.event (/home/ec2-user/hd/node_modules/puppeteer/lib/Page.js:140:56) at CDPSession.emit (events.js:189:13) at CDPSession._onMessage (/home/ec2-user/hd/node_modules/puppeteer/lib/Connection.js:200:12) at Connection._onMessage (/home/ec2-user/hd/node_modules/puppeteer/lib/Connection.js:112:17) at WebSocketTransport._ws.addEventListener.event (/home/ec2-user/hd/node_modules/puppeteer/lib/WebSocketTransport.js:41:24) at WebSocket.onMessage (/home/ec2-user/hd/node_modules/ws/lib/event-target.js:120:16) at WebSocket.emit (events.js:189:13) at Receiver.receiverOnMessage (/home/ec2-user/hd/node_modules/ws/lib/websocket.js:789:20) at Receiver.emit (events.js:189:13) (node:12938) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1) (node:12938) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code. (node:12938) UnhandledPromiseRejectionWarning: Error: Navigation failed because browser has disconnected! at CDPSession.LifecycleWatcher._eventListeners.helper.addEventListener (/home/ec2-user/hd/node_modules/puppeteer/lib/LifecycleWatcher.js:47:107) at CDPSession.emit (events.js:189:13) at CDPSession._onClosed (/home/ec2-user/hd/node_modules/puppeteer/lib/Connection.js:215:10) at Connection._onClose (/home/ec2-user/hd/node_modules/puppeteer/lib/Connection.js:138:15) at WebSocketTransport._ws.addEventListener.event (/home/ec2-user/hd/node_modules/puppeteer/lib/WebSocketTransport.js:45:22) at WebSocket.onClose (/home/ec2-user/hd/node_modules/ws/lib/event-target.js:124:16) at WebSocket.emit (events.js:189:13) at WebSocket.emitClose (/home/ec2-user/hd/node_modules/ws/lib/websocket.js:191:10) at Socket.socketOnClose (/home/ec2-user/hd/node_modules/ws/lib/websocket.js:850:15) at Socket.emit (events.js:189:13) -- ASYNC -- at Frame. (/home/ec2-user/hd/node_modules/puppeteer/lib/helper.js:110:27) at Page.goto (/home/ec2-user/hd/node_modules/puppeteer/lib/Page.js:656:49) at Page. (/home/ec2-user/hd/node_modules/puppeteer/lib/helper.js:111:23) at func (/home/ec2-user/hd/scrape_bk.js:13:13) at process._tickCallback (internal/process/next_tick.js:68:7) (node:12938) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
我不知道如何确定问题的根源,因为它突然出现,没有对机器或代码进行任何更改,并且在我的本地机器上它工作正常。
puppeteer 似乎能够启动浏览器,但它随后随机崩溃。这不应该发生在普通机器上,并且可能由于资源限制而发生。
您应该检查您的系统 运行 是否正确。特别是,您可能想要检查:
- 内存使用
- CPU 用法
- 磁盘使用情况
内存太少或电量不足CPU可能会导致随机崩溃以及硬盘不足space。