初始化 selenium webdriver 时如何修复 python-selenium 错误 "connection refused"?

How to fix python-selenium error "connection refused" when initializing a selenium webdriver?

我正在 运行 在非 public 网页上进行非常复杂的 python-selenium 测试。在大多数情况下,这些测试 运行 很好,但有时其中一个测试会在 webdriver 本身的初始化过程中失败。

提示:尝试初始化网络驱动程序时会发生此错误,即在执行以下操作时:

# Start of the tests
mydriver =  webdriver.Firefox(firefox_profile=profile, log_path=logfile)
# ERROR HAPPENS HERE

# Doing other stuff here
....
# Doing tests here
....
# Doing shutdown here
mydriver.quit()

这是此类错误的完整示例:

___________ ERROR at setup of TestSuite.test_synaptic_events_fitting ___________

>   lambda: ihook(item=item, **kwds),
    when=when,
            )

/usr/local/lib/python2.7/dist-packages/flaky/flaky_pytest_plugin.py:273: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
conftest.py:157: in basedriver
    mydriver = firefox.get_driver(*args)
bsp_usecase_tests/tools/firefox.py:44: in get_driver
    driver = webdriver.Firefox(firefox_profile=profile, log_path=logfile)  #### INITIALIZING OF WEBDRIVER HERE
/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py:158: in __init__
    keep_alive=True)
/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py:154: in __init__
    self.start_session(desired_capabilities, browser_profile)
/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py:243: in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py:311: in execute
    self.error_handler.check_response(response)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <selenium.webdriver.remote.errorhandler.ErrorHandler object at 0x7efd3b702f90>
response = {'status': 500, 'value': '{"value":{"error":"unknown error","message":"connection refused","stacktrace":"stack backtra...s::imp::thread::{{impl}}::new::thread_start\n                        at /checkout/src/libstd/sys/unix/thread.rs:84"}}'}

    def check_response(self, response):
        """
            Checks that a JSON response from the WebDriver does not have an error.

            :Args:
             - response - The JSON response from the WebDriver server as a dictionary
               object.

            :Raises: If the response contains an error message.
            """
        status = response.get('status', None)
        if status is None or status == ErrorCode.SUCCESS:
            return
        value = None
        message = response.get("message", "")
        screen = response.get("screen", "")
        stacktrace = None
        if isinstance(status, int):
            value_json = response.get('value', None)
            if value_json and isinstance(value_json, basestring):
                import json
                try:
                    value = json.loads(value_json)
                    if len(value.keys()) == 1:
                        value = value['value']
                    status = value.get('error', None)
                    if status is None:
                        status = value["status"]
                        message = value["value"]
                        if not isinstance(message, basestring):
                            value = message
                            message = message.get('message')
                    else:
                        message = value.get('message', None)
                except ValueError:
                    pass

        exception_class = ErrorInResponseException
        if status in ErrorCode.NO_SUCH_ELEMENT:
            exception_class = NoSuchElementException
        elif status in ErrorCode.NO_SUCH_FRAME:
            exception_class = NoSuchFrameException
        elif status in ErrorCode.NO_SUCH_WINDOW:
            exception_class = NoSuchWindowException
        elif status in ErrorCode.STALE_ELEMENT_REFERENCE:
            exception_class = StaleElementReferenceException
        elif status in ErrorCode.ELEMENT_NOT_VISIBLE:
            exception_class = ElementNotVisibleException
        elif status in ErrorCode.INVALID_ELEMENT_STATE:
            exception_class = InvalidElementStateException
        elif status in ErrorCode.INVALID_SELECTOR \
                or status in ErrorCode.INVALID_XPATH_SELECTOR \
                or status in ErrorCode.INVALID_XPATH_SELECTOR_RETURN_TYPER:
            exception_class = InvalidSelectorException
        elif status in ErrorCode.ELEMENT_IS_NOT_SELECTABLE:
            exception_class = ElementNotSelectableException
        elif status in ErrorCode.ELEMENT_NOT_INTERACTABLE:
            exception_class = ElementNotInteractableException
        elif status in ErrorCode.INVALID_COOKIE_DOMAIN:
            exception_class = InvalidCookieDomainException
        elif status in ErrorCode.UNABLE_TO_SET_COOKIE:
            exception_class = UnableToSetCookieException
        elif status in ErrorCode.TIMEOUT:
            exception_class = TimeoutException
        elif status in ErrorCode.SCRIPT_TIMEOUT:
            exception_class = TimeoutException
        elif status in ErrorCode.UNKNOWN_ERROR:
            exception_class = WebDriverException
        elif status in ErrorCode.UNEXPECTED_ALERT_OPEN:
            exception_class = UnexpectedAlertPresentException
        elif status in ErrorCode.NO_ALERT_OPEN:
            exception_class = NoAlertPresentException
        elif status in ErrorCode.IME_NOT_AVAILABLE:
            exception_class = ImeNotAvailableException
        elif status in ErrorCode.IME_ENGINE_ACTIVATION_FAILED:
            exception_class = ImeActivationFailedException
        elif status in ErrorCode.MOVE_TARGET_OUT_OF_BOUNDS:
            exception_class = MoveTargetOutOfBoundsException
        elif status in ErrorCode.JAVASCRIPT_ERROR:
            exception_class = JavascriptException
        elif status in ErrorCode.SESSION_NOT_CREATED:
            exception_class = SessionNotCreatedException
        elif status in ErrorCode.INVALID_ARGUMENT:
            exception_class = InvalidArgumentException
        elif status in ErrorCode.NO_SUCH_COOKIE:
            exception_class = NoSuchCookieException
        elif status in ErrorCode.UNABLE_TO_CAPTURE_SCREEN:
            exception_class = ScreenshotException
        elif status in ErrorCode.ELEMENT_CLICK_INTERCEPTED:
            exception_class = ElementClickInterceptedException
        elif status in ErrorCode.INSECURE_CERTIFICATE:
            exception_class = InsecureCertificateException
        elif status in ErrorCode.INVALID_COORDINATES:
            exception_class = InvalidCoordinatesException
        elif status in ErrorCode.INVALID_SESSION_ID:
            exception_class = InvalidSessionIdException
        elif status in ErrorCode.UNKNOWN_METHOD:
            exception_class = UnknownMethodException
        else:
            exception_class = WebDriverException
        if value == '' or value is None:
            value = response['value']
        if isinstance(value, basestring):
            if exception_class == ErrorInResponseException:
                raise exception_class(response, value)
            raise exception_class(value)
        if message == "" and 'message' in value:
            message = value['message']

        screen = None
        if 'screen' in value:
            screen = value['screen']

        stacktrace = None
        if 'stackTrace' in value and value['stackTrace']:
            stacktrace = []
            try:
                for frame in value['stackTrace']:
                    line = self._value_or_default(frame, 'lineNumber', '')
                    file = self._value_or_default(frame, 'fileName', '<anonymous>')
                    if line:
                        file = "%s:%s" % (file, line)
                    meth = self._value_or_default(frame, 'methodName', '<anonymous>')
                    if 'className' in frame:
                        meth = "%s.%s" % (frame['className'], meth)
                    msg = "    at %s (%s)"
                    msg = msg % (meth, file)
                    stacktrace.append(msg)
            except TypeError:
                pass
        if exception_class == ErrorInResponseException:
            raise exception_class(response, message)
        elif exception_class == UnexpectedAlertPresentException and 'alert' in value:
            raise exception_class(message, screen, stacktrace, value['alert'].get('text'))
>       raise exception_class(message, screen, stacktrace)
E       WebDriverException: Message: connection refused

/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py:237: WebDriverException

这些测试 运行 作为 jenkins 计划的一部分在 docker 容器内进行,以确保始终保持完全相同的环境。以下是所用软件包及其版本的列表:

该错误大致出现在所有测试的大约 1% 中。大约有 15 个不同的测试,错误似乎是随机出现的(即不总是相同的测试)。

这是 firefox/selenium/geckodriver 中的错误吗?有没有办法解决这个问题?

下面的代码片段不是我正在使用的代码!这只是关于如何解决我的上述问题的一个想法。这可能是解决我原来问题的好方法吗?

while counter<5:
    try:
        webdriver = webdriver.Firefox(firefox_profile=profile, log_path=logfile) 
        break
    except WebDriverException:
        counter +=1

有更好的方法吗?

这个错误信息...

{'status': 500, 'value': '{"value":{"error":"unknown error","message":"connection refused","stacktrace":"stack backtra...s::imp::thread::{{impl}}::new::thread_start\n at /checkout/src/libstd/sys/unix/thread.rs:84"}}'}

...意味着 GeckoDriver 无法 initiate/spawn 新的 WebBrowsing Session Firefox 浏览器 会话。

comment within the discussion DELETE '/session/{session id}' no longer working @andreastt 中提到:

geckodriver is implicitly ending the (previous) session when the last window closes. If driver.quit() is called as the subsequent command it will fail because the session has already been implicitly deleted.

In these cases GeckoDriver should detect that the session has been closed implicitly after driver.close() or ignore the response from driver.quit() in case the session has already been closed.

在这种情况下,会生成以下跟踪日志:

1505753594121   webdriver::server   DEBUG   Last window was closed, deleting session
1505753594121   webdriver::server   DEBUG   Deleting session
1505753594121   geckodriver::marionette DEBUG   Stopping browser process
1505753594364   webdriver::server   DEBUG   <- 200 OK {"value": []}
1505753594523   webdriver::server   DEBUG   -> DELETE /session/a8312282-af00-4931-94d4-0d401abf01c9 
1505753594524   webdriver::server   DEBUG   <- 500 Internal Server Error {"value":{"error":"session not created","message":"Tried to run command without establishing a connection","stacktrace":"stack backtrace:\n   0:           0x4f388c - backtrace::backtrace::trace::h736111741fa0878e\n   1:           0x4f38c2 - backtrace::capture::Backtrace::new::h63b8a5c0787510c9\n   2:           0x442c61 - webdriver::error::WebDriverError::new::hc4fe6a1ced4e57dd\n   3:           0x42a926 - <webdriver::server::Dispatcher<T, U>>::run::hba9181b5aacf8f04\n   4:           0x402c59 - std::sys_common::backtrace::__rust_begin_short_backtrace::h19de262639927233\n   5:           0x40c065 - std::panicking::try::do_call::h6c1659fc4d01af51\n   6:           0x5e38ec - panic_unwind::__rust_maybe_catch_panic\n                        at /checkout/src/libpanic_unwind/lib.rs:98\n   7:           0x420d32 - <F as alloc::boxed::FnBox<A>>::call_box::h953e5f59694972c5\n   8:           0x5dc00b - alloc::boxed::{{impl}}::call_once<(),()>\n                        at /checkout/src/liballoc/boxed.rs:661\n                         - std::sys_common::thread::start_thread\n                        at /checkout/src/libstd/sys_common/thread.rs:21\n                         - std::sys::imp::thread::{{impl}}::new::thread_start\n                        at /checkout/src/libstd/sys/unix/thread.rs:84"}}
1505753594533   webdriver::server   DEBUG   -> DELETE /session/a8312282-af00-4931-94d4-0d401abf01c9 
1505753594542   webdriver::server   DEBUG   <- 500 Internal Server Error {"value":{"error":"session not created","message":"Tried to run command without establishing a connection","stacktrace":"stack backtrace:\n   0:           0x4f388c - backtrace::backtrace::trace::h736111741fa0878e\n   1:           0x4f38c2 - backtrace::capture::Backtrace::new::h63b8a5c0787510c9\n   2:           0x442c61 - webdriver::error::WebDriverError::new::hc4fe6a1ced4e57dd\n   3:           0x42a926 - <webdriver::server::Dispatcher<T, U>>::run::hba9181b5aacf8f04\n   4:           0x402c59 - std::sys_common::backtrace::__rust_begin_short_backtrace::h19de262639927233\n   5:           0x40c065 - std::panicking::try::do_call::h6c1659fc4d01af51\n   6:           0x5e38ec - panic_unwind::__rust_maybe_catch_panic\n                        at /checkout/src/libpanic_unwind/lib.rs:98\n   7:           0x420d32 - <F as alloc::boxed::FnBox<A>>::call_box::h953e5f59694972c5\n   8:           0x5dc00b - alloc::boxed::{{impl}}::call_once<(),()>\n                        at /checkout/src/liballoc/boxed.rs:661\n                         - std::sys_common::thread::start_thread\n                        at /checkout/src/libstd/sys_common/thread.rs:21\n                         - std::sys::imp::thread::{{impl}}::new::thread_start\n                        at /checkout/src/libstd/sys/unix/thread.rs:84"}}
1505753594549   webdriver::server   DEBUG   -> GET /shutdown 
1505753594551   webdriver::server DEBUG <- 404 Not Found {"value":{"error":"unknown command","message":"GET /shutdown did not match a known command","stacktrace":"stack backtrace:\n 0: 0x4f388c - backtrace::backtrace::trace::h736111741fa0878e\n 1: 0x4f38c2 - backtrace::capture::Backtrace::new::h63b8a5c0787510c9\n 2: 0x442d88 - webdriver::error::WebDriverError::new::hea6d4dbf778b2b24\n 3: 0x43c65f - <webdriver::server::HttpHandler<U> as hyper::server::Handler>::handle::hd03629bd67672697\n 4: 0x403a04 - std::sys_common::backtrace::__rust_begin_short_backtrace::h32e6ff325c0d7f46\n 5: 0x40c036 - std::panicking::try::do_call::h5f902dc1eea01ffe\n 6: 0x5e38ec - panic_unwind::__rust_maybe_catch_panic\n at /checkout/src/libpanic_unwind/lib.rs:98\n 7: 0x4209a2 - <F as alloc::boxed::FnBox<A>>::call_box::h032bafb4b576d1cd\n 8: 0x5dc00b - alloc::boxed::{{impl}}::call_once<(),()>\n 

尽管您看到的错误的错误代码是 'status': 500 而我提供的错误示例是 404 Not Found,表面看起来不一样,核心原因类似:

"message":"connection refused" 

由于:

imp::thread::{{impl}}::new::thread_start

来自:

/checkout/src/libstd/sys/unix/thread.rs:84

从另一个角度来看,当您使用 GeckoDriverSeleniumFirefox 时,请确保二进制文件兼容如下:


分析

自从 geckodriver 0.19.1 可用以来,geckodriver 二进制文件发生了重大变化。一些变化是:

  • GeckoDriver v0.22.0 (2018-09-15):
    • 用于 [脚本超时] 和 [超时] 错误的 HTTP 状态代码已从请求超时 (408) 更改为内部服务器错误 (500),以便不中断 HTTP/1.1 Keep-Alive 支持,因为 HTTP 客户端将旧状态代码解释为意味着它们应该复制请求。
    • 持久连接的 HTTP/1.1 Keep-Alive 超时已增加到 90 秒。
    • 当没有活动会话时,return现在会出现 [无效会话 ID] 错误。
    • geckodriver 连接到 Marionette 时的握手已通过在失败时终止 Firefox 进程而得到强化。
    • 握手读取超时已减少到 10 秒,而不是永远等待。
  • GeckoDriver v0.21.0 (2018-06-15):
    • 没有 return 值的 WebDriver 命令现在可以正确 return {value: null} 而不是空字典。
    • 强制使用 IPv4 网络堆栈。
    • On certain system configurations, where localhost resolves to an IPv6 address, geckodriver would attempt to connect to Firefox on the wrong IP stack, causing the connection attempt to time out after 60 seconds. We now ensure that geckodriver uses IPv4 consistently to both connect to Firefox and for allocating a free port.

  • GeckoDriver v0.20.1 (2018-04-06):
    • 避免尝试终止已停止的 Firefox 进程。
    • With the change to allow Firefox enough time to shut down in 0.20.0, geckodriver started unconditionally killing the process to reap its exit status. This caused geckodriver to inaccurately report a successful Firefox shutdown as a failure.

  • GeckoDriver v0.20.0 (2018-03-08):
    • geckodriver 的回溯不再替代缺失的 Marionette 堆栈跟踪。
    • Firefox 进程现在有足够的时间关闭,从而有足够的时间让 Firefox 关闭挂起监视器启动。
    • Firefox has an integrated background monitor that observes long-running threads during shutdown. These threads will be killed after 63 seconds in the event of a hang. To allow Firefox to shut down these threads on its own, geckodriver has to wait that time and some additional seconds.


解决方案

  • Selenium 升级到当前水平 Version 3.14.0
  • GeckoDriver 升级到 GeckoDriver v0.22.0 级别。
  • Firefox 版本升级到 Firefox v62.0.2 级别。
  • 如果您的基础 Web 客户端 版本太旧,则通过 Revo Uninstaller 卸载它并安装最新的 GA 和发布版本的 Web 客户端.
  • 始终在 tearDown(){} 方法中调用 driver.quit() 以优雅地关闭和销毁 WebDriverWeb Client 实例.
  • 以非 root 用户身份执行 Test

更新

根据你的问题 update of-coarse 你可以引入一个循环进行多次试验来初始化 selenium webdriver 实例,如下所示:

  • 通过调用 taskkill 命令确保没有 geckodriver 的悬空实例(Windows 操作系统特定 ) 如下:

    os.system("taskkill /f /im geckodriver.exe /T") 
    
  • 通过调用kill()命令确保没有geckodriver的悬挂实例(跨平台 ) 如下:

    from selenium import webdriver
    import psutil
    from selenium.common.exceptions import WebDriverException
    
    for counter in range(5):
        try:
            webdriver = webdriver.Firefox(executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe') 
            print("WebDriver and WebBrowser initialized ...")
            break
        except WebDriverException:
            #Cross platform
            PROCNAME = "geckodriver"
            for proc in psutil.process_iter():
                # check whether the process name matches
                if proc.name() == PROCNAME:
                    proc.kill()        
            print("Retrying ...")
    print("Out of loop ...")