如何将会话和 cookie 从 Selenium 浏览器加载到 Python 中的请求库？

Question

如何从 Selenium 浏览器加载会话和 cookie？以下代码：

import requests

cookies = [{u'domain': u'academics.vit.ac.in',
            u'name': u'ASPSESSIONIDAEQDTQRB',
            u'value': u'ADGIJGJDDGLFIIOCEZJHJCGC',
            u'expiry': None, u'path': u'/',
            u'secure': True}]
response = requests.get(url2, cookies=cookies)

给我以下异常：

Traceback (most recent call last):
  File "F:\PYTHON\python_scripts\cookies\cookies3.py", line 23, in <module>
    response = requests.get(url2, cookies=cookies)
  File "C:\Python27\lib\site-packages\requests\api.py", line 68, in get
    return request('get', url, **kwargs)<br/>
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 450, in request
    prep = self.prepare_request(req)
    cookies = cookiejar_from_dict(cookies)
  File "C:\Python27\lib\site-packages\requests\cookies.py", line 439, in cookiejar_from_dict
    cookiejar.set_cookie(create_cookie(name, cookie_dict[name]))
TypeError: list indices must be integers, not dict

Answer 1

首先，您必须从您的驱动程序实例中获取 cookie：

cookies = driver.get_cookies()

这 returns cookie dictionaries 适合您的课程。

接下来，在 requests 中设置这些 cookie：

s = requests.Session()
for cookie in cookies:
    s.cookies.set(cookie['name'], cookie['value'])

Answer 2

我做了这个解决方案，它解决了意外的关键字参数过期问题

def set_cookies(cookies, s):
    for cookie in cookies:
        if 'httpOnly' in cookie:
            httpO = cookie.pop('httpOnly')
            cookie['rest'] = {'httpOnly': httpO}
        if 'expiry' in cookie:
            cookie['expires'] = cookie.pop('expiry')
        s.cookies.set(**cookie)
    return s

Answer 3

您可以使用第 3 方包，例如 selenium-requests 或 requestium。它们提供在请求和 selenium 之间共享 cookie 的功能。

Answer 4

我认为这并不能直接回答问题，但如果您的目标是发出假装与 selenium 相同的会话的请求，您可以这样做：

resp = browser.execute_script(f'''
    var xmlHttp = new XMLHttpRequest();
    xmlHttp.open( "GET", "{yourUrl}", false );
    xmlHttp.send( null );
    return xmlHttp.responseText;
'''
)

如何将会话和 cookie 从 Selenium 浏览器加载到 Python 中的请求库？

How do I load session and cookies from Selenium browser to requests library in Python?

python

python-requests

selenium-webdriver