- [SSL: CERTIFICATE_VERIFY_FAILED] 在 Linux 上处理 BeautifulSoup4

Question

问题很简单。我这里有这个小代码：

from bs4 import BeautifulSoup
import requests
from bs4 import BeautifulSoup

url = requests.get("https://www.docenti.unina.it/#!/professor/47494f434f4e44414d4f5343415249454c4c4f4d5343474e4435344c36354634383143/avvisi")
soup = BeautifulSoup(url.content, "html.parser")  # Requesting the source code with bs4

print(url)

但是如果我尝试运行这个我会得到这个错误（如果这有点长我很抱歉）：

Traceback (most recent call last):
  File "/home/simon/.local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
httplib_response = self._make_request(
  File "/home/simon/.local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 381, in _make_request
self._validate_conn(conn)
  File "/home/simon/.local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn
conn.connect()
  File "/home/simon/.local/lib/python3.8/site-packages/urllib3/connection.py", line 362, in connect
self.sock = ssl_wrap_socket(
  File "/home/simon/.local/lib/python3.8/site-packages/urllib3/util/ssl_.py", line 384, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3.8/ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
  File "/usr/lib/python3.8/ssl.py", line 1040, in _create
self.do_handshake()
  File "/usr/lib/python3.8/ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/simon/.local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
  File "/home/simon/.local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 726, in urlopen
retries = retries.increment(
  File "/home/simon/.local/lib/python3.8/site-packages/urllib3/util/retry.py", line 439, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.docenti.unina.it', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/simon/Linux_Storage/Projects/test_file/main.py", line 5, in <module>
url = requests.get("https://www.docenti.unina.it/#!/professor/47494f434f4e44414d4f5343415249454c4c4f4d5343474e4435344c36354634383143/avvisi")
  File "/home/simon/.local/lib/python3.8/site-packages/requests/api.py", line 76, in get
return request('get', url, params=params, **kwargs)
  File "/home/simon/.local/lib/python3.8/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
  File "/home/simon/.local/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
  File "/home/simon/.local/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
  File "/home/simon/.local/lib/python3.8/site-packages/requests/adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.docenti.unina.it', port=443): Max retries exceeded with url: /
 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)')))

真正的问题是它只发生在这个网站上。使用 google 或 youtube 或我尝试过的任何其他网站都可以正常工作并给我一个 <Response [200]>，但不是这个。我在某处读到这是 http 或类似内容的认证问题，但我不确定。我已经尝试在堆栈上查看许多“解决方案”，但我找不到适合我的真正解决方案。有什么想法吗？

Answer 1

为了解决这个错误你可以使用 verify=False 到 requests.get(url, verify=False).

例如：

from bs4 import BeautifulSoup
import requests

url = requests.get("https://www.docenti.unina.it/#!/professor/47494f434f4e44414d4f5343415249454c4c4f4d5343474e4435344c36354634383143/avvisi",verify=False)
soup = BeautifulSoup(url.content, "html.parser")  # Requesting the source code with bs4

print(url.status_code)

结果：

<Response [200]>

- [SSL: CERTIFICATE_VERIFY_FAILED] 在 Linux 上处理 BeautifulSoup4

- [SSL: CERTIFICATE_VERIFY_FAILED] while working on BeautifulSoup4 on Linux

python

beautifulsoup

ssl-certificate

web-scraping