使用 selenium webdriver 时的超时问题

Question

您好，我正在尝试提取此网页的赔率：https://paris-sportifs.pmu.fr/pari/sport/25/football

这是我的 python 脚本：

#!/usr/bin/python3
# -*- coding: utf-8 -*-

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
import os

options = Options()
options.headless = True
options.add_argument("window-size=1400,800")
options.add_argument("--no-sandbox")
options.add_argument("--disable-gpu")
options.add_argument("start-maximized")
options.add_argument("enable-automation")
options.add_argument("--disable-infobars")
options.add_argument("--disable-dev-shm-usage")

driver = webdriver.Chrome(options=options)

driver.get('https://paris-sportifs.pmu.fr/pari/sport/25/football')

odds = [my_elem.text for my_elem in WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.XPATH, '//a[contains(@href, "#")]')))]

print(odds, '\n')

driver.close()
driver.quit()

输出结果是：

Traceback (most recent call last):
  File "./azerty.py", line 31, in <module>
    odds = [my_elem.text for my_elem in WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, '/-a[contains(@href, "#")]')))]
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/support/wait.py", line 80, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:

此脚本运行与其他网页完美搭配，但在本例中并非如此。一些帮助，谢谢

Answer 1

请测试此代码；请增加等待时间并使用EC.presence_of_all_elements_located.

from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
import os

options = Options()
options.headless = True
options.add_argument("window-size=1400,800")
options.add_argument("--no-sandbox")
options.add_argument("--disable-gpu")
options.add_argument("start-maximized")
options.add_argument("enable-automation")
options.add_argument("--disable-infobars")
options.add_argument("--disable-dev-shm-usage")

driver = webdriver.Chrome(options=options)

driver.get('https://paris-sportifs.pmu.fr/pari/sport/25/football')

odds = [my_elem.text for my_elem in WebDriverWait(driver, 50).until(EC.presence_of_all_elements_located((By.XPATH, '//a[contains(@href, "#")]')))]

print(odds, '\n')

使用 selenium webdriver 时的超时问题

Timeout problem when I use selenium webdriver

python

extract

selenium-webdriver