为什么在 Selenium 中等待元素加载后元素仍未出现？

Question

我有一个特定的 URL 我想抓取，但是，某个元素在页面完全加载之前不会加载，我无法让 Selenium 找到它：

from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

driver = webdriver.Chrome(Insert Driver Path Here)
game_url = "https://rbsdm.com/stats/box_scores/?_inputs_&type=%22reg%22&away=%22ARI%22&home=%22SF%22&year=2020"
driver.get(game_url)
element = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.CLASS_NAME, "table table-striped table-hover table-responsive"))
    )
print(element)

具有我想要的 table 的 <div> 具有 ID qbs，它会加载，但是，显然，它在初始加载时不包含任何内容。 table 具有我在创建 element 变量时使用的 class 名称。

我给它的 10 秒 window 后肯定会加载页面数据，所以我很困惑这不起作用。

Answer 1

在您的代码中，当您搜索所需的 HTML 元素时：

element = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.CLASS_NAME, "table table-striped table-hover table-responsive"))
    )

您正在寻找 class 姓名，但 table table-striped table-hover table-responsive 不是 class 姓名，而是 CSS 选择器（用点 . 链接空格）：

table.table-striped.table-hover.table-responsive

所以，要么你使用上面的 CSS 选择器（注意 By.CSS_SELECTOR):

element = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.CSS_SELECTOR, "table.table-striped.table-hover.table-responsive"))
    )

或使用 class 名称：

element = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.CLASS_NAME, "table"))
    )

这是一个完整的工作示例：

from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait


driver = webdriver.Chrome()
game_url = "https://rbsdm.com/stats/box_scores/?_inputs_&type=%22reg%22&away=%22ARI%22&home=%22SF%22&year=2020"
driver.get(game_url)
element = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.CLASS_NAME, "table"))
    )
print(element.text)

输出：

Player Plays EPA/play Total EPA aDoT CMP% xCMP% CPOE xQBR
K.Murray 54 0.18 9.9 5.2 70.3 72.5 -2.2 63.6
J.Garoppolo 38 0.1 3.6 7 59.4 65.6 -6.2 54.9

为什么在 Selenium 中等待元素加载后元素仍未出现？

Why doesn't an element appear even after waiting for the element to load in Selenium?

python

selenium

google-chrome

beautifulsoup

web-scraping