selenium + BeautifulSoup find_all 没有找到全部 :)？

Question

我正在尝试对比赛得分进行自动化处理。我正在使用硒和 BeautifulSoup。该脚本只让我获得前 10 场比赛，而不是其他比赛。我该如何解决？

from bs4 import BeautifulSoup
import lxml
from webdriver_manager.chrome import ChromeDriverManager
import time
import os
team_both = []
team_one = []
team_two = []

driver = webdriver.Chrome(ChromeDriverManager().install())
time.sleep(1)
os.system('clear')
time.sleep(1)
driver.get('https://www.bet365.de/#/IP/B1')

time.sleep(1)

driver.find_element_by_class_name('iip-IntroductoryPopup_Cross').click()
time.sleep(1)
driver.get('https://www.bet365.de/#/IP/B1')
time.sleep(1)

soup = BeautifulSoup(driver.page_source, 'lxml')

time.sleep(1)

tab_matches = soup.find_all('div', {"class" : "ovm-Fixture_Container"})

print(len(tab_matches))

Answer 1

问题是您没有滚动网页，所以所有元素都没有加载。我添加了将页面滚动到底部的功能，它对我有用。我还删除了 BeautifulSoup，因为当您已经在使用 Selenium 时再使用它是没有意义的。

from webdriver_manager.chrome import ChromeDriverManager
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains

team_both = []
team_one = []
team_two = []

driver = webdriver.Chrome(ChromeDriverManager().install())
time.sleep(1)
#os.system('clear')
time.sleep(1)
driver.get('https://www.bet365.de/#/IP/B1')

time.sleep(1)

driver.find_element_by_class_name('iip-IntroductoryPopup_Cross').click()
time.sleep(1)
driver.get('https://www.bet365.de/#/IP/B1')
time.sleep(1)

# Scroll to bottom
element = driver.find_element_by_class_name("ovm-OverviewScroller-enabled")
actions = ActionChains(driver)
actions.move_to_element(element).click_and_hold().perform()

time.sleep(5)

tab_matches = driver.find_elements(by=By.CLASS_NAME, value="ovm-Fixture_Container")

print(len(tab_matches))

selenium + BeautifulSoup find_all 没有找到全部 :)？

selenium + BeautifulSoup find_all don't find all :)?

python

selenium

beautifulsoup