如何使用 python selenium 对具有相同 div 的 no.of 元素进行抓取

Question

我正在尝试抓取该网站中的所有匹配项

https://web.bet9ja.com/Sport/OddsToday.aspx?IDSport=590

我需要的条件是

1.click 我分享的 link 中的匹配名称，例如 kuttosh kujand link

中存在的匹配项

到目前为止，我用我的代码编写了这样的代码，我能够执行上面提到的匹配过程，但是我怎样才能完成所有匹配的过程

我写的代码：

# Here using selenium for scraping
# importing necessary modules
import selenium.webdriver
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
import mysql.connector
import pymysql
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# the relevant url
url = 'https://web.bet9ja.com/Sport/OddsToday.aspx?IDSport=590'

# the driver path
driver = webdriver.Chrome(r"c:/Users/SATYA/mysite/chromedriver")
driver.get(url)
driver.implicitly_wait(10) # seconds
buttons = WebDriverWait(driver,15).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.Event.ng-binding")))
for btn in buttons:
    btn.click()
    headings= [item.text for item in driver.find_elements_by_css_selector("div.SECQ.ng-binding")]
    keys = [item.text for item in driver.find_elements_by_css_selector("div.SEOdd.g1")]
    values = [item.text for item in driver.find_elements_by_css_selector("div.SEOddLnk.ng-binding")]
    driver.execute_script("window.history.go(-1)")
    print(headings,keys,values)

谁能帮我解决这个问题

在抓取第一个匹配数据后，我在编写的代码中遇到了这个错误

Traceback (most recent call last):
  File "dynamicscrape.py", line 21, in <module>
    btn.click()
  File "C:\Users\SATYA\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
    self._execute(Command.CLICK_ELEMENT)
  File "C:\Users\SATYA\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "C:\Users\SATYA\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Users\SATYA\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
  (Session info: chrome=81.0.4044.113)

Answer 1

stale element reference: element is not attached to the page document 当元素未附加到您刷新页面后已捕获的页面时出现错误。

要解决此问题，您需要再次重新分配元素以避免过时。

buttons = WebDriverWait(driver,15).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.Event.ng-binding")))
for btn in range(len(buttons)):
    #elements re-assigned again to avoid stale.
    buttons = WebDriverWait(driver, 15).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.Event.ng-binding")))
    buttons[btn].click()
    headings= [item.text for item in driver.find_elements_by_css_selector("div.SECQ.ng-binding")]
    keys = [item.text for item in driver.find_elements_by_css_selector("div.SEOdd.g1")]
    values = [item.text for item in driver.find_elements_by_css_selector("div.SEOddLnk.ng-binding")]
    driver.execute_script("window.history.go(-1)")
    print(headings,keys,values)

If 子句。

buttons = WebDriverWait(driver,15).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.Event.ng-binding")))
for btn in range(len(buttons)):
    buttons = WebDriverWait(driver, 15).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.Event.ng-binding")))
    if (btn==1) or (btn==3) or (btn==4):
        buttons[btn].click()
        headings= [item.text for item in driver.find_elements_by_css_selector("div.SECQ.ng-binding")]
        keys = [item.text for item in driver.find_elements_by_css_selector("div.SEOdd.g1")]
        values = [item.text for item in driver.find_elements_by_css_selector("div.SEOddLnk.ng-binding")]
        driver.execute_script("window.history.go(-1)")
        print(headings,keys,values)

如何使用 python selenium 对具有相同 div 的 no.of 元素进行抓取

how to do scraping for the no.of elements having the same div using python selenium

selenium

python-3.x

selenium-chromedriver

webdriverwait