如何在延迟加载网页上单击 'Show More' 按钮

Question

我正在尝试单击网页上的 'Show More' 按钮。我写了这段代码，但在代码下方出现错误。

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait

#Open Chrome
driver=webdriver.Chrome(executable_path="C:/Users/chromedriver.exe")

web="https://www.expedia.com/Hotel-Search?adults=1&destination=Montreal%2C%20Quebec%2C%20Canada&endDate=2019-09-16&latLong=45.50195%2C-73.56714&localDateFormat=M%2Fd%2Fyyyy&regionId=178288&sort=recommended&startDate=2019-09-15&useRewards=true"
driver.get(web)
driver.maximize_window()

#parse html        
html =driver.page_source
soup=BeautifulSoup(html,"html.parser")
time.sleep(5)
WebDriverWait(driver, 5)

#click show more
show_more=driver.find_element_by_link_text('Show More')
#Another element is covering the element you are to click.    
driver.execute_script("arguments[0].click();", show_more)

错误：

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"link text","selector":"Show More"}

起初，我没有包括最后一行 (execute_script...)，但我认为如果我包括最后一行，但仍然是一样的。

如有任何帮助，我们将不胜感激。

另外一个问题是，有什么方法可以多次点击'Show More'按钮吗？因为我注意到我必须多次点击才能查找所有酒店列表。

Answer 1

我会简单地使用下面的 css。

button[data-stid='show-more-results']

脚本：

# scroll to the element before clicking
driver.find_element_by_css_selector("button[data-stid='show-more-results']").location_once_scrolled_into_view
# click on the show more button
driver.find_element_by_css_selector("button[data-stid='show-more-results']").click()

截图：

Answer 2

在 Viewport and to achieve that you can use the following :

中滚动后，您需要多次单击文本为 显示更多 的按钮

代码块：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("start-maximized")
chrome_options.add_experimental_option("excludeSwitches", ["enable-automation"])
chrome_options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=chrome_options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get("https://www.expedia.com/Hotel-Search?adults=1&destination=Montreal%2C%20Quebec%2C%20Canada&endDate=2019-09-16&latLong=45.50195%2C-73.56714&localDateFormat=M%2Fd%2Fyyyy&regionId=178288&sort=recommended&startDate=2019-09-15&useRewards=true")
while True:
    try:
        driver.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(driver,20).until(EC.visibility_of_element_located((By.XPATH, "//span[text()='Show More']"))))
        driver.execute_script("arguments[0].click();", WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH, "//span[text()='Show More']"))))
        print("Show More button clicked")
    except:
        print("No more Show More button")
        break
driver.quit()

控制台输出：

Show More button clicked
Show More button clicked
Show More button clicked
.
No more Show More button

Answer 3

尝试： driver.findElement(By.xpath("//span[包含(text(),'Show More')]")).click()

如何在延迟加载网页上单击 'Show More' 按钮

How to click on the 'Show More' button on a lazy loading webpage

javascript

python

selenium

lazy-loading

webdriverwait