Selenium 没有加载完整的页面

Question

我正在尝试抓取此网站：

https://www.lobstersnowboards.com/shop/eiki-pro-model--459

我想获取的字段之一是可用型号：

但这部分没有被 selenium 加载：

我试过 firefox 和 chrome，得到了相同的结果。

Answer 1

所以它不起作用的问题与Selenium无关。

这是因为本地化。正如您在顶部栏中看到的 REST OF THE WORLD 而您不在 United States 中应该显示美国价格

下面的脚本显示了它是如何工作的

from selenium import webdriver
driver = webdriver.Chrome()
driver.get("https://www.lobstersnowboards.com/shop/eiki-pro-model--459")
driver.find_element_by_css_selector("button[title='Rest of the World'] span.caret").click()
driver.find_element_by_xpath("//li[.='United States']").click()
import time
time.sleep(4)
driver.find_element_by_css_selector("div.options [title='Select'] span.caret").click()
elems = driver.find_elements_by_css_selector("div.options ul > li a span")
for elem in elems:
    print(elem.text.strip())

我使用了一个肮脏的快速睡眠，你不应该这样做，但这只是为了演示

下面是 chrome

中的实际页面

一些以后会影响你但不会影响你的事情 Chrome 65

On January 19, 2017, a public posting to the mozilla.dev.security.policy newsgroup drew attention to a series of questionable website authentication certificates issued by Symantec Corporation’s PKI. Symantec’s PKI business, which operates a series of Certificate Authorities under various brand names, including Thawte, VeriSign, Equifax, GeoTrust, and RapidSSL, had issued numerous certificates that did not comply with the industry-developed CA/Browser Forum Baseline Requirements. During the subsequent investigation, it was revealed that Symantec had entrusted several organizations with the ability to issue certificates without the appropriate or necessary oversight, and had been aware of security deficiencies at these organizations for some time.

发件人：https://security.googleblog.com/2017/09/chromes-plan-to-distrust-symantec.html

The SSL certificate used to load resources from https://www.lobstersnowboards.com will be distrusted in M66. Once distrusted, users will be prevented from loading these resources. See https://g.co/chrome/symantecpkicerts for more information.

这是 Chrome 65 中显示的错误。

Selenium 没有加载完整的页面

Selenium doesn't load the complete page

python

selenium

screen-scraping