无法在 Python 上通过 XPath 定位元素

Question

我试图使用 XPath 从 Bloomberg 获取公司网站。我被卡住了，因为它总是 return 一个空列表。我做了几次测试，发现我无法在此网页上找到任何元素。这是我正在使用的代码。

import re 
import requests
from lxml import html

url = "https://www.bloomberg.com/profile/company/FWLT:US"
requests=requests.get(url)
tree = html.fromstring(requests.content)
website = tree.xpath('//*[@id="root"]/div/section/div[2]/section/div/section[7]/div/text()')
print(website)

我也尝试过 selenium，但最终遇到了同样的问题。有人可以帮我解决这个问题吗？

Answer 1

这将为您带来网站价值 -

website = tree.xpath('//h2[contains(text(), \'WEBSITE\')]/following-sibling::div')

请注意，我已经从 'WEBSITE'

中转义了单引号

Answer 2

使用 to print the text www.amecfw.com you can use either of the following :

使用xpath，遵循和get_attribute()：

print(driver.find_element_by_xpath("//h2[text()='WEBSITE']//following::div").get_attribute("innerHTML"))

使用 xpath、following-sibling 和 text属性：

print(driver.find_element_by_xpath("//h2[text()='WEBSITE']//following-sibling::div").text)

理想情况下，打印文本 www.amecfw.com you have to induce for the visibility_of_element_located() and you can use either of the following :

使用xpath，遵循和get_attribute()：

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//h2[text()='WEBSITE']//following::div"))).get_attribute("innerHTML"))

使用 xpath、following-sibling 和 text属性：

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//h2[text()='WEBSITE']//following-sibling::div"))).text)

注意：您必须添加以下导入：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

You can find a relevant discussion in

结尾

Link 到有用的文档：

get_attribute()方法Gets the given attribute or property of the element.
text属性returnsThe text of the element.
Difference between text and innerHTML using Selenium

无法在 Python 上通过 XPath 定位元素

Cannot locate elements by XPath on Python

python

selenium

xpath

python-requests

webdriverwait

结尾