从页面 https://www.stubhub.com/new-york-rangers-tickets/performer/2764/ 创建链接列表，其中包含文本 New York Rangers
Create a list of links from the page https://www.stubhub.com/new-york-rangers-tickets/performer/2764/ that contain the text New York Rangers
我正在尝试在 python 中创建一个列表，其中包含来自包含特定字符串的页面的所有链接。例如，我想要此页面 https://www.stubhub.com/new-york-rangers-tickets/performer/2764/ .
中包含 "New York Rangers @" 的所有链接
感谢您提供的所有帮助 - 如果这是一个愚蠢的问题但无法在任何地方找到它，我们深表歉意。
数据嵌入在 <srcipt> 标签内的页面中。您可以使用此示例来解析数据（使用 re 和 json 模块）：
import re
import json
import requests
headers = {'User-Agent':'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0'}
url = 'https://www.stubhub.com/new-york-rangers-tickets/performer/2764/'
txt = requests.get(url, headers=headers).text
data = json.loads(re.search(r'window.__INITIAL_STATE__\s*=\s*(.*})<', txt)[1])
# print(json.dumps(data, indent=4)) # <-- uncomment to see all data (prices, dates, etc.)
for event in data['EVENT_SEO_LIST']['events']:
if 'PARKING PASSES ONLY' in event['name']:
continue
print('{:<45} {}'.format(event['name'], 'https://www.stubhub.com/' + event['webURI']))
打印：
Detroit Red Wings at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-1-31-2020/event/104217508/
New York Rangers at Detroit Red Wings https://www.stubhub.com/detroit-red-wings-tickets-detroit-red-wings-detroit-little-caesars-arena-2-1-2020/event/104215245/
Dallas Stars at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-3-2020/event/104212773/
Toronto Maple Leafs at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-5-2020/event/104215469/
Buffalo Sabres at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-7-2020/event/104217518/
Los Angeles Kings at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-9-2020/event/104214839/
New York Rangers at Winnipeg Jets Tickets (Replica Hall of Fame Banner Giveaway) https://www.stubhub.com/winnipeg-jets-tickets-winnipeg-bell-mts-place-2-11-2020/event/104212882/
New York Rangers at Minnesota Wild https://www.stubhub.com/minnesota-wild-tickets-minnesota-wild-saint-paul-xcel-energy-center-2-13-2020/event/104216234/
New York Rangers at Columbus Blue Jackets https://www.stubhub.com/columbus-blue-jackets-tickets-columbus-blue-jackets-columbus-nationwide-arena-2-14-2020/event/104212942/
Boston Bruins at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-16-2020/event/104217520/
New York Rangers at Chicago Blackhawks https://www.stubhub.com/chicago-blackhawks-tickets-chicago-blackhawks-chicago-united-center-2-19-2020/event/104213910/
New York Rangers at Carolina Hurricanes https://www.stubhub.com/carolina-hurricanes-tickets-carolina-hurricanes-raleigh-pnc-arena-2-21-2020/event/104212812/
San Jose Sharks at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-22-2020/event/104217524/
New York Rangers at New York Islanders https://www.stubhub.com/new-york-islanders-tickets-new-york-islanders-uniondale-nycb-live-home-of-the-nassau-veterans-memorial-coliseum-2-25-2020/event/104354662/
New York Rangers at Montreal Canadiens https://www.stubhub.com/montreal-canadiens-tickets-montreal-bell-centre-2-27-2020/event/104215418/
New York Rangers at Philadelphia Flyers https://www.stubhub.com/philadelphia-flyers-tickets-philadelphia-flyers-philadelphia-wells-fargo-center-philadelphia-2-28-2020/event/104212712/
Philadelphia Flyers at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-1-2020/event/104215027/
St. Louis Blues at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-3-2020/event/104217528/
Washington Capitals at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-5-2020/event/104215030/
New Jersey Devils at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-7-2020/event/104215474/
New York Rangers at Dallas Stars https://www.stubhub.com/dallas-stars-tickets-dallas-stars-dallas-american-airlines-center-3-10-2020/event/104214902/
嗯，首先你需要得到你要搜索链接的网页内容。我强烈推荐使用 requests，一个用于 Python:
的简单 HTTP 库
import requests
response = request.get(https://www.stubhub.com/new-york-rangers-tickets/performer/2764/)
这个特定的 URL 出于某种原因需要一个 User-Agent header，所以你应该根据请求发送一个：
url = 'https://www.stubhub.com/new-york-rangers-tickets/performer/2764/'
user_agent = 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0'
response = requests.get(url, headers={'User-Agent':user_agent})
然后您可以使用beautifulsoup4开始分析页面内容。您可以使用方法 find_all 将已编译的正则表达式作为 text 参数传递来查找包含特定文本的所有 a 标签：
from bs4 import BeautifulSoup
import re
soup = BeautifulSoup(response.content, "html.parser")
rangers_anchor_tags = soup.find_all("a", text=re.compile(r".*\bNew York Rangers at\b.*")
urls = [anchor["href"] for anchor in rangers_anchor_tags]
urls，然后，将是 URL 的列表，锚标记的相应内部文本包含有问题的字符串。
使用 you won't need beautifulsoup and to create a list of all links i.e. href attributes from the page https://www.stubhub.com/new-york-rangers-tickets/performer/2764/ that contains the text New York Rangers you need to to induce WebDriverWait for the visibility_of_all_elements_located() and you can use the following ：
使用XPATH:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
# configuring the driver for optimum results
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get("https://www.stubhub.com/new-york-rangers-tickets/performer/2764/")
# just one line of code
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//a[./div[contains(., 'New York Rangers')]]")))])
控制台输出：
['https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-1-31-2020/event/104217508/', 'https://www.stubhub.com/detroit-red-wings-tickets-detroit-red-wings-detroit-little-caesars-arena-2-1-2020/event/104215245/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-3-2020/event/104212773/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-5-2020/event/104215469/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-7-2020/event/104217518/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-9-2020/event/104214839/', 'https://www.stubhub.com/winnipeg-jets-tickets-winnipeg-bell-mts-place-2-11-2020/event/104212882/', 'https://www.stubhub.com/minnesota-wild-tickets-minnesota-wild-saint-paul-xcel-energy-center-2-13-2020/event/104216234/', 'https://www.stubhub.com/columbus-blue-jackets-tickets-columbus-blue-jackets-columbus-nationwide-arena-2-14-2020/event/104212942/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-16-2020/event/104217520/', 'https://www.stubhub.com/chicago-blackhawks-tickets-chicago-blackhawks-chicago-united-center-2-19-2020/event/104213910/', 'https://www.stubhub.com/carolina-hurricanes-tickets-carolina-hurricanes-raleigh-pnc-arena-2-21-2020/event/104212812/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-22-2020/event/104217524/', 'https://www.stubhub.com/new-york-islanders-tickets-new-york-islanders-uniondale-nycb-live-home-of-the-nassau-veterans-memorial-coliseum-2-25-2020/event/104354662/', 'https://www.stubhub.com/montreal-canadiens-tickets-montreal-bell-centre-2-27-2020/event/104215418/', 'https://www.stubhub.com/philadelphia-flyers-tickets-philadelphia-flyers-philadelphia-wells-fargo-center-philadelphia-2-28-2020/event/104212712/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-1-2020/event/104215027/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-3-2020/event/104217528/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-5-2020/event/104215030/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-7-2020/event/104215474/']
我正在尝试在 python 中创建一个列表，其中包含来自包含特定字符串的页面的所有链接。例如，我想要此页面 https://www.stubhub.com/new-york-rangers-tickets/performer/2764/ .
中包含 "New York Rangers @" 的所有链接感谢您提供的所有帮助 - 如果这是一个愚蠢的问题但无法在任何地方找到它，我们深表歉意。
数据嵌入在 <srcipt> 标签内的页面中。您可以使用此示例来解析数据（使用 re 和 json 模块）：
import re
import json
import requests
headers = {'User-Agent':'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0'}
url = 'https://www.stubhub.com/new-york-rangers-tickets/performer/2764/'
txt = requests.get(url, headers=headers).text
data = json.loads(re.search(r'window.__INITIAL_STATE__\s*=\s*(.*})<', txt)[1])
# print(json.dumps(data, indent=4)) # <-- uncomment to see all data (prices, dates, etc.)
for event in data['EVENT_SEO_LIST']['events']:
if 'PARKING PASSES ONLY' in event['name']:
continue
print('{:<45} {}'.format(event['name'], 'https://www.stubhub.com/' + event['webURI']))
打印：
Detroit Red Wings at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-1-31-2020/event/104217508/
New York Rangers at Detroit Red Wings https://www.stubhub.com/detroit-red-wings-tickets-detroit-red-wings-detroit-little-caesars-arena-2-1-2020/event/104215245/
Dallas Stars at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-3-2020/event/104212773/
Toronto Maple Leafs at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-5-2020/event/104215469/
Buffalo Sabres at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-7-2020/event/104217518/
Los Angeles Kings at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-9-2020/event/104214839/
New York Rangers at Winnipeg Jets Tickets (Replica Hall of Fame Banner Giveaway) https://www.stubhub.com/winnipeg-jets-tickets-winnipeg-bell-mts-place-2-11-2020/event/104212882/
New York Rangers at Minnesota Wild https://www.stubhub.com/minnesota-wild-tickets-minnesota-wild-saint-paul-xcel-energy-center-2-13-2020/event/104216234/
New York Rangers at Columbus Blue Jackets https://www.stubhub.com/columbus-blue-jackets-tickets-columbus-blue-jackets-columbus-nationwide-arena-2-14-2020/event/104212942/
Boston Bruins at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-16-2020/event/104217520/
New York Rangers at Chicago Blackhawks https://www.stubhub.com/chicago-blackhawks-tickets-chicago-blackhawks-chicago-united-center-2-19-2020/event/104213910/
New York Rangers at Carolina Hurricanes https://www.stubhub.com/carolina-hurricanes-tickets-carolina-hurricanes-raleigh-pnc-arena-2-21-2020/event/104212812/
San Jose Sharks at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-22-2020/event/104217524/
New York Rangers at New York Islanders https://www.stubhub.com/new-york-islanders-tickets-new-york-islanders-uniondale-nycb-live-home-of-the-nassau-veterans-memorial-coliseum-2-25-2020/event/104354662/
New York Rangers at Montreal Canadiens https://www.stubhub.com/montreal-canadiens-tickets-montreal-bell-centre-2-27-2020/event/104215418/
New York Rangers at Philadelphia Flyers https://www.stubhub.com/philadelphia-flyers-tickets-philadelphia-flyers-philadelphia-wells-fargo-center-philadelphia-2-28-2020/event/104212712/
Philadelphia Flyers at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-1-2020/event/104215027/
St. Louis Blues at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-3-2020/event/104217528/
Washington Capitals at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-5-2020/event/104215030/
New Jersey Devils at New York Rangers https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-7-2020/event/104215474/
New York Rangers at Dallas Stars https://www.stubhub.com/dallas-stars-tickets-dallas-stars-dallas-american-airlines-center-3-10-2020/event/104214902/
嗯，首先你需要得到你要搜索链接的网页内容。我强烈推荐使用 requests，一个用于 Python:
的简单 HTTP 库import requests
response = request.get(https://www.stubhub.com/new-york-rangers-tickets/performer/2764/)
这个特定的 URL 出于某种原因需要一个 User-Agent header，所以你应该根据请求发送一个：
url = 'https://www.stubhub.com/new-york-rangers-tickets/performer/2764/'
user_agent = 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0'
response = requests.get(url, headers={'User-Agent':user_agent})
然后您可以使用beautifulsoup4开始分析页面内容。您可以使用方法 find_all 将已编译的正则表达式作为 text 参数传递来查找包含特定文本的所有 a 标签：
from bs4 import BeautifulSoup
import re
soup = BeautifulSoup(response.content, "html.parser")
rangers_anchor_tags = soup.find_all("a", text=re.compile(r".*\bNew York Rangers at\b.*")
urls = [anchor["href"] for anchor in rangers_anchor_tags]
urls，然后，将是 URL 的列表，锚标记的相应内部文本包含有问题的字符串。
使用href attributes from the page https://www.stubhub.com/new-york-rangers-tickets/performer/2764/ that contains the text New York Rangers you need to to induce WebDriverWait for the visibility_of_all_elements_located() and you can use the following
使用
XPATH:from selenium import webdriver from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.common.by import By from selenium.webdriver.support import expected_conditions as EC # configuring the driver for optimum results options = webdriver.ChromeOptions() options.add_argument("start-maximized") options.add_experimental_option("excludeSwitches", ["enable-automation"]) options.add_experimental_option('useAutomationExtension', False) driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe') driver.get("https://www.stubhub.com/new-york-rangers-tickets/performer/2764/") # just one line of code print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//a[./div[contains(., 'New York Rangers')]]")))])控制台输出：
['https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-1-31-2020/event/104217508/', 'https://www.stubhub.com/detroit-red-wings-tickets-detroit-red-wings-detroit-little-caesars-arena-2-1-2020/event/104215245/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-3-2020/event/104212773/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-5-2020/event/104215469/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-7-2020/event/104217518/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-9-2020/event/104214839/', 'https://www.stubhub.com/winnipeg-jets-tickets-winnipeg-bell-mts-place-2-11-2020/event/104212882/', 'https://www.stubhub.com/minnesota-wild-tickets-minnesota-wild-saint-paul-xcel-energy-center-2-13-2020/event/104216234/', 'https://www.stubhub.com/columbus-blue-jackets-tickets-columbus-blue-jackets-columbus-nationwide-arena-2-14-2020/event/104212942/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-16-2020/event/104217520/', 'https://www.stubhub.com/chicago-blackhawks-tickets-chicago-blackhawks-chicago-united-center-2-19-2020/event/104213910/', 'https://www.stubhub.com/carolina-hurricanes-tickets-carolina-hurricanes-raleigh-pnc-arena-2-21-2020/event/104212812/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-2-22-2020/event/104217524/', 'https://www.stubhub.com/new-york-islanders-tickets-new-york-islanders-uniondale-nycb-live-home-of-the-nassau-veterans-memorial-coliseum-2-25-2020/event/104354662/', 'https://www.stubhub.com/montreal-canadiens-tickets-montreal-bell-centre-2-27-2020/event/104215418/', 'https://www.stubhub.com/philadelphia-flyers-tickets-philadelphia-flyers-philadelphia-wells-fargo-center-philadelphia-2-28-2020/event/104212712/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-1-2020/event/104215027/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-3-2020/event/104217528/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-5-2020/event/104215030/', 'https://www.stubhub.com/new-york-rangers-tickets-new-york-rangers-new-york-madison-square-garden-3-7-2020/event/104215474/']