如何使用 BeautifulSoup 访问“查看更多”选项卡的内容

Question

我正在尝试从 website

的“查看更多”部分抓取日期

我找到了 See More 执行器的特定 div：

div _ngcontent-flo-app-c111

但无法访问其下的任何内容。我在这里错过了什么？

Answer 1

您点击 See More 后看到的数据以 Json 的形式嵌入到页面中。要解析它，您可以使用下一个示例：

import json
import requests
from bs4 import BeautifulSoup


id_num = "7417030"  # ID is from URL
url = "https://tv.varsity.com/results/7417030-2022-spirit-sports-west-palm-beach-nationals-didii/31606"

soup = BeautifulSoup(requests.get(url).content, "html.parser")

x = soup.select_one("#flo-app-state").contents[0]
x = (
    x.replace("&q;", '"')
    .replace("&l;", "<")
    .replace("&g;", ">")
    .replace("&a;", "&")
)
x = json.loads(x)[
    f"G.https://tv-api.varsity.com/api/result-containers/{id_num}?"
]


soup = BeautifulSoup(
    x["body"]["data"]["event_node"]["description"], "html.parser"
)
print(soup.get_text(separator="\n"))

打印：

Welcome to the 2022 Spirit Sports West Palm Beach Nationals DI/DII event hub! Click 'Read More' below to find the very best coverage of the competition including a live stream, the order of competition, results, photos, articles, news, and more! 
Live Stream Information
Detailed coverage information will be posted as it becomes available.
Live Stream Availability  
Due to music rights, live streams are available in USA, UK, Canada and Puerto Rico only. However, all other content, including on-demand competition routine videos, original documentaries and articles are available everywhere.
Championship Schedule
Saturday, February 12th
8:00 AM - 7:47 PM
Sunday, February 13th
8:00 AM - 4:52 PM
*Schedule is subject to change.
Competition Routine Videos

...and so on.

如何使用 BeautifulSoup 访问“查看更多”选项卡的内容

How to access contents of See More tab with BeautifulSoup

html

beautifulsoup