从 table 抓取数据不会产生任何结果

Question

我正在尝试使脚本能够使用我在 Internet 上搜索到的内容从 table 中获取一行数据。此脚本不产生任何输出，也没有错误。任何帮助都可以。谢谢你。

import requests
from bs4 import BeautifulSoup

headers = {"User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:92.0) Gecko/20100101 Firefox/92.0"}
url = "https://bscscan.com/token/generic-tokentxns2?m=normal&contractAddress=0x0D0b63b32595957ae58D4dD60aa5409E79A5Aa96&a=0x81e0ef68e103ee65002d3cf766240ed1c070334d&sid=87aaaf5bf5aad80385b57f5f3fa2aa3a&p=1"

s = requests.Session()
r = s.get(url,headers=headers, timeout=5)
soupblockdetails = BeautifulSoup(r.content, 'html.parser')

for row in soupblockdetails.select("tr:has(td)")[:3]:
    item1 = row.find_all("td")[0].text[0:].strip()
    item2 = row.find_all("td")[1].text[0:].strip()
    item3 = row.find_all("td")[2].text[0:].strip()
    item4 = row.find_all("td")[1].a.get('href').split('a=')[-1]
    print ("{:<2} {:<43}   {:>25} {}".format(item1, item2, item3, item4))

当前输出：

no display

预期输出：

0xb07b68f72f0b58e8cfb8c8e896736f49b13775ebda25301475d24554a601ff97   115 days 12 hrs ago   Yooshiba Inu: Deployer   KIPS: Locked Wallet  1,870.82

Answer 1

会发生什么？

这里的主要问题是 table 嵌入在 iframe 中 src 是由 JavaScript

动态生成的

如何修复？

一种选择是使用 Selenium 导航到呈现的 iframe 并获取 table:

...
driver.get(url)
driver.switch_to.frame('tokentxnsiframe')

soupblockdetails = BeautifulSoup(driver.page_source, 'html.parser')

for row in soupblockdetails.select("tr:has(td)")[:3]:
    print(f"{' '.join(row.stripped_strings)}\t{row.a.get('href').split('/')[-1]}")
...

例子

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from bs4 import BeautifulSoup

url = 'https://bscscan.com/token/0x0D0b63b32595957ae58D4dD60aa5409E79A5Aa96?a=0x81e0ef68e103ee65002d3cf766240ed1c070334d'

driver = webdriver.Chrome(ChromeDriverManager().install())
driver.maximize_window()
driver.get(url)
driver.switch_to.frame('tokentxnsiframe')

soupblockdetails = BeautifulSoup(driver.page_source, 'html.parser')

for row in soupblockdetails.select("tr:has(td)")[:3]:
    print(f"{' '.join(row.stripped_strings)}\t{row.a.get('href').split('/')[-1]}")

输出

0xb07b68f72f0b58e8cfb8c8e896736f49b13775ebda25301475d24554a601ff97 0x129fa7e8 2021-12-09 17:23:58 115 days 14 hrs ago Yooshiba Inu: Deployer IN KIPS: Locked Wallet 1,870.828693386970691791    0xb07b68f72f0b58e8cfb8c8e896736f49b13775ebda25301475d24554a601ff97

从 table 抓取数据不会产生任何结果

Grabbing data from table does not produce any result

python

beautifulsoup

web-scraping

会发生什么？

如何修复？

例子

输出