通过抓取从网页中提取单个 URL

Question

我一直在尝试抓取如下所示的网站。在页脚中有一堆 link 他们的社交媒体，其中 LinkedIn URL 是我关注的焦点。有没有办法只找出 link 可能使用正则表达式或 Python.

中可用的任何其他库

这是我目前尝试过的-

import requests
from bs4 import BeautifulSoup
url = "https://www.southcoast.org/"
req = requests.get(url)
soup = BeautifulSoup(reqs.text,"html.parser")
for link in soup.find_all('a'):
 print(link.get('href'))

但我正在获取所有 URL 而不是我正在寻找的那个。

注意：我希望能有一个动态代码，我也可以将其用于其他网站。

预先感谢您suggestion/help。

Answer 1

一种方法是使用 css selectors 并在 href 属性的值中查找字符串 linkedin.com/company/：

soup.select_one('a[href*="linkedin.com/company/"]')['href']

例子

import requests
from bs4 import BeautifulSoup
url = "https://www.southcoast.org/"
req = requests.get(url)
soup = BeautifulSoup(req.text,"html.parser")

# single (first) link
link = e['href'] if(e := soup.select_one('a[href*="linkedin.com/company/"]')) else None
# multiple links
links = [link['href'] for link in soup.select('a[href*="linkedin.com/company/"]')]

通过抓取从网页中提取单个 URL

Extract single URL from webpage by scraping

python

regex

beautifulsoup

web-scraping

python-3.x

例子