我正在尝试使用 python3 为我的 wordpress 网站创建爬虫

Question

import requests
from bs4 import BeautifulSoup

def page(current_page):
    current = "h2"
    while current == current_page:
        url = 'https://vishrantkhanna.com/?s=' + str(current)
        source_code = requests.get(url)
        plain_text = source_code.txt
        soup = BeautifulSoup(plain_text)
        for link in soup.findAll('h2', {'class': 'entry-title'}):
            href = "https://vishrantkhanna.com/" + link.get('href')
            title = link.string
            print(href)
            print(title)

page("h2")

我正在尝试复制并打印文章标题和与之关联的 href link。

Answer 1

您需要从标题中提取 <a> 标签：

import requests
from bs4 import BeautifulSoup

URL = 'https://vishrantkhanna.com/?s=1'

html = requests.get(URL).text
bs = BeautifulSoup(html, 'html.parser')
for link in bs.find_all('h2', {'class': 'entry-title'}):
    a = link.find('a', href=True)
    href = "https://vishrantkhanna.com/" + a.get('href')
    title = link.string
    print(href)
    print(title)