从 html、python 中的特定部分提取文本

Question

我正在尝试制作一个显示歌曲歌词的程序，但我遇到了这个错误：

AttributeError: 'NoneType' object has no attribute 'text'

代码如下：

def get_lyrics(url):
    lyrics_html = requests.get(url)
    soup = BeautifulSoup(lyrics_html.content, "html.parser")
    lyrics = soup.find('div', {"class": "lyrics"})
    return lyrics.text

这是site我拿歌词的地方。我无法解释哪里出了问题，例如我将搜索 this song, so here's the lyrics of the song: click 的歌词。你自己可以看到，在页面中歌词所在的“地方”，一个div和class个“歌词”。这就是本网站所有歌词页面的制作方式。有人可以帮我吗？泰

Answer 1

你应该使用这个 link https://genius.com/Luis-sal-ciao-mi-chiamo-luis-lyrics 而不是您作为歌曲提到的 https://genius.com/。

def get_lyrics(url):
    lyrics_html = requests.get(url)
    soup = BeautifulSoup(lyrics_html.text, "lxml")
    lyrics_text = []
    lyrics = soup.find_all('div', class_="Lyrics__Container-sc-1ynbvzw-2 jgQsqn")
    for i in lyrics:
        lyrics_text.append(i.text.strip())
        # print(i.text.strip())
    return lyrics_text

output = get_lyrics("https://genius.com/Luis-sal-ciao-mi-chiamo-luis-lyrics")

输出将是：

['[Intro]Ah, mhh (ehi)Ho la bocca pienaVa bene[Verse]Ciao, mi chiamo Luis (eh, eh-eh)Ciao, mi chiamo Luis (eh, eh-eh)Ciao, Ciao mi chiamo Luis (eh, eh-eh)Ciao, mi chiamo LuisSi, si, si SalA a a a Si si si si si siProprio così mi chiamo ioCiao mi chiamo Luis AahLuis Sal, Luis, Luis, Luis SalCiao mi chiamo Luis, Luis SalEeemEeeCiao, Ciao BolognaMi chiamo LuisCiao Mamma (Eee) EeeCiao, Ciao anche a voi LuistiMi chiamo Luis, Lo youtuber EeeEeeCiao, Sono uno youtuberMi chiamo LuisSono uno youtuberEeeCiao, Sono uno youtuberMi chiamo LuisSono uno youtuberA e (Diglielo Luis) a e ă a e e a ă a a a-aaaaCiao mi chiamo LuisEee (Ma chi ti caga)Eee Ciao (Ma chi vuoi che ti guardi)Mi chiamo LuisHahahahaEeeVoglio diventare uno youtuberEee', '', '[Outro]Uuu BolognaDuemila EeeEee EeeEe']

Answer 2

页面 returns 有两个版本的页面（可能会混淆 scrapers 和机器人）。一个版本 class 从 "Lyrics__Container..." 开始，另一个版本 class lyrics。如果找不到 class Lyrics__Container 的标签，则歌词在 class lyrics.

的标签内

这应该总是打印歌词：

import requests
from bs4 import BeautifulSoup


url = 'https://genius.com/Luis-sal-ciao-mi-chiamo-luis-lyrics'
soup = BeautifulSoup(requests.get(url).content, 'html.parser')

text = soup.select_one('div[class^="Lyrics__Container"], .lyrics').get_text(strip=True, separator='\n')
print(text)

打印：

[Intro]
Ah, mhh (ehi)
Ho la bocca piena
Va bene
[Verse]
Ciao, mi chiamo Luis (eh, eh-eh)
Ciao, mi chiamo Luis (eh, eh-eh)
Ciao, Ciao mi chiamo Luis (eh, eh-eh)
Ciao, mi chiamo Luis
Si, si, si Sal
A a a a Si si si si si si
Proprio così mi chiamo io
Ciao mi chiamo Luis Aah

... and so on.

编辑：更新版本：

import requests
from bs4 import BeautifulSoup


url = 'https://genius.com/Avicii-the-nights-lyrics'
soup = BeautifulSoup(requests.get(url).content, 'lxml')

def get_text(elements):
    text = ''
    for c in elements:
        for t in c.select('a, span'):
            t.unwrap()
        if c:
            c.smooth()
            text += c.get_text(strip=True, separator='\n')
    return text


cs = soup.select('div[class^="Lyrics__Container"]')
if cs:
    text = get_text(cs)
else:
    text = get_text(soup.select('.lyrics'))

print(text)

打印：

[Verse 1]
(Hey)
Once upon a younger year
When all our shadows disappeared
The animals inside came out to play (Hey)
Hey, went face to face with all our fears
Learned our lessons through the tears
Made memories we knew would never fade
[Pre-Chorus]
One day my father he told me
Son, don't let it slip away

...etc.

从 html、python 中的特定部分提取文本

extract text from specific sections in html, python

beautifulsoup

python-3.7