在两个 BeautifulSoup 元素之间拉取文本

Question

在几个地方提出并回答了整个问题： http://www.resolvinghere.com/sof/18408799.shtml

How to get all text between just two specified tags using BeautifulSoup?

但在尝试实施时，我得到了非常繁琐的字符串。

我的设置：我正试图从总统辩论中提取抄本文本，我想我应该从这里开始：http://www.presidency.ucsb.edu/ws/index.php?pid=111500

我可以只用

隔离成绩单

transcript = soup.find_all("span", class_="displaytext")[0]

成绩单的格式不理想。每几行文本都有一个 <p>，它们表示说话者的变化，嵌套 <b>。例如：

<p><b>TRUMP:</b> First of all, I have to say, as a businessman, I get along with everybody. I have business all over the world. [<i>booing</i>]</p>,
<p>I know so many of the people in the audience. And by the way, I'm a self-funder. I don't have — I have my wife and I have my son. That's all I have. I don't have this. [<i>applause</i>]</p>,
<p>So let me just tell you, I get along with everybody, which is my obligation to my company, to myself, et cetera.</p>,
<p>Obviously, the war in Iraq was a big, fat mistake. All right? Now, you can take it any way you want, and it took — it took Jeb Bush, if you remember at the beginning of his announcement, when he announced for president, it took him five days.</p>,
<p>He went back, it was a mistake, it wasn't a mistake. It took him five days before his people told him what to say, and he ultimately said, "It was a mistake." The war in Iraq, we spent  trillion, thousands of lives, we don't even have it. Iran has taken over Iraq, with the second-largest oil reserves in the world.</p>,
<p>Obviously, it was a mistake.</p>,
<p><b>DICKERSON:</b> So...</p>

但正如我所说，这不是一个新问题。定义一个开始和结束标签，遍历元素，只要current != next，添加文本.

所以我在单个元素上进行测试以确保细节正确。

startTag = transcript.find_all('b')[165]
endTag = transcript.find_all('b')[166]
content = []
content += startTag.string
content

我得到的结果是 [u'R', u'U', u'B', u'I', u'O', u':'] 而不是 [u'RUBIO:']。

我错过了什么？

Answer 1

想法是找到抄本中的所有 b 元素，然后获取每个 b 元素的父元素并查找下一段，直到其中包含 b 元素.实施：

from bs4 import BeautifulSoup, Tag
import requests

url = "http://www.presidency.ucsb.edu/ws/index.php?pid=111500"
response = requests.get(url)

soup = BeautifulSoup(response.content, "html5lib")
transcript = soup.find("span", class_="displaytext")
for item in transcript.find_all("b")[3:]:  # skipping first irrelevant parts
    part = [" ".join(sibling.get_text(strip=True) if isinstance(sibling, Tag) else sibling.strip()
                     for sibling in item.next_siblings)]
    for paragraph in item.parent.find_next_siblings("p"):
        if paragraph.b:
            break

        part.append(paragraph.get_text(strip=True))

    print(item.get_text(strip=True))
    print("\n".join(part))
    print("-----")

打印：

DICKERSON:
Good evening. I'm John Dickerson. This holiday weekend, as America honors our first president, we're about to hear from six men who hope to be the 45th. The candidates for the Republican nomination are here in South Carolina for their ninth debate, one week before this state holds the first-in-the-South primary.
George Washington ...
-----
DICKERSON:
Before we get started, candidates, here are the rules. When we ask you a question, you will have one minute to answer, and 30 seconds more if we ask a follow-up. If you're attacked by another candidate, you get 30 seconds to respond.
...
-----
TRUMP:
Well, I can say this. If the president, and if I were president now, ...

在两个 BeautifulSoup 元素之间拉取文本

Pull text between two BeautifulSoup elements

python

beautifulsoup

html-parsing