如何将新闻网络碎片提取到 csv 文件中以及如何追加新记录？

Question

python 的新手并构建了一个网络抓取工具以从 cnn 头条新闻中提取新的新闻文章。尝试获取当我 print() 时看起来像一行一行的输出。希望将结果提取到 csv 文件中，以便每个标题都是它自己的行。然后还可以编写附加版本，以便每次我运行它时，它都会附加到文件而不是覆盖它。问题是如何使结果在 csv 文件中看起来像这样：

1) 抓取数据中的标题 1 2) 抓取数据中的标题 2 3) 抓取数据中的标题 3 等等。

我在下面粘贴了我的代码：

from bs4 import BeautifulSoup
import requests
import csv
#nterwebsite you wish to pull from that has news articles
res = requests.get('http://money.cnn.com/')
soup = BeautifulSoup(res.text, 'lxml')
#need to pul the ulcode from the website by right clicking and choosing inspecting element
news_box = soup.find('ul', {'class': '_6322dd28 ad271c3f'})
#drill down into the li's as they should always show a, which signals the header for the news article shown.
all_news = news_box.find_all('a')

for news in all_news:
  test=  (news.text)
  print(test)
with open('index.csv', 'w') as fobj:
    csvwriter = csv.writer(fobj, delimiter=',')
    for row in test:
        csvwriter.writerow(test)

Answer 1

您可以将 re.compile 与 BeautifulSoup.find_all 一起使用：

from bs4 import BeautifulSoup as soup
import requests, re
import csv
d = soup(requests.get('http://money.cnn.com/').text, 'html.parser')
articles = list(filter(None, [i.text for i in d.find_all('span', {'class':re.compile('^\w+ _\w+|^\w+$')})]))[2:]
with open('articles.csv', 'a') as f:
  write = csv.writer(f)
  write.writerows([[i] for i in articles])

输出：

What higher wages means for Domino's and McDonald's  
'Jurassic World' sequel has big opening day amid a surging box office 
Crying migrant girl: What the iconic photo says about press access 
Chanel reveals earnings for the first time in its 108-year history 
Why GE may need to stop paying its 119-year old dividend 
A top Netflix executive is out after using the N-word 
ZTE pays  billion fine to US over sanctions violations 
Tariffs on European cars would hurt US auto jobs 
Etsy sellers confront unknowns after Supreme Court ruling 
Chipotle hopes quesadillas and milkshakes bring customers back 
This group is getting ahead in America 
OPEC strikes deal to increase oil production 
Wall Street banks are healthier than ever 
Self-driving Uber driver may have been streaming 'The Voice' 
GM's new Chevy Blazer will be built in Mexico 
"GM is bringing back the Chevy Blazer, an SUV classic "
...

如何将新闻网络碎片提取到 csv 文件中以及如何追加新记录？

How to extract news web scrapings into csv file and how to append new records?

python

web-scraping

export-to-csv