如何在抓取中添加 url 之间的等待 python

Question

我想在抓取这些网址之间添加等待时间。我想每分钟抓取 2 个网址，所以 30 秒的等待就足够了，但不知道如何在网址之间添加等待时间。新手在此感谢帮助！

import cloudscraper
from bs4 import BeautifulSoup
scraper = cloudscraper.create_scraper()
urls = ["https://www.brandbucket.com/names?page=1","https://www.brandbucket.com/names?page=2","https://www.brandbucket.com/names?page=3","https://www.brandbucket.com/names?page=4","https://www.brandbucket.com/names?page=5"]
for url in urls:
    r = scraper.get(url)
    html = r.content
    soup = BeautifulSoup(html, 'html.parser')
    titles = soup.find_all("div", class_="domainCardDetail")
    for title in titles:
        print(title.text)

Answer 1

您可以使用time.sleep()

导入时间模块 import time

然后使用 time.sleep("number of seconds you want to wait")

如何在抓取中添加 url 之间的等待 python

How to add wait between urls in scraping python

python

beautifulsoup

web-scraping