循环浏览搜索结果页面并解析数据

Question

我只是想让Python浏览一个网站的搜索结果的所有网页：

remainder = "latter_part_of_url"

page = '?s=0'

urlstring = 'https://domain/search/' + str(page) + str(remainder)

pagenumber = str(page)+120

for i in range(120, 100000, 120):
    # I am trying to increment by 120, how do I do that?

print(urlstring)

Answer 1

那岂不是很简单:

    for new_page_num in range(int(page)+120, 100000, 120):
        urlstring = 'https://domain/search/' + str(new_page_num) + str(remainder)

虽然我会使用一些字符串格式而不是 +。

循环浏览搜索结果页面并解析数据

Cycle through pages of search results and parse data

python

beautifulsoup