Python: AttributeError: 'bytes' object has no attribute 'find_all'

Question

刚刚一直在测试如何让网络抓取正常工作，但这让我很烦恼。这是我的代码。

import requests
from bs4 import BeautifulSoup
from csv import writer

page = requests.get('https://www.ebay.com/sch/i.html?_from=R40&_trksid=m570.l1313&_nkw=pc&_sacat=0')

soup = BeautifulSoup(page.text,'html.parser').encode("utf-8")
posts = soup.find_all(class_='s-item__wrapper clearfix')

with open('ebay.csv', 'w') as csv_file:
    csv_writer = writer(csv_file)
    headers = ['Title', 'Price', 'Link']
    csv_writer.writerow(headers)

    for post in posts:
        price = post.find(class_='s-item__price').get_text().replace('\n', '')
        title = post.find(class_='s-item__title').get_text().replace('\n', '')
        link = post.find('a')['href']
        csv_writer.writerow([title, price, link])

我一直收到这个错误

Traceback (most recent call last):
  File "path/WebScraping.py", line 8, in <module>
    posts = soup.find_all(class_='s-item__wrapper clearfix')
AttributeError: 'bytes' object has no attribute 'find_all'

试图找到其他解决方案，但找不到适合我的解决方案。该代码有效，但仅适用于页面的三分之一。

Answer 1

您正在尝试对 soup 对象进行编码，该对象会生成该对象的 bytes 对象表示，并且 bytes 对象中没有名为 find_all 的方法。

替换：

soup = BeautifulSoup(page.text,'html.parser').encode("utf-8")

搭配：

soup = BeautifulSoup(page.text,'html.parser')

Answer 2

import requests
from bs4 import BeautifulSoup
import csv



def Main(url):
    r = requests.get(url)
    soup = BeautifulSoup(r.content, 'html.parser')
    names = [item.text for item in soup.findAll(
        "h3", class_="s-item__title")]
    prices = [item.text for item in soup.findAll(
        "span", class_="s-item__price")]
    urls = [item.get("href")
            for item in soup.findAll("a", class_="s-item__link")]
    with open("result.csv", 'w', newline="", encoding="UTF-8") as f:
        wrriter = csv.writer(f)
        wrriter.writerow(["Name", "Price", "Url"])
        data = zip(names, prices, urls)
        wrriter.writerows(data)


Main("https://www.ebay.com/sch/i.html?_from=R40&_trksid=m570.l1313&_nkw=pc&_sacat=0")

输出：view-online

Note: You were using soup.find_all(class_='s-item__wrapper clearfix') instead of soup.find_all("div", class_='s-item__wrapper clearfix')

这就是您收到 AttributeError

的原因

关于 UnicodeEncodeError，我相信您使用的是 Windows，其中默认编码是 cp1252，这将无法读取 Unicode，例如 .

尝试：

print(\u2714)

输出应该是：✔ 但是你的系统会给出 UnicodeEncodeError.

所以您有 2 个选择，或者更改您的系统 locale 设置，

打开cmd.exe然后输入chcp，我相信你会得到437也就是United States.

现在您需要将其更改为 chcp 65001，即 UTF-8 支持。 Ref..

或者用 utf-8 和 encoding="UTF-8"

编码 csv

Python: AttributeError: 'bytes' object has no attribute 'find_all'

Python: AttributeError: 'bytes' object has no attribute 'find_all'

python

beautifulsoup

attributeerror