ERROR: Spider error processing <GET - .parse callback not defined
ERROR: Spider error processing <GET - .parse callback not defined
我对 python 和 scrapy 非常陌生。我已尝试 运行ning 现有代码，但抛出了这些错误。我正在 运行在 windows 10 上安装最新版本的 scrapy，并使用 Visual Code Studio 运行 我的测试等等
终端调试
2020-12-07 20:52:52 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.newegg.com/robots.txt> (referer: None)
2020-12-07 20:52:52 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.newegg.com/asus-radeon-rx-6800-rx6800-16g/p/N82E16814126473> (referer: None)
2020-12-07 20:52:52 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.newegg.com/white-bluedio-ht-wireless/p/0TH-0357-00002?Item=9SIACD56WH3830> (referer: None)
Traceback (most recent call last):
File "C:\ProgramData\Miniconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "C:\ProgramData\Miniconda3\lib\site-packages\scrapy\spiders\__init__.py", line 90, in _parse
return self.parse(response, **kwargs)
File "C:\ProgramData\Miniconda3\lib\site-packages\scrapy\spiders\__init__.py", line 93, in parse
raise NotImplementedError(f'{self.__class__.__name__}.parse callback is not defined')
NotImplementedError: NeweggSpider.parse callback is not defined
egg_spider.py
class NeweggSpider(scrapy.Spider):
name = "newegg"
USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) " \
"Chrome/43.0.2357.130 Safari/537.36 "
allowed_domains = ['newegg.com']
start_urls = [
"https://www.newegg.com/white-bluedio-ht-wireless/p/0TH-0357-00002?Item=9SIACD56WH3830", ]
def parse(self, response):
# Finding Product Status.
my_list = []
try:
product = response.xpath("//*[@class='btn btn-primary btn-wide']//text()").get().strip()
my_list = [product]
print(f"Product is Currently: {my_list}")
except AttributeError:
pass
尽可能以最通俗易懂的方式提供帮助。谢谢！
您需要按照 gangabass 评论缩进您的代码。
import scrapy
class NeweggSpider(scrapy.Spider):
name = "newegg"
USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) " \
"Chrome/43.0.2357.130 Safari/537.36 "
allowed_domains = ['newegg.com']
start_urls = [
"https://www.newegg.com/white-bluedio-ht-wireless/p/0TH-0357-00002?Item=9SIACD56WH3830", ]
def parse(self, response):
# Finding Product Status.
my_list = []
try:
product = response.xpath("//*[@class='btn btn-primary btn-wide']//text()").get().strip()
my_list = [product]
print(f"Product is Currently: {my_list}")
except AttributeError:
pass
我对 python 和 scrapy 非常陌生。我已尝试 运行ning 现有代码，但抛出了这些错误。我正在 运行在 windows 10 上安装最新版本的 scrapy，并使用 Visual Code Studio 运行 我的测试等等
终端调试
2020-12-07 20:52:52 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.newegg.com/robots.txt> (referer: None)
2020-12-07 20:52:52 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.newegg.com/asus-radeon-rx-6800-rx6800-16g/p/N82E16814126473> (referer: None)
2020-12-07 20:52:52 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.newegg.com/white-bluedio-ht-wireless/p/0TH-0357-00002?Item=9SIACD56WH3830> (referer: None)
Traceback (most recent call last):
File "C:\ProgramData\Miniconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "C:\ProgramData\Miniconda3\lib\site-packages\scrapy\spiders\__init__.py", line 90, in _parse
return self.parse(response, **kwargs)
File "C:\ProgramData\Miniconda3\lib\site-packages\scrapy\spiders\__init__.py", line 93, in parse
raise NotImplementedError(f'{self.__class__.__name__}.parse callback is not defined')
NotImplementedError: NeweggSpider.parse callback is not defined
egg_spider.py
class NeweggSpider(scrapy.Spider):
name = "newegg"
USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) " \
"Chrome/43.0.2357.130 Safari/537.36 "
allowed_domains = ['newegg.com']
start_urls = [
"https://www.newegg.com/white-bluedio-ht-wireless/p/0TH-0357-00002?Item=9SIACD56WH3830", ]
def parse(self, response):
# Finding Product Status.
my_list = []
try:
product = response.xpath("//*[@class='btn btn-primary btn-wide']//text()").get().strip()
my_list = [product]
print(f"Product is Currently: {my_list}")
except AttributeError:
pass
尽可能以最通俗易懂的方式提供帮助。谢谢！
您需要按照 gangabass 评论缩进您的代码。
import scrapy
class NeweggSpider(scrapy.Spider):
name = "newegg"
USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) " \
"Chrome/43.0.2357.130 Safari/537.36 "
allowed_domains = ['newegg.com']
start_urls = [
"https://www.newegg.com/white-bluedio-ht-wireless/p/0TH-0357-00002?Item=9SIACD56WH3830", ]
def parse(self, response):
# Finding Product Status.
my_list = []
try:
product = response.xpath("//*[@class='btn btn-primary btn-wide']//text()").get().strip()
my_list = [product]
print(f"Product is Currently: {my_list}")
except AttributeError:
pass