Scrpay ImportError: cannot import name Request
Scrpay ImportError: cannot import name Request
我在这个论坛上看到过类似的问题,但这与那些不同
我有这样的商品 class
...
class NewCarItem(Item):
car_petrol_engine_type = Item()
car_petrol_engine_size = Item()
car_petrol_engine_max_power = Item()
car_petrol_engine_max_torque = Item()
car_petrol_engine_fuel_supply_system = Item()
car_diesel_engine_type = Item()
car_diesel_engine_size = Item()
car_diesel_engine_max_power = Item()
car_diesel_engine_max_torque = Item()
car_diesel_engine_fuel_supply_system = Item()
car_transmission_type = Item()
car_suspension_front = Item()
car_suspension_rear = Item()
car_dimension_overall_length_width_height = Item()
car_dimension_wheel = Item()
car_dimension_fuel_tank_capacity = Item()
car_dimension_turning_circle_radius = Item()
car_dimension_boot_space = Item()
car_dimension_tyre = Item()
car_tyre_is_tube_less = Item()
我的蜘蛛class像这样
from scrapy import Request
from scrapy.selector import HtmlXPathSelector
from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor
from scrapy.contrib.spiders import CrawlSpider, Rule
from car_planet.items import NewCarItem
from car_planet.lib.html_utils import *
class NewCarSpiderSpider(CrawlSpider):
name = 'new_car_spider'
allowed_domains = ['toyotabharat.com']
start_urls = ['http://www.toyotabharat.com/cars/new_cars/']
rules = (
Rule(SgmlLinkExtractor(allow=r'Items/'), callback='parse_item', follow=True),
)
def parse(self, response):
hxs = HtmlXPathSelector(response)
urls = hxs.select("//div[@id='sr-img']/a/@href").extract()
items = []
for url in urls:
formed_url = "http://www.toyotabharat.com"+get_matched_strings("^(.*[\/])",url)[0]+"spec_org.aspx"
yield Request(formed_url,callback=self.parse_level_one)
def parse_level_one(self,response):
hxs = HtmlXPathSelector(response)
meta_tags = hxs.select("//meta").extract()
item = NewCarItem()
item['url'] = response.url
return item
当我尝试 运行 蜘蛛时
爬行 new_car_spider
这个命令我得到这样的错误
from scrapy import Request
ImportError: cannot import name Request
看起来像是某种配置错误。可能的原因:
软件包未安装;
不在PYTHONPATH下,所以python就是找不到;
前一点的子集 - 您安装了多个 python 版本,并且此库不适用于当前版本(例如,您只安装了 python 3,并且尝试通过 python 导入 2)
等
您可以做什么来调试此问题 - 只需 import sys; print sys.path
并检查您尝试导入的库文件是否在该路径下可用。
您从错误的地方导入请求。它位于 scrapy.http。
像这样更改您的导入:
from scrapy.http import Request
我在这个论坛上看到过类似的问题,但这与那些不同
我有这样的商品 class ...
class NewCarItem(Item):
car_petrol_engine_type = Item()
car_petrol_engine_size = Item()
car_petrol_engine_max_power = Item()
car_petrol_engine_max_torque = Item()
car_petrol_engine_fuel_supply_system = Item()
car_diesel_engine_type = Item()
car_diesel_engine_size = Item()
car_diesel_engine_max_power = Item()
car_diesel_engine_max_torque = Item()
car_diesel_engine_fuel_supply_system = Item()
car_transmission_type = Item()
car_suspension_front = Item()
car_suspension_rear = Item()
car_dimension_overall_length_width_height = Item()
car_dimension_wheel = Item()
car_dimension_fuel_tank_capacity = Item()
car_dimension_turning_circle_radius = Item()
car_dimension_boot_space = Item()
car_dimension_tyre = Item()
car_tyre_is_tube_less = Item()
我的蜘蛛class像这样
from scrapy import Request
from scrapy.selector import HtmlXPathSelector
from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor
from scrapy.contrib.spiders import CrawlSpider, Rule
from car_planet.items import NewCarItem
from car_planet.lib.html_utils import *
class NewCarSpiderSpider(CrawlSpider):
name = 'new_car_spider'
allowed_domains = ['toyotabharat.com']
start_urls = ['http://www.toyotabharat.com/cars/new_cars/']
rules = (
Rule(SgmlLinkExtractor(allow=r'Items/'), callback='parse_item', follow=True),
)
def parse(self, response):
hxs = HtmlXPathSelector(response)
urls = hxs.select("//div[@id='sr-img']/a/@href").extract()
items = []
for url in urls:
formed_url = "http://www.toyotabharat.com"+get_matched_strings("^(.*[\/])",url)[0]+"spec_org.aspx"
yield Request(formed_url,callback=self.parse_level_one)
def parse_level_one(self,response):
hxs = HtmlXPathSelector(response)
meta_tags = hxs.select("//meta").extract()
item = NewCarItem()
item['url'] = response.url
return item
当我尝试 运行 蜘蛛时
爬行 new_car_spider
这个命令我得到这样的错误
from scrapy import Request
ImportError: cannot import name Request
看起来像是某种配置错误。可能的原因:
软件包未安装;
不在PYTHONPATH下,所以python就是找不到;
前一点的子集 - 您安装了多个 python 版本,并且此库不适用于当前版本(例如,您只安装了 python 3,并且尝试通过 python 导入 2)
等
您可以做什么来调试此问题 - 只需 import sys; print sys.path
并检查您尝试导入的库文件是否在该路径下可用。
您从错误的地方导入请求。它位于 scrapy.http。 像这样更改您的导入:
from scrapy.http import Request