如何访问解析方法中的对象？

Question

我不知道如何访问 parse 函数中的对象。我想创建一个对象 Check，需要创建 Product 个对象。 products 属性是作为 urls.

来源的对象列表

class GenericScraper(scrapy.Spider):
    name = 'will_be_overriden'
    custom_settings = {'CONCURRENT_REQUESTS': 32,
                       'DOWNLOAD_DELAY':0.5}
    def __init__(self, occs):
        super(GenericScraper,self).__init__()
        self.name = products[0].site.name
        self.products = products
        self.xpath = self.product[0].site.xpaths.first().xpath

    def start_requests(self):
        for product in self.products:
            yield scrapy.Request(url=product.url, callback=self.parse)

    def parse(self, response):
        hxs = HtmlXPathSelector(response)
        text = hxs.select(self.xpath+'/text()').extract()
        Check.objects.create(text=text,product=product) # CAN'T ACCESS CURRENT PRODUCT

        responselog.debug(response)

可能吗？

Answer 1

使用 Request meta attribute 在回调之间进行通信。我假设您想将一个 product 对象关联到您发出的每个请求，例如：

def start_requests(self):
    for product in self.products:
        yield scrapy.Request(
            url=product.url, 
            callback=self.parse, 
            meta={'product': product},
        )

def parse(self, response):
    current_product = response.meta['product']
    ...

如何访问解析方法中的对象？

How to access object inside parse method?

python

web-crawler

scrapy

web-scraping

scrapy-spider