如何 return 在 scrapy 循环中加载项目

Question

代码如下，每次returns只有第一个循环，最后9个循环都消失了。那么我应该怎么做才能得到所有循环？

我尝试添加 "m = []" 和 m.append(l) ，但出现错误 "ERROR: Spider must return Request, BaseItem, dict or None, got 'ItemLoader'"

link 是 http://ajax.lianjia.com/ajax/housesell/area/district?ids=23008619&limit_offset=0&limit_count=100&sort=&&city_id=110000

def parse(self, response):
    jsonresponse = json.loads(response.body_as_unicode())
    for i in range(0,len(jsonresponse['data']['list'])):
        l = ItemLoader(item = ItjuziItem(),response=response)
        house_code = jsonresponse['data']['list'][i]['house_code']
        price_total = jsonresponse['data']['list'][i]['price_total']
        ctime = jsonresponse['data']['list'][i]['ctime']
        title = jsonresponse['data']['list'][i]['title']
        frame_hall_num = jsonresponse['data']['list'][i]['frame_hall_num']
        tags = jsonresponse['data']['list'][i]['tags']
        house_area = jsonresponse['data']['list'][i]['house_area']
        community_id = jsonresponse['data']['list'][i]['community_id']
        community_name = jsonresponse['data']['list'][i]['community_name']
        is_two_five = jsonresponse['data']['list'][i]['is_two_five']
        frame_bedroom_num = jsonresponse['data']['list'][i]['frame_bedroom_num']
        l.add_value('house_code',house_code)
        l.add_value('price_total',price_total)
        l.add_value('ctime',ctime)
        l.add_value('title',title)
        l.add_value('frame_hall_num',frame_hall_num)
        l.add_value('tags',tags)
        l.add_value('house_area',house_area)
        l.add_value('community_id',community_id)
        l.add_value('community_name',community_name)
        l.add_value('is_two_five',is_two_five)
        l.add_value('frame_bedroom_num',frame_bedroom_num)
        print l
        return l.load_item()

Answer 1

错误：

ERROR: Spider must return Request, BaseItem, dict or None, got 'ItemLoader'

有点误导，因为您还可以 return 生成器！这里发生的是 return 打破了循环和整个功能。你可以把这个函数变成一个生成器来避免这种情况。

只需将最后一行的 return 替换为 yield 即可。

return l.load_item()

至：

yield l.load_item()

如何 return 在 scrapy 循环中加载项目

how to return item load in scrapy loop

python

ajax

json

web-crawler

scrapy