条件语句语法

Question

我现在正在写一个网络爬虫，我的 Python 已经生锈了，所以我只是想知道是否有更短的语法来完成以下...

def parse(self, response):
    prc_path = '//span[@class="result-meta"]/span[@class="result-price"]/text()'
    sqf_path = '//span[@class="result-meta"]/span[@class="housing"]/text()'
    loc_path = '//span[@class="result-meta"]/span[@class="result-hood"]/text()'
    prc_resp = response.xpath(prc_path).extract_first()
    sqf_resp = response.xpath(sqf_path).extract_first()
    loc_resp = response.xpath(loc_path).extract_first()
    if sqf_resp and loc_resp:
        yield {
            'prc': response.xpath(prc_path).extract_first(),
            'sqf': response.xpath(sqf_path).extract_first(),
            'loc': response.xpath(loc_path).extract_first()
        }
    elif sqf_resp:
        yield {
            'prc': response.xpath(prc_path).extract_first(),
            'sqf': response.xpath(sqf_path).extract_first()
        }
    else:
        yield {
            'prc': response.xpath(prc_path).extract_first(),
            'loc': response.xpath(loc_path).extract_first()
        }

如您所见，有相当多的重复，我希望尽可能保持干燥。

Answer 1

您可以创建字典，然后向其中添加适当的条目。

result = { 'prc': response.xpath(prc_path).extract_first() }
if sqf_path:
    result['sqf'] = response.xpath(sqf_path).extract_first()
if loc_path:
    result['loc'] = response.xpath(loc_path).extract_first()
yield result

您还可以通过字典理解分解出 extract_path 位。

result = { 'prc': prc_path, 'sqf': sqf_path, 'loc': loc_path }
yield { key : response.xpath(value).extract_first()
          for (key, value) in result.items() if value }

在 Python 的早期版本中，这将是：

result = { 'prc': prc_path, 'sqf': sqf_path, 'loc': loc_path }
yield dict((key, response.xpath(value).extract_first())
          for (key, value) in result.items() if value)

Answer 2

我会使用查找地图：

def parse(self, response):
    # initialize your prc_path/sqf_path/loc_path here
    lookup_map = {"prc": prc_path, "sqf": sqf_path, "loc": loc_path}  # add as many as needed
    return {k: response.xpath(v).extract_first() for k, v in lookup_map.items() if v}

条件语句语法

Conditional Statement Syntax

python

if-statement

yield

scrapy