Scrapy

Question

我刚刚开始使用 Scrapy。我想 从 this page 中抓取 所有职位并将其保存在 CSV 文件中。但是当我运行命令时：

scrapy crawl jobscraper -o file.csv

创建了一个空文件。我做错了什么？

import scrapy


class JobScraper(scrapy.Spider):
    name = "jobscraper"
    start_urls = [
        'https://www.pracuj.pl/praca/it%20-%20rozw%c3%b3j%20oprogramowania;cc,5016/%c5%82%c3%b3dzkie;r,5?rd=0',
    ]

    def parse(self, response):
        for offer in response.xpath('//div[contains(@class, "offer__info")]'):
            data = {
                'title': offer.xpath('//a[@class="offer-details__title-link]').extract_first()
            }
            yield data

Answer 1

offer.xpath('//a[@class="offer-details__title-link]')

不是有效的 XPath 表达式。你忘了最后的双引号：

offer.xpath('//a[@class="offer-details__title-link"]')

Scrapy - 找不到 xpath 元素

Scrapy - Can't locate xpath element

python

xpath

web-scraping