Scrapy css 选择器找不到结果 - 但浏览器可以

Question

无法弄清楚为什么 scrapy 中的以下选择器失败：

>>> response.css('a[href="./viewforum.php?f=18"]::text')[0].get()
IndexError: list index out of range

但是使用 Chrome:

的同一页面同样适用

> $$('a[href="./viewforum.php?f=18"]')[0].innerText
'Contribute and help the VideoLAN project'

要复制，请使用以下网页：

scrapy shell "https://forum.videolan.org/viewtopic.php?f=18&t=29792"

Answer 1

In [1]: response.css('a[href*="./viewforum.php?f=18"]::text').get()                                                    
Out[1]: 'Contribute and help the VideoLAN project'

编辑：

a[href="./viewforum.php?f=18"]:
Selects every <a> element whose href attribute value is "./viewforum.php?f=18"

a[href*="./viewforum.php?f=18"]:
Selects every <a> element whose href attribute value contains the substring "./viewforum.php?f=18"

由于 href 值为“./viewforum.php?f=18&sid=be611e......0d35”，我们希望匹配不带 'sid' 部分的值选择器 '*='.

Scrapy css 选择器找不到结果 - 但浏览器可以

Scrapy css selector can't find result - but browser can

css-selectors

scrapy