我如何从网络上的 url 抓取数据？

Question

我想创建一个从 https://www.futbin.com/21/player/560/aubameyang located at the bottom of the page is the daily and hourly graph sections , the hourly graph is what i want which can be found in the network section of the inspect element which is called https://www.futbin.com/21/playerPrices?player=188567&rids=84074647&_=1608811830598 抓取数据的程序，这为我提供了使用 LCPrice 的最近销售历史的所有平台 (ps,xbox,pc) 的列表， LCPrice2 等...这就是我喜欢 scrape/extract.

在此示例中，每个播放器也由一个 ID 使用，此播放器的 ID 是 188567，通过网络选项卡找到，它给出了价格列表，我当前的代码是这样的：它不会 print/give 支持任何帮助，我们将不胜感激

import requests
from datetime import datetime

player_ids = {
  'Arturo Vidal': 181872,
  'Pierre-Emerick Aubameyang': 188567,
  'Robert Lewandowski': 188545,
  'Jerome Boateng': 183907,
  'Sergio Ramos': 155862,
  'Antoine Griezmann': 194765,
  'David Alaba': 197445,
  'Paulo Dybala': 211110,
  'Radja Nainggolan': 178518
}

for (name,id) in player_ids.items():
    r = requests.get('https://www.futbin.com/21/playerPrices?player={0}'.format(id))
    data = r.json()

    print(name)
    print("-"*20)
    #Change ps to xbox or pc to get other prices
    for price in data['ps']:
        price = price[1]
        print(price)

Answer 1

问题应该改进，但根据我的理解，您正在搜索类似以下示例的内容。

有什么区别

以正确的方式访问播放器和控制台的数据

data[str(id)]['prices']['ps'].values()

示例：

import requests
from datetime import datetime

player_ids = {
  'Arturo Vidal': 181872,
  'Pierre-Emerick Aubameyang': 188567,
  'Robert Lewandowski': 188545,
  'Jerome Boateng': 183907,
  'Sergio Ramos': 155862,
  'Antoine Griezmann': 194765,
  'David Alaba': 197445,
  'Paulo Dybala': 211110,
  'Radja Nainggolan': 178518
}

for (name,id) in player_ids.items():
    r = requests.get('https://www.futbin.com/21/playerPrices?player={0}'.format(id))
    data = r.json()

    print(name)
    print("-"*20)

    psPrices = list(data[str(id)]['prices']['ps'].values())
    print(psPrices)
    xboxPrices = list(data[str(id)]['prices']['xbox'].values())
    print(xboxPrices)

输出：

Arturo Vidal
--------------------
['0', '0', '0', '0', '0', '10 weeks ago', '3,600', '65,000', '0']
['0', '0', '0', '0', '0', '10 weeks ago', '2,100', '37,500', '100']
Pierre-Emerick Aubameyang
--------------------
['59,000', '59,000', '0', '0', '0', '13 mins ago', '12,250', '230,000', '21']
['57,000', '57,500', '58,000', '58,000', '58,000', '14 mins ago', '11,000', '210,000', '23']
Robert Lewandowski
--------------------
['72,500', '72,500', '72,500', '72,500', '72,500', '14 mins ago', '6,000', '110,000', '63']
['73,500', '73,500', '73,500', '73,500', '73,500', '2 mins ago', '7,400', '140,000', '49']
Jerome Boateng
--------------------
['1,400', '1,400', '1,400', '1,400', '1,400', '15 mins ago', '700', '10,000', '7']
['1,300', '1,300', '1,300', '1,300', '1,300', '4 mins ago', '700', '10,000', '6']
Sergio Ramos
--------------------
['50,000', '50,500', '50,500', '50,500', '50,500', '19 mins ago', '8,000', '150,000', '29']
['51,000', '51,000', '51,000', '51,000', '0', '15 mins ago', '7,200', '140,000', '32']
Antoine Griezmann
--------------------
['29,250', '29,250', '29,250', '29,250', '29,250', '35 mins ago', '2,800', '50,000', '56']
['32,750', '32,750', '33,000', '33,000', '33,000', '37 mins ago', '2,900', '55,000', '57']
David Alaba
--------------------
['0', '0', '0', '0', '0', '14 mins ago', '700', '10,000', '100']
['0', '0', '0', '0', '0', '16 mins ago', '700', '11,000', '100']
Paulo Dybala
--------------------
['36,000', '36,000', '36,000', '36,250', '36,500', '19 mins ago', '3,600', '65,000', '52']
['37,500', '37,500', '37,500', '38,000', '38,000', '1 min ago', '3,100', '55,000', '66']
Radja Nainggolan
--------------------
['2,100', '2,100', '2,100', '2,100', '2,100', '21 mins ago', '700', '10,000', '15']
['1,900', '1,900', '1,900', '1,900', '1,900', '32 mins ago', '700', '10,000', '12']

我如何从网络上的 url 抓取数据？

how can i scrape data from a url on the network?

html

python

web-scraping

python-requests-html