我怎样才能使用 Google Colab 在 Python 中执行此 POST 请求。我目前收到 400 响应代码

Question

我正在尝试提取本网站上所有职位的信息： https://www.americanmobile.com/travel-nursing-jobs/search/

查看网络 activity 选项卡，我需要的所有数据似乎都来自此处发出的 POST 请求： https://jobs.amnhealthcare.com/api/jobs//search。我附上了一张图片，可以帮助确认我所引用的内容。 example_1

我在 Google Colab 中编写了以下代码，以尝试至少获得前 10 个结果。参考，我知道很多 headers 甚至可能都没有必要。我试过在没有任何 headers 的情况下发送此请求。

我想做的事情有可能吗？到目前为止，我只收到了 400 响应代码。

如果可以做到这一点，是否可以为所有 4k + 作业提取此信息？

import requests
import re
payload = {
  "PageNumber": "1",
    "JobsPerPage": "10",
    "Filters": {
        "Locations": "[]",
        "Skillset": "[]",
        "StartDates": "[]",
        "Shifts": "[]",
        "Durations": "[]",
        "IsCovid19Job": "false",
        "Exclusive": "false",
        "Skillsets": "[]",
        "DivisionCompanyId": "2",
        "LocationSearch": "",
        "KeywordSearch": "",
        "DaxtraJobIds": "[]"
    },
    "SortOrder": {
        "Header": "MaxPayRate",
        "SortDirection": "Descending"
    }
}

headers = {
    "Host": "jobs.amnhealthcare.com",
    "Connection": "keep-alive",
    "Content-Length": "315",
    "Accept": "application/json, text/plain, */*",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36",
    "Content-Type": "application/json;charset=UTF-8",
    "Origin": "https://www.americanmobile.com",
    "Sec-Fetch-Site": "cross-site",
    "Sec-Fetch-Mode": "cors",
    "Sec-Fetch-Dest": "empty",
    "Referer": "https://www.americanmobile.com/",
    "Accept-Encoding": "gzip, deflate, br",
    "Accept-Language": "en-US,en;q=0.9"
}
response = requests.post('https://jobs.amnhealthcare.com/api/jobs//search' , data=p , headers=headers)

谢谢

Answer 1

您 data 的格式不完全正确。这应该有效：

import requests

headers = {
    "Host": "jobs.amnhealthcare.com",
    "Connection": "keep-alive",
    "Content-Length": "315",
    "Accept": "application/json, text/plain, */*",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36",
    "Content-Type": "application/json;charset=UTF-8",
    "Origin": "https://www.americanmobile.com",
    "Sec-Fetch-Site": "cross-site",
    "Sec-Fetch-Mode": "cors",
    "Sec-Fetch-Dest": "empty",
    "Referer": "https://www.americanmobile.com/",
    "Accept-Encoding": "gzip, deflate, br",
    "Accept-Language": "en-US,en;q=0.9"
}

data = '{"PageNumber":1,"JobsPerPage":10,"Filters":{"Locations":[],"Skillset":[],"StartDates":[],"Shifts":[],"Durations":[],"IsCovid19Job":false,"Exclusive":false,"Skillsets":[],"DivisionCompanyId":2,"LocationSearch":"","KeywordSearch":"","DaxtraJobIds":[]},"SortOrder":{"Header":"MaxPayRate","SortDirection":"Descending"}}'

response = requests.post('https://jobs.amnhealthcare.com/api/jobs//search', headers=headers, data=data)

您现在可以调整 JobsPerPage 和 PageNumber 以在 for 循环中检索您需要的所有帖子。

我怎样才能使用 Google Colab 在 Python 中执行此 POST 请求。我目前收到 400 响应代码

How can I get this POST request to execute in Python using Google Colab. I am currently getting a 400 response code

python

google-colaboratory