在 Python 中发送 POST 请求

Question

我正在尝试抓取一个网站，我需要在其中向表单发送 POST 请求以查询数据。这是我正在使用的代码。

import requests

headers = {'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36"}

with requests.Session() as s:
    
    r = s.get('https://data.rabbu.com', headers=headers)
    
    soup = BeautifulSoup(r.text, 'html.parser')
    hidden = soup.find_all("input", {'type':'hidden'})
    payload = {x["name"]: x["value"] for x in hidden}
    payload['search'] = '16101 Tampa Street, Brooksville FL 34604'
    payload['bedrooms'] = '2'
    
    r = s.post('https://data.rabbu.com/e', headers=headers, data=payload)
    soup = BeautifulSoup(r.text, 'html.parser')
    print(soup.text)

但是我无法正确发送 POST 请求，因为我收到以下错误消息：

"The change you wanted was rejected (422)"

我尝试使用“json”参数代替“数据”- 无济于事。

你知道如何绕过这个问题吗？任何帮助将不胜感激。

Answer 1

您的参数需要更改。尝试以下操作：

from bs4 import BeautifulSoup
import requests

headers = {'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36"}

with requests.Session() as s:
    r = s.get('https://data.rabbu.com', headers=headers)
    
    soup = BeautifulSoup(r.content, 'html.parser')
    hidden = soup.find_all("input", {'type':'hidden'})
    
    payload = {x["name"]: x["value"] for x in hidden}
    payload['estimate[address]'] = '16101 Tampa Street, Brooksville FL 34604'
    payload['estimate[bedrooms]'] = '2'
    r = s.post('https://data.rabbu.com/e', headers=headers, params=payload)
    soup = BeautifulSoup(r.content, 'html.parser')
    
    print(soup.title.text)

给你：

16101 Tampa St, Brooksville, FL 34604, USA | Revenue Projection: ,639/mo | 2 to 2bds | 13 comps | Rabbu

在 Python 中发送 POST 请求

Send POST request in Python

python

beautifulsoup

python-requests