如何从使用 graphql 的网站页面中抓取产品数据
how to scrape product data from website pages that uses graphql
我以前使用下面的代码在 https://www.walmart.com/ 上抓取单词搜索的搜索结果，例如书。他们目前已经更改了他们的请求和响应参数，此代码没有再次获得任何响应。
params = {
'query': 'book',
'cat_id': 0,
'ps': 24,
'offset': 0,
'prg': 'desktop',
'stores': re.search(r'store/(\d+)', url).group(1)
}
try:
data1 = requests.get(api_url, params=params).json()
except Exception as e:
print("Sleeping for 10 seconds", e)
time.sleep(10)
try:
data1 = requests.get(api_url, params=params).json()
except Exception as e:
print("sleeping for 60 seconds", e)
time.sleep(60)
try:
data1 = requests.get(api_url, params=params).json()
except Exception as e:
print("sleeping for 360 seconds")
time.sleep(360)
data1 = requests.get(api_url, params=params).json()
我想获取产品页面的 json 响应，例如此 url
中的产品
https://www.walmart.com/ip/SKIPPY-Natural-Creamy-Peanut-Butter-Spread-15-oz/37447671
我如何使用当前参数重写代码以获得 json 响应？
根据您的问题，要获得 json 响应，您可以按照我的工作解决方案作为示例。实际上，隐藏的 api 调用 json 响应是 here。有趣的是，请求方法是 post 但它同时发送查询字符串参数和请求 payload/formdata 和下一页我第一次遇到哪种类型的响应，我必须同时进行获得所需 json 响应的参数类型。我还根据 json 响应进行了分页，您可以根据 json 响应 maxpage 增加或减少它。
import requests
import json
data= {
"query":"query Browse( $query:String $page:Int $prg:Prg! $facet:String $sort:Sort $catId:String! $max_price:String $min_price:String $module_search:String $affinityOverride:AffinityOverride $ps:Int $ptss:String $beShelfId:String $fitmentFieldParams:JSON ={}$fitmentSearchParams:JSON ={}$rawFacet:String $seoPath:String $trsp:String $fetchMarquee:Boolean! $fetchSkyline:Boolean! $additionalQueryParams:JSON ={}){search( query:$query page:$page prg:$prg facet:$facet sort:$sort cat_id:$catId max_price:$max_price min_price:$min_price module_search:$module_search affinityOverride:$affinityOverride additionalQueryParams:$additionalQueryParams ps:$ps ptss:$ptss trsp:$trsp _be_shelf_id:$beShelfId ){query searchResult{...BrowseResultFragment}}contentLayout( channel:\"WWW\" pageType:\"BrowsePage\" tenant:\"WM_GLASS\" version:\"v1\" searchArgs:{query:$query cat_id:$catId _be_shelf_id:$beShelfId prg:$prg}){modules{...ModuleFragment configs{...on EnricherModuleConfigsV1{zoneV1}__typename...on _TempoWM_GLASSWWWSearchSortFilterModuleConfigs{facetsV1{...FacetFragment}}...on TempoWM_GLASSWWWPillsModuleConfigs{moduleSource pillsV2{...PillsModuleFragment}}...on TempoWM_GLASSWWWSearchFitmentModuleConfigs{fitments( fitmentSearchParams:$fitmentSearchParams fitmentFieldParams:$fitmentFieldParams ){...FitmentFragment sisFitmentResponse{...BrowseResultFragment}}}...on TempoWM_GLASSWWWStoreSelectionHeaderConfigs{fulfillmentMethodLabel storeDislayName}...on TempoWM_GLASSWWWBreadcrumbConfigs{_rawConfigs}...on TempoWM_GLASSWWWSponsoredProductCarouselConfigs{_rawConfigs}...PopularInModuleFragment...CopyBlockModuleFragment...BannerModuleFragment...HeroPOVModuleFragment...InlineSearchModuleFragment...MarqueeDisplayAdConfigsFragment @include(if:$fetchMarquee)...SkylineDisplayAdConfigsFragment @include(if:$fetchSkyline)...HorizontalChipModuleConfigsFragment}}...LayoutFragment pageMetadata{location{postalCode stateOrProvinceCode city storeId}pageContext}}seoBrowseMetaData( id:$catId facets:$rawFacet path:$seoPath facet_query_param:$facet _be_shelf_id:$beShelfId ){metaTitle metaDesc metaCanon h1}}fragment BrowseResultFragment on SearchInterface{title aggregatedCount...BreadCrumbFragment...DebugFragment...ItemStacksFragment...PageMetaDataFragment...PaginationFragment...RequestContextFragment...ErrorResponse modules{facetsV1{...FacetFragment}pills{...PillsModuleFragment}}}fragment ModuleFragment on TempoModule{name version type moduleId schedule{priority}matchedTrigger{zone}}fragment LayoutFragment on ContentLayout{layouts{id layout}}fragment BreadCrumbFragment on SearchInterface{breadCrumb{id name url}}fragment DebugFragment on SearchInterface{debug{sisUrl}}fragment ItemStacksFragment on SearchInterface{itemStacks{displayMessage meta{adsBeacon{adUuid moduleInfo max_ads}query stackId stackType title layoutEnum totalItemCount totalItemCountDisplay viewAllParams{query cat_id sort facet affinityOverride recall_set min_price max_price}}itemsV2{...ItemFragment...InGridMarqueeAdFragment}}}fragment ItemFragment on Product{__typename id usItemId fitmentLabel name checkStoreAvailabilityATC seeShippingEligibility brand type shortDescription imageInfo{...ProductImageInfoFragment}canonicalUrl externalInfo{url}category{path{name url}}badges{flags{...on BaseBadge{key text type id}}tags{...on BaseBadge{key text type}}}classType averageRating numberOfReviews esrb mediaRating salesUnitType sellerId sellerName hasSellerBadge availabilityStatusV2{display value}productLocation{displayValue aisle{zone aisle}}badge{type dynamicDisplayName}fulfillmentSpeed offerId preOrder{...PreorderFragment}priceInfo{...ProductPriceInfoFragment}variantCriteria{...VariantCriteriaFragment}fulfillmentBadge fulfillmentTitle fulfillmentType brand manufacturerName showAtc sponsoredProduct{spQs clickBeacon spTags}showOptions}fragment ProductImageInfoFragment on ProductImageInfo{thumbnailUrl}fragment ProductPriceInfoFragment on ProductPriceInfo{priceRange{minPrice maxPrice}currentPrice{...ProductPriceFragment}wasPrice{...ProductPriceFragment}unitPrice{...ProductPriceFragment}listPrice{...ProductPriceFragment}shipPrice{...ProductPriceFragment}subscriptionPrice{priceString subscriptionString}priceDisplayCodes{priceDisplayCondition finalCostByWeight}}fragment PreorderFragment on PreOrder{isPreOrder preOrderMessage preOrderStreetDateMessage}fragment ProductPriceFragment on ProductPrice{price priceString}fragment VariantCriteriaFragment on VariantCriterion{name type id isVariantTypeSwatch variantList{id images name rank swatchImageUrl availabilityStatus products selectedProduct{canonicalUrl usItemId}}}fragment InGridMarqueeAdFragment on MarqueePlaceholder{__typename type moduleLocation lazy}fragment PageMetaDataFragment on SearchInterface{pageMetadata{storeSelectionHeader{fulfillmentMethodLabel storeDislayName}title canonical description location{addressId}}}fragment PaginationFragment on SearchInterface{paginationV2{maxPage pageProperties}}fragment RequestContextFragment on SearchInterface{requestContext{vertical isFitmentFilterQueryApplied searchMatchType categories{id name}}}fragment ErrorResponse on SearchInterface{errorResponse{correlationId source errors{errorType statusCode statusMsg source}}}fragment PillsModuleFragment on PillsSearchInterface{title url image:imageV1{src alt}baseSeoURL}fragment BannerModuleFragment on TempoWM_GLASSWWWSearchBannerConfigs{moduleType viewConfig{title image imageAlt displayName description url urlAlt appStoreLink appStoreLinkAlt playStoreLink playStoreLinkAlt}}fragment PopularInModuleFragment on TempoWM_GLASSWWWPopularInBrowseConfigs{seoBrowseRelmData(id:$catId){relm{id name url}}}fragment CopyBlockModuleFragment on TempoWM_GLASSWWWCopyBlockConfigs{copyBlock(id:$catId){cwc}}fragment FacetFragment on Facet{name type layout min max selectedMin selectedMax unboundedMax stepSize values{id name description type itemCount isSelected baseSeoURL}}fragment FitmentFragment on Fitments{partTypeIDs result{status formId position quantityTitle extendedAttributes{...FitmentFieldFragment}labels{...LabelFragment}resultSubTitle}labels{...LabelFragment}savedVehicle{vehicleYear{...VehicleFieldFragment}vehicleMake{...VehicleFieldFragment}vehicleModel{...VehicleFieldFragment}additionalAttributes{...VehicleFieldFragment}}fitmentFields{...VehicleFieldFragment}fitmentForms{id fields{...FitmentFieldFragment}title labels{...LabelFragment}}}fragment LabelFragment on FitmentLabels{ctas{...FitmentLabelEntityFragment}messages{...FitmentLabelEntityFragment}links{...FitmentLabelEntityFragment}images{...FitmentLabelEntityFragment}}fragment FitmentLabelEntityFragment on FitmentLabelEntity{id label}fragment VehicleFieldFragment on FitmentVehicleField{id label value}fragment FitmentFieldFragment on FitmentField{id displayName value extended data{value label}dependsOn}fragment HeroPOVModuleFragment on TempoWM_GLASSWWWHeroPovConfigsV1{povCards{card{povStyle image{mobileImage{...TempoCommonImageFragment}desktopImage{...TempoCommonImageFragment}}heading{text textColor textSize}subheading{text textColor}detailsView{backgroundColor isTransparent}ctaButton{button{linkText clickThrough{value}}}logo{...TempoCommonImageFragment}links{link{linkText}}}}}fragment TempoCommonImageFragment on TempoCommonImage{src alt assetId uid clickThrough{value}}fragment InlineSearchModuleFragment on TempoWM_GLASSWWWInlineSearchConfigs{headingText placeholderText}fragment MarqueeDisplayAdConfigsFragment on TempoWM_GLASSWWWMarqueeDisplayAdConfigs{_rawConfigs ad{...DisplayAdFragment}}fragment DisplayAdFragment on Ad{...AdFragment adContent{type data{__typename...AdDataDisplayAdFragment}}}fragment AdFragment on Ad{status moduleType platform pageId pageType storeId stateCode zipCode pageContext moduleConfigs adsContext adRequestComposite}fragment AdDataDisplayAdFragment on AdData{...on DisplayAd{json status}}fragment SkylineDisplayAdConfigsFragment on TempoWM_GLASSWWWSkylineDisplayAdConfigs{_rawConfigs ad{...SkylineDisplayAdFragment}}fragment SkylineDisplayAdFragment on Ad{...SkylineAdFragment adContent{type data{__typename...SkylineAdDataDisplayAdFragment}}}fragment SkylineAdFragment on Ad{status moduleType platform pageId pageType storeId stateCode zipCode pageContext moduleConfigs adsContext adRequestComposite}fragment SkylineAdDataDisplayAdFragment on AdData{...on DisplayAd{json status}}fragment HorizontalChipModuleConfigsFragment on TempoWM_GLASSWWWHorizontalChipModuleConfigs{chipModuleSource:moduleSource chipModule{title url{linkText title clickThrough{type value}}}chipModuleWithImages{title url{linkText title clickThrough{type value}}image{alt clickThrough{type value}height src title width}}}",
"variables":{
"id":"",
"affinityOverride":"default",
"dealsId":"",
"query":"",
"page":1,
"prg":"desktop",
"catId":"3920",
"facet":"",
"sort":"best_seller",
"rawFacet":"",
"seoPath":"",
"ps":40,
"ptss":"",
"trsp":"",
"beShelfId":"",
"recall_set":"",
"module_search":"",
"min_price":"",
"max_price":"",
"storeSlotBooked":"",
"additionalQueryParams":None,
"fitmentFieldParams":None,
"fitmentSearchParams":{
"id":"",
"affinityOverride":"default",
"dealsId":"",
"query":"",
"page":1,
"prg":"desktop",
"catId":"3920",
"facet":"",
"sort":"best_seller",
"rawFacet":"",
"seoPath":"",
"ps":40,
"ptss":"",
"trsp":"",
"beShelfId":"",
"recall_set":"",
"module_search":"",
"min_price":"",
"max_price":"",
"storeSlotBooked":"",
"additionalQueryParams":None,
"cat_id":"3920",
"_be_shelf_id":""
},
"fetchMarquee":True,
"fetchSkyline":True,
"fetchSbaTop":False
}
}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36',
'content-type':'application/json',
'wm_mp': 'true',
'wm_page_url': 'https://www.walmart.com/browse/books/3920?sort=best_seller&affinityOverride=default',
'wm_qos.correlation_id': 'FWpup9KEKUrLFOY68gppqfprABL16K6qE76g',
'x-apollo-operation-name': 'Browse',
'x-enable-server-timing': '1',
'x-latency-trace': '1',
'x-o-ccm': 'server',
'x-o-correlation-id': 'FWpup9KEKUrLFOY68gppqfprABL16K6qE76g',
'x-o-gql-query': 'query Browse',
'x-o-market': 'us',
'x-o-platform': 'rweb',
'x-o-platform-version': 'main-176-e8acb5',
'x-o-segment': 'oaoh'
}
params= {
"affinityOverride": "default",
"page": "1",
"prg": "desktop",
"catId": "3920",
"sort": "best_seller",
"ps": "40",
"fetchMarquee": "true",
"fetchSkyline": "true",
"fetchSbaTop": "false"}
for i in range(1,25,1):
params['maxPage']=i
api_url='https://www.walmart.com/orchestra/home/graphql/browse'
resp = requests.post(api_url, data=json.dumps(data), headers=headers,params=params)
r=resp.json()
print(r)
# items = r['data']['search']['searchResult']['itemStacks'][0]['itemsV2']
# for item in items:
# price = item['priceInfo']['currentPrice']['price']
# print(price)
输出：
'https://i5.walmartimages.com/asr/deaaef8d-ca7f-4fc1-b556-1209c4c9000c.c6df15d971f1b9cddde27f80f17ae80b.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Sonic-The-Hedgehog-Coloring-Book-For-Kids-Girls-Adults-Toddlers-Kids-ages-2-8-Unofficial-25-high-quality-illustrations-Pages-8-5-x-11-Paperback-9781677024223/614029660?athbdg=L1600', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': [{'key': 'BESTSELLER', 'text': 'Best seller', 'type': 'LABEL', 'id': 'L1600'}], 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 'REGULAR', 'averageRating': 3.9, 'numberOfReviews': 8, 'esrb': None, 'mediaRating': None, 'salesUnitType': 'EACH', 'sellerId': 'F55CDC31AB754BB68FE0B39041159D63', 'sellerName': 'Walmart.com', 'hasSellerBadge': None, 'availabilityStatusV2': {'display': 'In stock',
'value': 'IN_STOCK'}, 'productLocation': None, 'badge': [{'type': 'bestSeller', 'dynamicDisplayName': None}], 'fulfillmentSpeed': None, 'offerId': '27E4BA43A8704A1DABF0B37693611D16', 'preOrder': {'isPreOrder': False, 'preOrderMessage': None, 'preOrderStreetDateMessage': None}, 'priceInfo': {'priceRange': None, 'currentPrice': {'price': 6.99,
'priceString': '.99'}, 'wasPrice': None, 'unitPrice': None, 'listPrice': None, 'shipPrice': None, 'subscriptionPrice': None, 'priceDisplayCodes': {'priceDisplayCondition': None, 'finalCostByWeight': None}}, 'variantCriteria': [], 'fulfillmentBadge': None,
'fulfillmentTitle': 'title_shipToHome_not_available', 'fulfillmentType': 'FC', 'manufacturerName': None, 'showAtc': True, 'sponsoredProduct': None, 'showOptions': False}, {'__typename': 'Product', 'id': '16FA2JT4ZT52', 'usItemId': '491355610', 'fitmentLabel': None, 'name': 'Bible Word Search Books: Word Search Bible Puzzle Book - Extra Large
Print: Bible Word Search Large Print Puzzles for Seniors and Adults - Beginners Edition (Large Print) (Paperback)', 'checkStoreAvailabilityATC': False, 'seeShippingEligibility': False, 'brand': None, 'type': 'REGULAR', 'shortDescription': '<li>Format:Paperback</li><li>Publication Date: 2019-11-22</li>', 'imageInfo': {'thumbnailUrl': 'https://i5.walmartimages.com/asr/46269778-a1bc-4a7a-aff2-75a825e35cf9.62e71231ecd42f6cda6d3701a3281b53.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Bible-Word-Search-Books-Puzzle-Book-Extra-Large-Print-Print-Puzzles-Seniors-Adults-Beginners-Edition-Large-Print-Paperback-9781710478792/491355610', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': None, 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 'REGULAR', 'averageRating': 4.7, 'numberOfReviews': 13, 'esrb': None, 'mediaRating': None, 'salesUnitType': 'EACH', 'sellerId': 'F55CDC31AB754BB68FE0B39041159D63', 'sellerName': 'Walmart.com', 'hasSellerBadge': None, 'availabilityStatusV2': {'display': 'In stock', 'value': 'IN_STOCK'}, 'productLocation': None, 'badge': None, 'fulfillmentSpeed': None, 'offerId': '0E3D37D69AE14435A4E83D3AE2789B7F', 'preOrder': {'isPreOrder': False, 'preOrderMessage': None, 'preOrderStreetDateMessage': None}, 'priceInfo': {'priceRange': None, 'currentPrice': {'price': 6.99, 'priceString': '.99'}, 'wasPrice': None, 'unitPrice': None, 'listPrice': None, 'shipPrice': None, 'subscriptionPrice': None, 'priceDisplayCodes': {'priceDisplayCondition': None, 'finalCostByWeight': None}}, 'variantCriteria': [], 'fulfillmentBadge': None, 'fulfillmentTitle': 'title_shipToHome_not_available', 'fulfillmentType': 'FC', 'manufacturerName': None, 'showAtc': True, 'sponsoredProduct': None, 'showOptions': False}, {'__typename': 'Product', 'id': '72BDRK2VT8QQ', 'usItemId': '599380007', 'fitmentLabel': None, 'name': 'Trace Letters and Numbers Workbook: Learn How to Write Alphabet Upper and Lower Case and Numbers (Series #2) (Paperback)', 'checkStoreAvailabilityATC': False, 'seeShippingEligibility': False, 'brand': None, 'type': 'REGULAR', 'shortDescription': '9781794540767', 'imageInfo': {'thumbnailUrl': 'https://i5.walmartimages.com/asr/535dff68-7946-4e20-899b-20d05015b05a_1.22a1f229111edd3725505c0db3fe1371.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Trace-Letters-and-Numbers-Workbook-Learn-How-to-Write-Alphabet-Upper-and-Lower-Case-and-Numbers-Series-2-Paperback/599380007?athbdg=L1600', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': [{'key': 'BESTSELLER', 'text': 'Best seller', 'type': 'LABEL', 'id': 'L1600'}], 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 'REGULAR', 'averageRating': 4.7, 'numberOfReviews': 37, 'esrb': None, 'mediaRating':
None, 'salesUnitType': 'EACH', 'sellerId': 'F55CDC31AB754BB68FE0B39041159D63', 'sellerName': 'Walmart.com', 'hasSellerBadge': None, 'availabilityStatusV2': {'display': 'In
stock', 'value': 'IN_STOCK'}, 'productLocation': None, 'badge': [{'type': 'bestSeller', 'dynamicDisplayName': None}], 'fulfillmentSpeed': None, 'offerId': 'B701DAA6361D4A97A599815F29FA450D', 'preOrder': {'isPreOrder': False, 'preOrderMessage': None, 'preOrderStreetDateMessage': None}, 'priceInfo': {'priceRange': None, 'currentPrice': {'price': 6.95, 'priceString': '.95'}, 'wasPrice': None, 'unitPrice': None, 'listPrice': None, 'shipPrice': None, 'subscriptionPrice': None, 'priceDisplayCodes': {'priceDisplayCondition': None, 'finalCostByWeight': None}}, 'variantCriteria': [], 'fulfillmentBadge': None, 'fulfillmentTitle': 'title_shipToHome_not_available', 'fulfillmentType': 'FC', 'manufacturerName': None, 'showAtc': True, 'sponsoredProduct': None, 'showOptions': False}, {'__typename': 'Product', 'id': '72DZILK2NY05', 'usItemId': '817841366', 'fitmentLabel': None, 'name': 'Toddler Coloring Book for Kids Age 1-3 : aby Activity Book Boys or Girls, Preschool coloring for Their Fun Early Learning of First Easy Number Shape and Color (Paperback)', 'checkStoreAvailabilityATC': False, 'seeShippingEligibility': False, 'brand': None, 'type': 'REGULAR', 'shortDescription': '<li>Format:Paperback</li><li>Publication Date: 2019-08-02</li>', 'imageInfo': {'thumbnailUrl': 'https://i5.walmartimages.com/asr/d2f8e8be-7fa1-4a25-a80c-e1741c6b2f6f.6392f3a67fccf0b396e1fb6ee2848b4b.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Toddler-Coloring-Book-Kids-Age-1-3-aby-Activity-Boys-Girls-Preschool-coloring-Their-Fun-Early-Learning-First-Easy-Number-Shape-Color-Paperback-9781086986501/817841366?athbdg=L1600', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': [{'key': 'BESTSELLER', 'text': 'Best seller', 'type': 'LABEL', 'id': 'L1600'}], 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 'REGULAR', 'averageRating': 5, 'numberOfReviews': 2, 'esrb': None, 'mediaRating': None, 'salesUnitType': 'EACH', 'sellerId': 'F55CDC31AB754BB68FE0B39041159D63', 'sellerName': 'Walmart.com', 'hasSellerBadge': None, 'availabilityStatusV2': {'display': 'In stock', 'value': 'IN_STOCK'},
'productLocation': None, 'badge': [{'type': 'bestSeller', 'dynamicDisplayName': None}], 'fulfillmentSpeed': None, 'offerId': 'D3D88022487D4BF19D540BED3742A75D', 'preOrder': {'isPreOrder': False, 'preOrderMessage': None, 'preOrderStreetDateMessage': None}, 'priceInfo': {'priceRange': None, 'currentPrice': {'price': 6.95, 'priceString': '.95'}, 'wasPrice': None, 'unitPrice': None, 'listPrice': None, 'shipPrice': None, 'subscriptionPrice': None, 'priceDisplayCodes': {'priceDisplayCondition': None, 'finalCostByWeight': None}}, 'variantCriteria': [], 'fulfillmentBadge': None, 'fulfillmentTitle': 'title_shipToHome_not_available', 'fulfillmentType': 'FC', 'manufacturerName': None, 'showAtc': True, 'sponsoredProduct': None, 'showOptions': False}, {'__typename': 'Product', 'id': '46CGMFA2PY1Y', 'usItemId': '56172624', 'fitmentLabel': None, 'name': 'Crystals for Beginners : The Guide to Get Started with the Healing Power of Crystals (Paperback)', 'checkStoreAvailabilityATC': False, 'seeShippingEligibility': False, 'brand': None, 'type': 'VARIANT', 'shortDescription': '<li>Format:Paperback</li><li>Publication
Date: 2017-10-17</li>', 'imageInfo': {'thumbnailUrl': 'https://i5.walmartimages.com/asr/d2954574-c30c-48af-8297-900867a2458e_1.03867d2efc65af18a6fdbff418a68afa.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Crystals-for-Beginners-The-Guide-to-Get-Started-with-the-Healing-Power-of-Crystals-Paperback-9781623159917/56172624?athbdg=L1600', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags':
[{'key': 'BESTSELLER', 'text': 'Best seller', 'type': 'LABEL', 'id': 'L1600'}], 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 'VARIANT', 'averageRating': 5, 'numberOfReviews': 13, 'esrb': None, 'mediaRating': None, 'salesUnitType': 'EACH', 'sellerId': 'F55CDC31AB754BB68FE0B39041159D63', 'sellerName': 'Walmart.com', 'hasSellerBadge': None, 'availabilityStatusV2': {'display': 'In stock', 'value': 'IN_STOCK'}, 'productLocation': None, 'badge': [{'type': 'bestSeller', 'dynamicDisplayName': None}], 'fulfillmentSpeed': None, 'offerId': '5F59F4B1DE6945728E7F2EC9A3005472', 'preOrder': {'isPreOrder': False, 'preOrderMessage': None, 'preOrderStreetDateMessage': None}, 'priceInfo': {'priceRange': None, 'currentPrice': {'price': 8.99, 'priceString': '.99'}, 'wasPrice': None, 'unitPrice': None, 'listPrice': {'price': 14.99, 'priceString': '.99'}, 'shipPrice': None, 'subscriptionPrice': None, 'priceDisplayCodes': {'priceDisplayCondition': None, 'finalCostByWeight': None}}, 'variantCriteria': [], 'fulfillmentBadge': None, 'fulfillmentTitle': 'title_shipToHome_not_available', 'fulfillmentType': 'FC', 'manufacturerName': None, 'showAtc': True, 'sponsoredProduct': None, 'showOptions': False}, {'__typename': 'Product', 'id': '7FF8DA7PEPDT', 'usItemId': '136868031', 'fitmentLabel': None, 'name': 'Hack Learning: Hacking School Discipline : 9 Ways to Create a Culture of Empathy and Responsibility Using Restorative Justice (Series #22) (Paperback)', 'checkStoreAvailabilityATC': False, 'seeShippingEligibility': False, 'brand': None, 'type': 'REGULAR', 'shortDescription': '<li>Format:Paperback</li><li>Publication Date: 2019-03-12</li>', 'imageInfo': {'thumbnailUrl': 'https://i5.walmartimages.com/asr/4c639aa7-2580-4782-84ce-33a428cae000.571d547af78d39ffc35bdd28f988023f.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Hack-Learning-Hacking-School-Discipline-9-Ways-Create-Culture-Empathy-Responsibility-Using-Restorative-Justice-Series-22-Paperback-9781948212137/136868031?athbdg=L1600', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': [{'key': 'BESTSELLER', 'text': 'Best seller', 'type': 'LABEL', 'id': 'L1600'}], 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType':
输出：提取的价格，例如：
22.99
1.22
6.99
9.95
5.95
9.81
5.99
13.17
4.52
6.99
4.99
7.99
6.79
5.99
6.5
6.95
6.99
5.99
4.99
5
4.99
11.93
5.99
4.99
6.99
6.99
6.95
6.95
8.99
14.81
5.13
7.29
3.95
5.99
5.5
5.99
16.88
6.99
6.99
1.99
22.99
1.22
6.99
9.95
5.95
9.81
5.99
13.17
4.52
6.99
4.99
7.99
6.7
虽然此问题具体涉及 Python，请考虑使用 Puppeteer or Playwright for browser automation. This allows you to initiate requests as the user from a high-level, and you can use page.on('response') 拦截 API 服务器响应。与 public UI 相比，这可以消除更可能发生变化的低级模式要求。您还可以解析用户看到的网页结果，从而避免低级数据抽象。
我以前使用下面的代码在 https://www.walmart.com/ 上抓取单词搜索的搜索结果，例如书。他们目前已经更改了他们的请求和响应参数，此代码没有再次获得任何响应。
params = {
'query': 'book',
'cat_id': 0,
'ps': 24,
'offset': 0,
'prg': 'desktop',
'stores': re.search(r'store/(\d+)', url).group(1)
}
try:
data1 = requests.get(api_url, params=params).json()
except Exception as e:
print("Sleeping for 10 seconds", e)
time.sleep(10)
try:
data1 = requests.get(api_url, params=params).json()
except Exception as e:
print("sleeping for 60 seconds", e)
time.sleep(60)
try:
data1 = requests.get(api_url, params=params).json()
except Exception as e:
print("sleeping for 360 seconds")
time.sleep(360)
data1 = requests.get(api_url, params=params).json()
我想获取产品页面的 json 响应，例如此 url
中的产品https://www.walmart.com/ip/SKIPPY-Natural-Creamy-Peanut-Butter-Spread-15-oz/37447671
我如何使用当前参数重写代码以获得 json 响应？
根据您的问题，要获得 json 响应，您可以按照我的工作解决方案作为示例。实际上，隐藏的 api 调用 json 响应是 here。有趣的是，请求方法是 post 但它同时发送查询字符串参数和请求 payload/formdata 和下一页我第一次遇到哪种类型的响应，我必须同时进行获得所需 json 响应的参数类型。我还根据 json 响应进行了分页，您可以根据 json 响应 maxpage 增加或减少它。
import requests
import json
data= {
"query":"query Browse( $query:String $page:Int $prg:Prg! $facet:String $sort:Sort $catId:String! $max_price:String $min_price:String $module_search:String $affinityOverride:AffinityOverride $ps:Int $ptss:String $beShelfId:String $fitmentFieldParams:JSON ={}$fitmentSearchParams:JSON ={}$rawFacet:String $seoPath:String $trsp:String $fetchMarquee:Boolean! $fetchSkyline:Boolean! $additionalQueryParams:JSON ={}){search( query:$query page:$page prg:$prg facet:$facet sort:$sort cat_id:$catId max_price:$max_price min_price:$min_price module_search:$module_search affinityOverride:$affinityOverride additionalQueryParams:$additionalQueryParams ps:$ps ptss:$ptss trsp:$trsp _be_shelf_id:$beShelfId ){query searchResult{...BrowseResultFragment}}contentLayout( channel:\"WWW\" pageType:\"BrowsePage\" tenant:\"WM_GLASS\" version:\"v1\" searchArgs:{query:$query cat_id:$catId _be_shelf_id:$beShelfId prg:$prg}){modules{...ModuleFragment configs{...on EnricherModuleConfigsV1{zoneV1}__typename...on _TempoWM_GLASSWWWSearchSortFilterModuleConfigs{facetsV1{...FacetFragment}}...on TempoWM_GLASSWWWPillsModuleConfigs{moduleSource pillsV2{...PillsModuleFragment}}...on TempoWM_GLASSWWWSearchFitmentModuleConfigs{fitments( fitmentSearchParams:$fitmentSearchParams fitmentFieldParams:$fitmentFieldParams ){...FitmentFragment sisFitmentResponse{...BrowseResultFragment}}}...on TempoWM_GLASSWWWStoreSelectionHeaderConfigs{fulfillmentMethodLabel storeDislayName}...on TempoWM_GLASSWWWBreadcrumbConfigs{_rawConfigs}...on TempoWM_GLASSWWWSponsoredProductCarouselConfigs{_rawConfigs}...PopularInModuleFragment...CopyBlockModuleFragment...BannerModuleFragment...HeroPOVModuleFragment...InlineSearchModuleFragment...MarqueeDisplayAdConfigsFragment @include(if:$fetchMarquee)...SkylineDisplayAdConfigsFragment @include(if:$fetchSkyline)...HorizontalChipModuleConfigsFragment}}...LayoutFragment pageMetadata{location{postalCode stateOrProvinceCode city storeId}pageContext}}seoBrowseMetaData( id:$catId facets:$rawFacet path:$seoPath facet_query_param:$facet _be_shelf_id:$beShelfId ){metaTitle metaDesc metaCanon h1}}fragment BrowseResultFragment on SearchInterface{title aggregatedCount...BreadCrumbFragment...DebugFragment...ItemStacksFragment...PageMetaDataFragment...PaginationFragment...RequestContextFragment...ErrorResponse modules{facetsV1{...FacetFragment}pills{...PillsModuleFragment}}}fragment ModuleFragment on TempoModule{name version type moduleId schedule{priority}matchedTrigger{zone}}fragment LayoutFragment on ContentLayout{layouts{id layout}}fragment BreadCrumbFragment on SearchInterface{breadCrumb{id name url}}fragment DebugFragment on SearchInterface{debug{sisUrl}}fragment ItemStacksFragment on SearchInterface{itemStacks{displayMessage meta{adsBeacon{adUuid moduleInfo max_ads}query stackId stackType title layoutEnum totalItemCount totalItemCountDisplay viewAllParams{query cat_id sort facet affinityOverride recall_set min_price max_price}}itemsV2{...ItemFragment...InGridMarqueeAdFragment}}}fragment ItemFragment on Product{__typename id usItemId fitmentLabel name checkStoreAvailabilityATC seeShippingEligibility brand type shortDescription imageInfo{...ProductImageInfoFragment}canonicalUrl externalInfo{url}category{path{name url}}badges{flags{...on BaseBadge{key text type id}}tags{...on BaseBadge{key text type}}}classType averageRating numberOfReviews esrb mediaRating salesUnitType sellerId sellerName hasSellerBadge availabilityStatusV2{display value}productLocation{displayValue aisle{zone aisle}}badge{type dynamicDisplayName}fulfillmentSpeed offerId preOrder{...PreorderFragment}priceInfo{...ProductPriceInfoFragment}variantCriteria{...VariantCriteriaFragment}fulfillmentBadge fulfillmentTitle fulfillmentType brand manufacturerName showAtc sponsoredProduct{spQs clickBeacon spTags}showOptions}fragment ProductImageInfoFragment on ProductImageInfo{thumbnailUrl}fragment ProductPriceInfoFragment on ProductPriceInfo{priceRange{minPrice maxPrice}currentPrice{...ProductPriceFragment}wasPrice{...ProductPriceFragment}unitPrice{...ProductPriceFragment}listPrice{...ProductPriceFragment}shipPrice{...ProductPriceFragment}subscriptionPrice{priceString subscriptionString}priceDisplayCodes{priceDisplayCondition finalCostByWeight}}fragment PreorderFragment on PreOrder{isPreOrder preOrderMessage preOrderStreetDateMessage}fragment ProductPriceFragment on ProductPrice{price priceString}fragment VariantCriteriaFragment on VariantCriterion{name type id isVariantTypeSwatch variantList{id images name rank swatchImageUrl availabilityStatus products selectedProduct{canonicalUrl usItemId}}}fragment InGridMarqueeAdFragment on MarqueePlaceholder{__typename type moduleLocation lazy}fragment PageMetaDataFragment on SearchInterface{pageMetadata{storeSelectionHeader{fulfillmentMethodLabel storeDislayName}title canonical description location{addressId}}}fragment PaginationFragment on SearchInterface{paginationV2{maxPage pageProperties}}fragment RequestContextFragment on SearchInterface{requestContext{vertical isFitmentFilterQueryApplied searchMatchType categories{id name}}}fragment ErrorResponse on SearchInterface{errorResponse{correlationId source errors{errorType statusCode statusMsg source}}}fragment PillsModuleFragment on PillsSearchInterface{title url image:imageV1{src alt}baseSeoURL}fragment BannerModuleFragment on TempoWM_GLASSWWWSearchBannerConfigs{moduleType viewConfig{title image imageAlt displayName description url urlAlt appStoreLink appStoreLinkAlt playStoreLink playStoreLinkAlt}}fragment PopularInModuleFragment on TempoWM_GLASSWWWPopularInBrowseConfigs{seoBrowseRelmData(id:$catId){relm{id name url}}}fragment CopyBlockModuleFragment on TempoWM_GLASSWWWCopyBlockConfigs{copyBlock(id:$catId){cwc}}fragment FacetFragment on Facet{name type layout min max selectedMin selectedMax unboundedMax stepSize values{id name description type itemCount isSelected baseSeoURL}}fragment FitmentFragment on Fitments{partTypeIDs result{status formId position quantityTitle extendedAttributes{...FitmentFieldFragment}labels{...LabelFragment}resultSubTitle}labels{...LabelFragment}savedVehicle{vehicleYear{...VehicleFieldFragment}vehicleMake{...VehicleFieldFragment}vehicleModel{...VehicleFieldFragment}additionalAttributes{...VehicleFieldFragment}}fitmentFields{...VehicleFieldFragment}fitmentForms{id fields{...FitmentFieldFragment}title labels{...LabelFragment}}}fragment LabelFragment on FitmentLabels{ctas{...FitmentLabelEntityFragment}messages{...FitmentLabelEntityFragment}links{...FitmentLabelEntityFragment}images{...FitmentLabelEntityFragment}}fragment FitmentLabelEntityFragment on FitmentLabelEntity{id label}fragment VehicleFieldFragment on FitmentVehicleField{id label value}fragment FitmentFieldFragment on FitmentField{id displayName value extended data{value label}dependsOn}fragment HeroPOVModuleFragment on TempoWM_GLASSWWWHeroPovConfigsV1{povCards{card{povStyle image{mobileImage{...TempoCommonImageFragment}desktopImage{...TempoCommonImageFragment}}heading{text textColor textSize}subheading{text textColor}detailsView{backgroundColor isTransparent}ctaButton{button{linkText clickThrough{value}}}logo{...TempoCommonImageFragment}links{link{linkText}}}}}fragment TempoCommonImageFragment on TempoCommonImage{src alt assetId uid clickThrough{value}}fragment InlineSearchModuleFragment on TempoWM_GLASSWWWInlineSearchConfigs{headingText placeholderText}fragment MarqueeDisplayAdConfigsFragment on TempoWM_GLASSWWWMarqueeDisplayAdConfigs{_rawConfigs ad{...DisplayAdFragment}}fragment DisplayAdFragment on Ad{...AdFragment adContent{type data{__typename...AdDataDisplayAdFragment}}}fragment AdFragment on Ad{status moduleType platform pageId pageType storeId stateCode zipCode pageContext moduleConfigs adsContext adRequestComposite}fragment AdDataDisplayAdFragment on AdData{...on DisplayAd{json status}}fragment SkylineDisplayAdConfigsFragment on TempoWM_GLASSWWWSkylineDisplayAdConfigs{_rawConfigs ad{...SkylineDisplayAdFragment}}fragment SkylineDisplayAdFragment on Ad{...SkylineAdFragment adContent{type data{__typename...SkylineAdDataDisplayAdFragment}}}fragment SkylineAdFragment on Ad{status moduleType platform pageId pageType storeId stateCode zipCode pageContext moduleConfigs adsContext adRequestComposite}fragment SkylineAdDataDisplayAdFragment on AdData{...on DisplayAd{json status}}fragment HorizontalChipModuleConfigsFragment on TempoWM_GLASSWWWHorizontalChipModuleConfigs{chipModuleSource:moduleSource chipModule{title url{linkText title clickThrough{type value}}}chipModuleWithImages{title url{linkText title clickThrough{type value}}image{alt clickThrough{type value}height src title width}}}",
"variables":{
"id":"",
"affinityOverride":"default",
"dealsId":"",
"query":"",
"page":1,
"prg":"desktop",
"catId":"3920",
"facet":"",
"sort":"best_seller",
"rawFacet":"",
"seoPath":"",
"ps":40,
"ptss":"",
"trsp":"",
"beShelfId":"",
"recall_set":"",
"module_search":"",
"min_price":"",
"max_price":"",
"storeSlotBooked":"",
"additionalQueryParams":None,
"fitmentFieldParams":None,
"fitmentSearchParams":{
"id":"",
"affinityOverride":"default",
"dealsId":"",
"query":"",
"page":1,
"prg":"desktop",
"catId":"3920",
"facet":"",
"sort":"best_seller",
"rawFacet":"",
"seoPath":"",
"ps":40,
"ptss":"",
"trsp":"",
"beShelfId":"",
"recall_set":"",
"module_search":"",
"min_price":"",
"max_price":"",
"storeSlotBooked":"",
"additionalQueryParams":None,
"cat_id":"3920",
"_be_shelf_id":""
},
"fetchMarquee":True,
"fetchSkyline":True,
"fetchSbaTop":False
}
}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36',
'content-type':'application/json',
'wm_mp': 'true',
'wm_page_url': 'https://www.walmart.com/browse/books/3920?sort=best_seller&affinityOverride=default',
'wm_qos.correlation_id': 'FWpup9KEKUrLFOY68gppqfprABL16K6qE76g',
'x-apollo-operation-name': 'Browse',
'x-enable-server-timing': '1',
'x-latency-trace': '1',
'x-o-ccm': 'server',
'x-o-correlation-id': 'FWpup9KEKUrLFOY68gppqfprABL16K6qE76g',
'x-o-gql-query': 'query Browse',
'x-o-market': 'us',
'x-o-platform': 'rweb',
'x-o-platform-version': 'main-176-e8acb5',
'x-o-segment': 'oaoh'
}
params= {
"affinityOverride": "default",
"page": "1",
"prg": "desktop",
"catId": "3920",
"sort": "best_seller",
"ps": "40",
"fetchMarquee": "true",
"fetchSkyline": "true",
"fetchSbaTop": "false"}
for i in range(1,25,1):
params['maxPage']=i
api_url='https://www.walmart.com/orchestra/home/graphql/browse'
resp = requests.post(api_url, data=json.dumps(data), headers=headers,params=params)
r=resp.json()
print(r)
# items = r['data']['search']['searchResult']['itemStacks'][0]['itemsV2']
# for item in items:
# price = item['priceInfo']['currentPrice']['price']
# print(price)
输出：
'https://i5.walmartimages.com/asr/deaaef8d-ca7f-4fc1-b556-1209c4c9000c.c6df15d971f1b9cddde27f80f17ae80b.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Sonic-The-Hedgehog-Coloring-Book-For-Kids-Girls-Adults-Toddlers-Kids-ages-2-8-Unofficial-25-high-quality-illustrations-Pages-8-5-x-11-Paperback-9781677024223/614029660?athbdg=L1600', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': [{'key': 'BESTSELLER', 'text': 'Best seller', 'type': 'LABEL', 'id': 'L1600'}], 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 'REGULAR', 'averageRating': 3.9, 'numberOfReviews': 8, 'esrb': None, 'mediaRating': None, 'salesUnitType': 'EACH', 'sellerId': 'F55CDC31AB754BB68FE0B39041159D63', 'sellerName': 'Walmart.com', 'hasSellerBadge': None, 'availabilityStatusV2': {'display': 'In stock',
'value': 'IN_STOCK'}, 'productLocation': None, 'badge': [{'type': 'bestSeller', 'dynamicDisplayName': None}], 'fulfillmentSpeed': None, 'offerId': '27E4BA43A8704A1DABF0B37693611D16', 'preOrder': {'isPreOrder': False, 'preOrderMessage': None, 'preOrderStreetDateMessage': None}, 'priceInfo': {'priceRange': None, 'currentPrice': {'price': 6.99,
'priceString': '.99'}, 'wasPrice': None, 'unitPrice': None, 'listPrice': None, 'shipPrice': None, 'subscriptionPrice': None, 'priceDisplayCodes': {'priceDisplayCondition': None, 'finalCostByWeight': None}}, 'variantCriteria': [], 'fulfillmentBadge': None,
'fulfillmentTitle': 'title_shipToHome_not_available', 'fulfillmentType': 'FC', 'manufacturerName': None, 'showAtc': True, 'sponsoredProduct': None, 'showOptions': False}, {'__typename': 'Product', 'id': '16FA2JT4ZT52', 'usItemId': '491355610', 'fitmentLabel': None, 'name': 'Bible Word Search Books: Word Search Bible Puzzle Book - Extra Large
Print: Bible Word Search Large Print Puzzles for Seniors and Adults - Beginners Edition (Large Print) (Paperback)', 'checkStoreAvailabilityATC': False, 'seeShippingEligibility': False, 'brand': None, 'type': 'REGULAR', 'shortDescription': '<li>Format:Paperback</li><li>Publication Date: 2019-11-22</li>', 'imageInfo': {'thumbnailUrl': 'https://i5.walmartimages.com/asr/46269778-a1bc-4a7a-aff2-75a825e35cf9.62e71231ecd42f6cda6d3701a3281b53.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Bible-Word-Search-Books-Puzzle-Book-Extra-Large-Print-Print-Puzzles-Seniors-Adults-Beginners-Edition-Large-Print-Paperback-9781710478792/491355610', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': None, 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 'REGULAR', 'averageRating': 4.7, 'numberOfReviews': 13, 'esrb': None, 'mediaRating': None, 'salesUnitType': 'EACH', 'sellerId': 'F55CDC31AB754BB68FE0B39041159D63', 'sellerName': 'Walmart.com', 'hasSellerBadge': None, 'availabilityStatusV2': {'display': 'In stock', 'value': 'IN_STOCK'}, 'productLocation': None, 'badge': None, 'fulfillmentSpeed': None, 'offerId': '0E3D37D69AE14435A4E83D3AE2789B7F', 'preOrder': {'isPreOrder': False, 'preOrderMessage': None, 'preOrderStreetDateMessage': None}, 'priceInfo': {'priceRange': None, 'currentPrice': {'price': 6.99, 'priceString': '.99'}, 'wasPrice': None, 'unitPrice': None, 'listPrice': None, 'shipPrice': None, 'subscriptionPrice': None, 'priceDisplayCodes': {'priceDisplayCondition': None, 'finalCostByWeight': None}}, 'variantCriteria': [], 'fulfillmentBadge': None, 'fulfillmentTitle': 'title_shipToHome_not_available', 'fulfillmentType': 'FC', 'manufacturerName': None, 'showAtc': True, 'sponsoredProduct': None, 'showOptions': False}, {'__typename': 'Product', 'id': '72BDRK2VT8QQ', 'usItemId': '599380007', 'fitmentLabel': None, 'name': 'Trace Letters and Numbers Workbook: Learn How to Write Alphabet Upper and Lower Case and Numbers (Series #2) (Paperback)', 'checkStoreAvailabilityATC': False, 'seeShippingEligibility': False, 'brand': None, 'type': 'REGULAR', 'shortDescription': '9781794540767', 'imageInfo': {'thumbnailUrl': 'https://i5.walmartimages.com/asr/535dff68-7946-4e20-899b-20d05015b05a_1.22a1f229111edd3725505c0db3fe1371.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Trace-Letters-and-Numbers-Workbook-Learn-How-to-Write-Alphabet-Upper-and-Lower-Case-and-Numbers-Series-2-Paperback/599380007?athbdg=L1600', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': [{'key': 'BESTSELLER', 'text': 'Best seller', 'type': 'LABEL', 'id': 'L1600'}], 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 'REGULAR', 'averageRating': 4.7, 'numberOfReviews': 37, 'esrb': None, 'mediaRating':
None, 'salesUnitType': 'EACH', 'sellerId': 'F55CDC31AB754BB68FE0B39041159D63', 'sellerName': 'Walmart.com', 'hasSellerBadge': None, 'availabilityStatusV2': {'display': 'In
stock', 'value': 'IN_STOCK'}, 'productLocation': None, 'badge': [{'type': 'bestSeller', 'dynamicDisplayName': None}], 'fulfillmentSpeed': None, 'offerId': 'B701DAA6361D4A97A599815F29FA450D', 'preOrder': {'isPreOrder': False, 'preOrderMessage': None, 'preOrderStreetDateMessage': None}, 'priceInfo': {'priceRange': None, 'currentPrice': {'price': 6.95, 'priceString': '.95'}, 'wasPrice': None, 'unitPrice': None, 'listPrice': None, 'shipPrice': None, 'subscriptionPrice': None, 'priceDisplayCodes': {'priceDisplayCondition': None, 'finalCostByWeight': None}}, 'variantCriteria': [], 'fulfillmentBadge': None, 'fulfillmentTitle': 'title_shipToHome_not_available', 'fulfillmentType': 'FC', 'manufacturerName': None, 'showAtc': True, 'sponsoredProduct': None, 'showOptions': False}, {'__typename': 'Product', 'id': '72DZILK2NY05', 'usItemId': '817841366', 'fitmentLabel': None, 'name': 'Toddler Coloring Book for Kids Age 1-3 : aby Activity Book Boys or Girls, Preschool coloring for Their Fun Early Learning of First Easy Number Shape and Color (Paperback)', 'checkStoreAvailabilityATC': False, 'seeShippingEligibility': False, 'brand': None, 'type': 'REGULAR', 'shortDescription': '<li>Format:Paperback</li><li>Publication Date: 2019-08-02</li>', 'imageInfo': {'thumbnailUrl': 'https://i5.walmartimages.com/asr/d2f8e8be-7fa1-4a25-a80c-e1741c6b2f6f.6392f3a67fccf0b396e1fb6ee2848b4b.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Toddler-Coloring-Book-Kids-Age-1-3-aby-Activity-Boys-Girls-Preschool-coloring-Their-Fun-Early-Learning-First-Easy-Number-Shape-Color-Paperback-9781086986501/817841366?athbdg=L1600', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': [{'key': 'BESTSELLER', 'text': 'Best seller', 'type': 'LABEL', 'id': 'L1600'}], 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 'REGULAR', 'averageRating': 5, 'numberOfReviews': 2, 'esrb': None, 'mediaRating': None, 'salesUnitType': 'EACH', 'sellerId': 'F55CDC31AB754BB68FE0B39041159D63', 'sellerName': 'Walmart.com', 'hasSellerBadge': None, 'availabilityStatusV2': {'display': 'In stock', 'value': 'IN_STOCK'},
'productLocation': None, 'badge': [{'type': 'bestSeller', 'dynamicDisplayName': None}], 'fulfillmentSpeed': None, 'offerId': 'D3D88022487D4BF19D540BED3742A75D', 'preOrder': {'isPreOrder': False, 'preOrderMessage': None, 'preOrderStreetDateMessage': None}, 'priceInfo': {'priceRange': None, 'currentPrice': {'price': 6.95, 'priceString': '.95'}, 'wasPrice': None, 'unitPrice': None, 'listPrice': None, 'shipPrice': None, 'subscriptionPrice': None, 'priceDisplayCodes': {'priceDisplayCondition': None, 'finalCostByWeight': None}}, 'variantCriteria': [], 'fulfillmentBadge': None, 'fulfillmentTitle': 'title_shipToHome_not_available', 'fulfillmentType': 'FC', 'manufacturerName': None, 'showAtc': True, 'sponsoredProduct': None, 'showOptions': False}, {'__typename': 'Product', 'id': '46CGMFA2PY1Y', 'usItemId': '56172624', 'fitmentLabel': None, 'name': 'Crystals for Beginners : The Guide to Get Started with the Healing Power of Crystals (Paperback)', 'checkStoreAvailabilityATC': False, 'seeShippingEligibility': False, 'brand': None, 'type': 'VARIANT', 'shortDescription': '<li>Format:Paperback</li><li>Publication
Date: 2017-10-17</li>', 'imageInfo': {'thumbnailUrl': 'https://i5.walmartimages.com/asr/d2954574-c30c-48af-8297-900867a2458e_1.03867d2efc65af18a6fdbff418a68afa.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Crystals-for-Beginners-The-Guide-to-Get-Started-with-the-Healing-Power-of-Crystals-Paperback-9781623159917/56172624?athbdg=L1600', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags':
[{'key': 'BESTSELLER', 'text': 'Best seller', 'type': 'LABEL', 'id': 'L1600'}], 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 'VARIANT', 'averageRating': 5, 'numberOfReviews': 13, 'esrb': None, 'mediaRating': None, 'salesUnitType': 'EACH', 'sellerId': 'F55CDC31AB754BB68FE0B39041159D63', 'sellerName': 'Walmart.com', 'hasSellerBadge': None, 'availabilityStatusV2': {'display': 'In stock', 'value': 'IN_STOCK'}, 'productLocation': None, 'badge': [{'type': 'bestSeller', 'dynamicDisplayName': None}], 'fulfillmentSpeed': None, 'offerId': '5F59F4B1DE6945728E7F2EC9A3005472', 'preOrder': {'isPreOrder': False, 'preOrderMessage': None, 'preOrderStreetDateMessage': None}, 'priceInfo': {'priceRange': None, 'currentPrice': {'price': 8.99, 'priceString': '.99'}, 'wasPrice': None, 'unitPrice': None, 'listPrice': {'price': 14.99, 'priceString': '.99'}, 'shipPrice': None, 'subscriptionPrice': None, 'priceDisplayCodes': {'priceDisplayCondition': None, 'finalCostByWeight': None}}, 'variantCriteria': [], 'fulfillmentBadge': None, 'fulfillmentTitle': 'title_shipToHome_not_available', 'fulfillmentType': 'FC', 'manufacturerName': None, 'showAtc': True, 'sponsoredProduct': None, 'showOptions': False}, {'__typename': 'Product', 'id': '7FF8DA7PEPDT', 'usItemId': '136868031', 'fitmentLabel': None, 'name': 'Hack Learning: Hacking School Discipline : 9 Ways to Create a Culture of Empathy and Responsibility Using Restorative Justice (Series #22) (Paperback)', 'checkStoreAvailabilityATC': False, 'seeShippingEligibility': False, 'brand': None, 'type': 'REGULAR', 'shortDescription': '<li>Format:Paperback</li><li>Publication Date: 2019-03-12</li>', 'imageInfo': {'thumbnailUrl': 'https://i5.walmartimages.com/asr/4c639aa7-2580-4782-84ce-33a428cae000.571d547af78d39ffc35bdd28f988023f.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Hack-Learning-Hacking-School-Discipline-9-Ways-Create-Culture-Empathy-Responsibility-Using-Restorative-Justice-Series-22-Paperback-9781948212137/136868031?athbdg=L1600', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': [{'key': 'BESTSELLER', 'text': 'Best seller', 'type': 'LABEL', 'id': 'L1600'}], 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType':
输出：提取的价格，例如：
22.99
1.22
6.99
9.95
5.95
9.81
5.99
13.17
4.52
6.99
4.99
7.99
6.79
5.99
6.5
6.95
6.99
5.99
4.99
5
4.99
11.93
5.99
4.99
6.99
6.99
6.95
6.95
8.99
14.81
5.13
7.29
3.95
5.99
5.5
5.99
16.88
6.99
6.99
1.99
22.99
1.22
6.99
9.95
5.95
9.81
5.99
13.17
4.52
6.99
4.99
7.99
6.7
虽然此问题具体涉及 Python，请考虑使用 Puppeteer or Playwright for browser automation. This allows you to initiate requests as the user from a high-level, and you can use page.on('response') 拦截 API 服务器响应。与 public UI 相比，这可以消除更可能发生变化的低级模式要求。您还可以解析用户看到的网页结果，从而避免低级数据抽象。