Pandas 按问题分组

Question

使用 dataframe groupby() 函数有一个快速问题。我有一个 BS4 html table 被解析为 Pandas 数据框，然后目的是使用 Groupby() 嵌套 'Field' 的多行。不知道为什么我得到

'AttributeError: 'list' object has no attribute 'groupby''

错误。我的 str(table) 不正确吗？

谢谢。

import requests
from bs4 import BeautifulSoup
import json
from datetime import datetime
import pandas as pd


starttime = datetime.now()

#Agent detail to prevent scraping bot detection
user_agent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.47 Safari/537.36'
header = {'User-Agent' : user_agent }



# Webpage connection
html = "http://factpages.npd.no/ReportServer?/FactPages/TableView/field
_production_monthly&rs:Command=Render&rc:Toolbar=false&rc:Parameters
=f&Top100=True&IpAddress=108.171.128.174&CultureCode=en"

r=requests.get(html, headers=header)
c=r.content
soup=BeautifulSoup(c,"html.parser")

table = soup.find('table', attrs={'class':'a133'})


#Pandas dataframe 
df = pd.read_html(str(table))


j = (df.groupby(["Field (Discovery)","NPDID information carrier"], 
    as_index=False)
      .apply(lambda x: x[[ 'Year','Month','Oil - saleable [mill Sm3]','Gas - 
      saleable [bill Sm3]','NGL - saleable [mill Sm3]','Condensate - 
      saleable [mill Sm3]','Oil equivalents - saleable [mill Sm3]','Water - 
      wellbores [mill Sm3]' ]].to_dict('r'))
     .reset_index(drop=True)
     .rename(columns={0: 'MonthlyProduction'})
     .to_json(orient='records'))

print(j)

print(json.dumps(json.loads(j), indent=2, sort_keys=True))

Answer 1

根据 pandas 文档 https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_html.html pd.read_html returns 数据帧列表（每个 table 一个数据帧）。因此，您需要选择要分组的对象。喜欢：

# for the first one:
df = pd.read_html(str(table))[0]

Pandas 按问题分组

Pandas groupby issue

python

beautifulsoup

pandas

pandas-groupby