使用 Python 抓取网页时如何删除 pandas 数据框中的字符？
How to remove characters in pandas data frame when web scraping with Python?
我正在尝试通过使用 Python 3 将本网站的 table 内容抓取到 .csv 文件中：
2011-2012 NBA National Schedule
table 开头是这样的：
Revised Schedule Original Schedule
Date Time Game Net Time Game Net
Sun., 12/25/11 12 PM BOS (1) at NY (1) TNT 12 PM BOS (7) at NY (7) ESPN
Sun., 12/25/11 2:30 PM MIA (1) at DAL (1) ABC 2:30 PM MIA (8) at DAL (5) ABC
Sun., 12/25/11 5 PM CHI (1) at LAL (1) ABC 5 PM CHI (6) at LAL (9) ABC
Sun., 12/25/11 8 PM ORL (1) at OKC (1) ESPN no game no game no game
Sun., 12/25/11 10:30 PM LAC (1) at GS (1) ESPN no game no game no game
Tue., 12/27/11 8 PM BOS (2) at MIA (2) TNT no game no game no game
Tue., 12/27/11 10:30 PM UTA (1) at LAL (2) TNT no game no game no game
我只对修订后的时间表感兴趣，即前 4 列。我想要的 .csv 文件中的输出如下所示：
我正在使用这些软件包：
import re
import requests
import pandas as pd
import numpy as np
from bs4 import BeautifulSoup
from itertools import groupby
这是我为匹配我想要的格式而做的代码：
df = pd.read_html("https://www.sportsmediawatch.com/2011/12/revised-2011-12-nba-national-tv-schedule/", header=0)[0]
revisedCols = ['Date'] + [ col for col in df.columns if 'Revised' in col ]
df = df[revisedCols]
df.columns = df.iloc[0,:]
df = df.iloc[1:,:].reset_index(drop=True)
# Format Date to m/d/y
df['Date'] = np.where(df.Date.str.startswith(('10/', '11/', '12/')), df.Date + ' 11', df.Date + ' 12')
df['Date']=pd.to_datetime(df['Date'])
df['Date']=df['Date'].dt.strftime('%m/%d/%Y')
# Split the Game column
df[['Away','Home']] = df.Game.str.split('at',expand=True)
# Final dataframe with desired columns
df = df[['Date','Time','Away','Home','Net']]
df.columns = ['Date', 'Time', 'Away', 'Home', 'Network']
print(df)
输出：
Date Time Away Home Network
0 12/25/2011 12 PM BOS (1) NY (1) TNT
1 12/25/2011 2:30 PM MIA (1) DAL (1) ABC
2 12/25/2011 5 PM CHI (1) LAL (1) ABC
3 12/25/2011 8 PM ORL (1) OKC (1) ESPN
4 12/25/2011 10:30 PM LAC (1) GS (1) ESPN
5 12/27/2011 8 PM BOS (2) MIA (2) TNT
6 12/27/2011 10:30 PM UTA (1) LAL (2) TNT
我注意到“客场”和“主场”栏中每个球队名称旁边都有 (1)、(2) 等。 我如何实施抓取工具以删除“客场”和“主场”列中每个球队名称旁边的 (1)、(2) 等？
你可以使用str.replace with the parenthesis and the number(s) and also str.strip，因为开头或结尾似乎有一些空格：
df['Away'] = df['Away'].str.replace('\(\d*\)', '').str.strip()
df['Home'] = df['Home'].str.replace('\(\d*\)', '').str.strip()
print (df.head())
Date Time Away Home Network
0 12/25/2011 12 PM BOS NY TNT
1 12/25/2011 2:30 PM MIA DAL ABC
2 12/25/2011 5 PM CHI LAL ABC
3 12/25/2011 8 PM ORL OKC ESPN
4 12/25/2011 10:30 PM LAC GS ESPN
import re
import numpy as np
import pandas as pd
dataset = pd.read_csv("Dataset.csv")
dataset.rename(columns={'Country(or dependent territory)': 'Country'}, inplace = True)
dataset.rename(columns={'% of worldpopulation': 'Percentage of World Population'}, inplace = True)
dataset.rename(columns={'Total Area': 'Total Area (km2)'}, inplace = True)
您可以在拆分游戏列后添加此代码
df['Away']=df['Away'].astype(str).str[0:-4]
df['Home']=df['Home'].astype(str).str[0:-4]
不要在 'at' 处拆分游戏列，不要特别声明分隔符。 .split() 将在每个白色 space 处拆分，然后您只需要 0 索引和 3rd 索引值。所以实际上只需更改 1 行代码：
来自 df[['Away','Home']] = df.Game.str.split('at',expand=True)
至 df[['Away','Home']] = df.Game.str.split(expand=True)[[0,3]]
import pandas as pd
import numpy as np
df = pd.read_html("https://www.sportsmediawatch.com/2011/12/revised-2011-12-nba-national-tv-schedule/", header=0)[0]
revisedCols = ['Date'] + [ col for col in df.columns if 'Revised' in col ]
df = df[revisedCols]
df.columns = df.iloc[0,:]
df = df.iloc[1:,:].reset_index(drop=True)
# Format Date to m/d/y
df['Date'] = np.where(df.Date.str.startswith(('10/', '11/', '12/')), df.Date + ' 11', df.Date + ' 12')
df['Date']=pd.to_datetime(df['Date'])
df['Date']=df['Date'].dt.strftime('%m/%d/%Y')
# Split the Game column
df[['Away','Home']] = df.Game.str.split(expand=True)[[0,3]]
# Final dataframe with desired columns
df = df[['Date','Time','Away','Home','Net']]
df.columns = ['Date', 'Time', 'Away', 'Home', 'Network']
print(df)
我正在尝试通过使用 Python 3 将本网站的 table 内容抓取到 .csv 文件中： 2011-2012 NBA National Schedule
table 开头是这样的：
Revised Schedule Original Schedule
Date Time Game Net Time Game Net
Sun., 12/25/11 12 PM BOS (1) at NY (1) TNT 12 PM BOS (7) at NY (7) ESPN
Sun., 12/25/11 2:30 PM MIA (1) at DAL (1) ABC 2:30 PM MIA (8) at DAL (5) ABC
Sun., 12/25/11 5 PM CHI (1) at LAL (1) ABC 5 PM CHI (6) at LAL (9) ABC
Sun., 12/25/11 8 PM ORL (1) at OKC (1) ESPN no game no game no game
Sun., 12/25/11 10:30 PM LAC (1) at GS (1) ESPN no game no game no game
Tue., 12/27/11 8 PM BOS (2) at MIA (2) TNT no game no game no game
Tue., 12/27/11 10:30 PM UTA (1) at LAL (2) TNT no game no game no game
我只对修订后的时间表感兴趣，即前 4 列。我想要的 .csv 文件中的输出如下所示：
我正在使用这些软件包：
import re
import requests
import pandas as pd
import numpy as np
from bs4 import BeautifulSoup
from itertools import groupby
这是我为匹配我想要的格式而做的代码：
df = pd.read_html("https://www.sportsmediawatch.com/2011/12/revised-2011-12-nba-national-tv-schedule/", header=0)[0]
revisedCols = ['Date'] + [ col for col in df.columns if 'Revised' in col ]
df = df[revisedCols]
df.columns = df.iloc[0,:]
df = df.iloc[1:,:].reset_index(drop=True)
# Format Date to m/d/y
df['Date'] = np.where(df.Date.str.startswith(('10/', '11/', '12/')), df.Date + ' 11', df.Date + ' 12')
df['Date']=pd.to_datetime(df['Date'])
df['Date']=df['Date'].dt.strftime('%m/%d/%Y')
# Split the Game column
df[['Away','Home']] = df.Game.str.split('at',expand=True)
# Final dataframe with desired columns
df = df[['Date','Time','Away','Home','Net']]
df.columns = ['Date', 'Time', 'Away', 'Home', 'Network']
print(df)
输出：
Date Time Away Home Network
0 12/25/2011 12 PM BOS (1) NY (1) TNT
1 12/25/2011 2:30 PM MIA (1) DAL (1) ABC
2 12/25/2011 5 PM CHI (1) LAL (1) ABC
3 12/25/2011 8 PM ORL (1) OKC (1) ESPN
4 12/25/2011 10:30 PM LAC (1) GS (1) ESPN
5 12/27/2011 8 PM BOS (2) MIA (2) TNT
6 12/27/2011 10:30 PM UTA (1) LAL (2) TNT
我注意到“客场”和“主场”栏中每个球队名称旁边都有 (1)、(2) 等。 我如何实施抓取工具以删除“客场”和“主场”列中每个球队名称旁边的 (1)、(2) 等？
你可以使用str.replace with the parenthesis and the number(s) and also str.strip，因为开头或结尾似乎有一些空格：
df['Away'] = df['Away'].str.replace('\(\d*\)', '').str.strip()
df['Home'] = df['Home'].str.replace('\(\d*\)', '').str.strip()
print (df.head())
Date Time Away Home Network
0 12/25/2011 12 PM BOS NY TNT
1 12/25/2011 2:30 PM MIA DAL ABC
2 12/25/2011 5 PM CHI LAL ABC
3 12/25/2011 8 PM ORL OKC ESPN
4 12/25/2011 10:30 PM LAC GS ESPN
import re
import numpy as np
import pandas as pd
dataset = pd.read_csv("Dataset.csv")
dataset.rename(columns={'Country(or dependent territory)': 'Country'}, inplace = True)
dataset.rename(columns={'% of worldpopulation': 'Percentage of World Population'}, inplace = True)
dataset.rename(columns={'Total Area': 'Total Area (km2)'}, inplace = True)
您可以在拆分游戏列后添加此代码
df['Away']=df['Away'].astype(str).str[0:-4]
df['Home']=df['Home'].astype(str).str[0:-4]
不要在 'at' 处拆分游戏列，不要特别声明分隔符。 .split() 将在每个白色 space 处拆分，然后您只需要 0 索引和 3rd 索引值。所以实际上只需更改 1 行代码：
来自 df[['Away','Home']] = df.Game.str.split('at',expand=True)
至 df[['Away','Home']] = df.Game.str.split(expand=True)[[0,3]]
import pandas as pd
import numpy as np
df = pd.read_html("https://www.sportsmediawatch.com/2011/12/revised-2011-12-nba-national-tv-schedule/", header=0)[0]
revisedCols = ['Date'] + [ col for col in df.columns if 'Revised' in col ]
df = df[revisedCols]
df.columns = df.iloc[0,:]
df = df.iloc[1:,:].reset_index(drop=True)
# Format Date to m/d/y
df['Date'] = np.where(df.Date.str.startswith(('10/', '11/', '12/')), df.Date + ' 11', df.Date + ' 12')
df['Date']=pd.to_datetime(df['Date'])
df['Date']=df['Date'].dt.strftime('%m/%d/%Y')
# Split the Game column
df[['Away','Home']] = df.Game.str.split(expand=True)[[0,3]]
# Final dataframe with desired columns
df = df[['Date','Time','Away','Home','Net']]
df.columns = ['Date', 'Time', 'Away', 'Home', 'Network']
print(df)