分组并减去 pandas 中的列
Groupby and subtract columns in pandas
我有一个包含 4 列的时间序列数据，我想对 FisherID、DateFishing 和 Total_Catch 列进行分组，并对 Weight 列求和.另外，我想用权重列中的值减去 Total_catch 列中的值，其结果将保存在名为 DIFF 的新列中。而且，我想在列 DIFF 中显示高于 0.1 的值。
这是我的代码。
df["DIFF"]=df.groupby(["FisherID", "DateFishing", "Total_Catch"]) ["Weight"].sum()-["Total_Catch"]>=0.1
我的数据：
FisherID DateFishing Total_Catch Weight
1 24-Oct-11 0.9 0.2
1 24-Oct-11 0.9 0.264
1 24-Oct-11 0.9 0.37
2 25-Oct-11 0.7 0.144
2 27-Oct-11 8.2 0.084
2 27-Oct-11 8.2 0.45
3 27-Oct-11 8.2 0.61
3 27-Oct-11 8.2 7
3 29-Oct-11 0.64 0.184
我认为您正在寻找 groupby + transform:
df['Sum'] = df.groupby(
["FisherID", "DateFishing", "Total_Catch"]
)["Weight"].transform('sum')
然后，通过从 Total_Catch.
中减去 Weight col 来找到 Diff
df['Diff'] = (df['Total_Catch'] - df['Weight'])
df
FisherID DateFishing Total_Catch Weight Sum Diff
0 1 24-Oct-11 0.90 0.200 0.834 0.700
1 1 24-Oct-11 0.90 0.264 0.834 0.636
2 1 24-Oct-11 0.90 0.370 0.834 0.530
3 2 25-Oct-11 0.70 0.144 0.144 0.556
4 2 27-Oct-11 8.20 0.084 0.534 8.116
5 2 27-Oct-11 8.20 0.450 0.534 7.750
6 3 27-Oct-11 8.20 0.610 7.610 7.590
7 3 27-Oct-11 8.20 7.000 7.610 1.200
8 3 29-Oct-11 0.64 0.184 0.184 0.456
或者，如果您尝试从 Total_Catch 中减去分组的 Weight，请使用：
df['Diff'] = df["Total_Catch"] -df.groupby(["FisherID", \
"DateFishing", "Total_Catch"])["Weight"].transform('sum')
df
FisherID DateFishing Total_Catch Weight Diff
0 1 24-Oct-11 0.90 0.200 0.066
1 1 24-Oct-11 0.90 0.264 0.066
2 1 24-Oct-11 0.90 0.370 0.066
3 2 25-Oct-11 0.70 0.144 0.556
4 2 27-Oct-11 8.20 0.084 7.666
5 2 27-Oct-11 8.20 0.450 7.666
6 3 27-Oct-11 8.20 0.610 0.590
7 3 27-Oct-11 8.20 7.000 0.590
8 3 29-Oct-11 0.64 0.184 0.456
查询行
本节以第二个选项的结果为基础。请注意，所有这些选项都将布尔掩码应用于数据框。如果您只想要掩码，请不要将其应用于数据框。只需应用条件并打印：
df.Diff > 0.1
0 False
1 False
2 False
3 True
4 True
5 True
6 True
7 True
8 True
Name: Diff, dtype: bool
如果您想提取所有有效行，有几个选项。
df.query
df.query('Diff > 0.1')
FisherID DateFishing Total_Catch Weight Diff
3 2 25-Oct-11 0.70 0.144 0.556
4 2 27-Oct-11 8.20 0.084 7.666
5 2 27-Oct-11 8.20 0.450 7.666
6 3 27-Oct-11 8.20 0.610 0.590
7 3 27-Oct-11 8.20 7.000 0.590
8 3 29-Oct-11 0.64 0.184 0.456
boolean indexing
df[df.Diff > 0.1]
FisherID DateFishing Total_Catch Weight Diff
3 2 25-Oct-11 0.70 0.144 0.556
4 2 27-Oct-11 8.20 0.084 7.666
5 2 27-Oct-11 8.20 0.450 7.666
6 3 27-Oct-11 8.20 0.610 0.590
7 3 27-Oct-11 8.20 7.000 0.590
8 3 29-Oct-11 0.64 0.184 0.456
df.eval
df[df.eval('Diff > 0.1')]
FisherID DateFishing Total_Catch Weight Diff
3 2 25-Oct-11 0.70 0.144 0.556
4 2 27-Oct-11 8.20 0.084 7.666
5 2 27-Oct-11 8.20 0.450 7.666
6 3 27-Oct-11 8.20 0.610 0.590
7 3 27-Oct-11 8.20 7.000 0.590
8 3 29-Oct-11 0.64 0.184 0.456
df.where 和 dropna
df.where(df.Diff > 0.1).dropna(how='all')
FisherID DateFishing Total_Catch Weight Diff
3 2.0 25-Oct-11 0.70 0.144 0.556
4 2.0 27-Oct-11 8.20 0.084 7.666
5 2.0 27-Oct-11 8.20 0.450 7.666
6 3.0 27-Oct-11 8.20 0.610 0.590
7 3.0 27-Oct-11 8.20 7.000 0.590
8 3.0 29-Oct-11 0.64 0.184 0.456
np.where 和 df.iloc:
df.iloc[np.where(df.Diff > 0.1)[0]]
FisherID DateFishing Total_Catch Weight Diff
3 2 25-Oct-11 0.70 0.144 0.556
4 2 27-Oct-11 8.20 0.084 7.666
5 2 27-Oct-11 8.20 0.450 7.666
6 3 27-Oct-11 8.20 0.610 0.590
7 3 27-Oct-11 8.20 7.000 0.590
8 3 29-Oct-11 0.64 0.184 0.456
请注意，这些结果具有原始索引 df。如果要重置索引，请使用 reset_index:
df[df.Diff > 0.1].reset_index(drop=True)
FisherID DateFishing Total_Catch Weight Diff
0 2 25-Oct-11 0.70 0.144 0.556
1 2 27-Oct-11 8.20 0.084 7.666
2 2 27-Oct-11 8.20 0.450 7.666
3 3 27-Oct-11 8.20 0.610 0.590
4 3 27-Oct-11 8.20 7.000 0.590
5 3 29-Oct-11 0.64 0.184 0.456
我有一个包含 4 列的时间序列数据，我想对 FisherID、DateFishing 和 Total_Catch 列进行分组，并对 Weight 列求和.另外，我想用权重列中的值减去 Total_catch 列中的值，其结果将保存在名为 DIFF 的新列中。而且，我想在列 DIFF 中显示高于 0.1 的值。
这是我的代码。
df["DIFF"]=df.groupby(["FisherID", "DateFishing", "Total_Catch"]) ["Weight"].sum()-["Total_Catch"]>=0.1
我的数据：
FisherID DateFishing Total_Catch Weight
1 24-Oct-11 0.9 0.2
1 24-Oct-11 0.9 0.264
1 24-Oct-11 0.9 0.37
2 25-Oct-11 0.7 0.144
2 27-Oct-11 8.2 0.084
2 27-Oct-11 8.2 0.45
3 27-Oct-11 8.2 0.61
3 27-Oct-11 8.2 7
3 29-Oct-11 0.64 0.184
我认为您正在寻找 groupby + transform:
df['Sum'] = df.groupby(
["FisherID", "DateFishing", "Total_Catch"]
)["Weight"].transform('sum')
然后，通过从 Total_Catch.
Weight col 来找到 Diff
df['Diff'] = (df['Total_Catch'] - df['Weight'])
df
FisherID DateFishing Total_Catch Weight Sum Diff
0 1 24-Oct-11 0.90 0.200 0.834 0.700
1 1 24-Oct-11 0.90 0.264 0.834 0.636
2 1 24-Oct-11 0.90 0.370 0.834 0.530
3 2 25-Oct-11 0.70 0.144 0.144 0.556
4 2 27-Oct-11 8.20 0.084 0.534 8.116
5 2 27-Oct-11 8.20 0.450 0.534 7.750
6 3 27-Oct-11 8.20 0.610 7.610 7.590
7 3 27-Oct-11 8.20 7.000 7.610 1.200
8 3 29-Oct-11 0.64 0.184 0.184 0.456
或者，如果您尝试从 Total_Catch 中减去分组的 Weight，请使用：
df['Diff'] = df["Total_Catch"] -df.groupby(["FisherID", \
"DateFishing", "Total_Catch"])["Weight"].transform('sum')
df
FisherID DateFishing Total_Catch Weight Diff
0 1 24-Oct-11 0.90 0.200 0.066
1 1 24-Oct-11 0.90 0.264 0.066
2 1 24-Oct-11 0.90 0.370 0.066
3 2 25-Oct-11 0.70 0.144 0.556
4 2 27-Oct-11 8.20 0.084 7.666
5 2 27-Oct-11 8.20 0.450 7.666
6 3 27-Oct-11 8.20 0.610 0.590
7 3 27-Oct-11 8.20 7.000 0.590
8 3 29-Oct-11 0.64 0.184 0.456
查询行
本节以第二个选项的结果为基础。请注意，所有这些选项都将布尔掩码应用于数据框。如果您只想要掩码，请不要将其应用于数据框。只需应用条件并打印：
df.Diff > 0.1
0 False
1 False
2 False
3 True
4 True
5 True
6 True
7 True
8 True
Name: Diff, dtype: bool
如果您想提取所有有效行，有几个选项。
df.query
df.query('Diff > 0.1')
FisherID DateFishing Total_Catch Weight Diff
3 2 25-Oct-11 0.70 0.144 0.556
4 2 27-Oct-11 8.20 0.084 7.666
5 2 27-Oct-11 8.20 0.450 7.666
6 3 27-Oct-11 8.20 0.610 0.590
7 3 27-Oct-11 8.20 7.000 0.590
8 3 29-Oct-11 0.64 0.184 0.456
boolean indexing
df[df.Diff > 0.1]
FisherID DateFishing Total_Catch Weight Diff
3 2 25-Oct-11 0.70 0.144 0.556
4 2 27-Oct-11 8.20 0.084 7.666
5 2 27-Oct-11 8.20 0.450 7.666
6 3 27-Oct-11 8.20 0.610 0.590
7 3 27-Oct-11 8.20 7.000 0.590
8 3 29-Oct-11 0.64 0.184 0.456
df.eval
df[df.eval('Diff > 0.1')]
FisherID DateFishing Total_Catch Weight Diff
3 2 25-Oct-11 0.70 0.144 0.556
4 2 27-Oct-11 8.20 0.084 7.666
5 2 27-Oct-11 8.20 0.450 7.666
6 3 27-Oct-11 8.20 0.610 0.590
7 3 27-Oct-11 8.20 7.000 0.590
8 3 29-Oct-11 0.64 0.184 0.456
df.where 和 dropna
df.where(df.Diff > 0.1).dropna(how='all')
FisherID DateFishing Total_Catch Weight Diff
3 2.0 25-Oct-11 0.70 0.144 0.556
4 2.0 27-Oct-11 8.20 0.084 7.666
5 2.0 27-Oct-11 8.20 0.450 7.666
6 3.0 27-Oct-11 8.20 0.610 0.590
7 3.0 27-Oct-11 8.20 7.000 0.590
8 3.0 29-Oct-11 0.64 0.184 0.456
np.where 和 df.iloc:
df.iloc[np.where(df.Diff > 0.1)[0]]
FisherID DateFishing Total_Catch Weight Diff
3 2 25-Oct-11 0.70 0.144 0.556
4 2 27-Oct-11 8.20 0.084 7.666
5 2 27-Oct-11 8.20 0.450 7.666
6 3 27-Oct-11 8.20 0.610 0.590
7 3 27-Oct-11 8.20 7.000 0.590
8 3 29-Oct-11 0.64 0.184 0.456
请注意，这些结果具有原始索引 df。如果要重置索引，请使用 reset_index:
df[df.Diff > 0.1].reset_index(drop=True)
FisherID DateFishing Total_Catch Weight Diff
0 2 25-Oct-11 0.70 0.144 0.556
1 2 27-Oct-11 8.20 0.084 7.666
2 2 27-Oct-11 8.20 0.450 7.666
3 3 27-Oct-11 8.20 0.610 0.590
4 3 27-Oct-11 8.20 7.000 0.590
5 3 29-Oct-11 0.64 0.184 0.456