在 pandas 中对具有固定列的多个列应用操作

Question

我有一个数据框，如下所示。最后一列显示所有列的值总和，即 A、B、D、K 和 T。请注意，某些列也有 NaN。

word1,A,B,D,K,T,sum
na,,63.0,,,870.0,933.0
sva,,1.0,,3.0,695.0,699.0
a,,102.0,,1.0,493.0,596.0
sa,2.0,487.0,,2.0,15.0,506.0
su,1.0,44.0,,136.0,214.0,395.0
waw,1.0,9.0,,34.0,296.0,340.0

如何计算每一行的熵？即我应该找到类似下面的内容

df['A']/df['sum']*log(df['A']/df['sum']) + df['B']/df['sum']*log(df['B']/df['sum']) + ...... + df['T']/df['sum']*log(df['T']/df['sum'])

条件是每当log里面的值变为zero或NaN时，整个值应该被视为零（根据定义，日志将return 错误，因为日志 0 未定义）。

我知道使用 lambda 运算应用于各个列。在这里，我无法考虑将固定列 sum 应用于不同列 A、B、D 等的纯 pandas 解决方案。尽管我可以想到对具有硬编码列值的 CSV 文件进行简单的循环迭代。

Answer 1

我想你可以使用 ix for selecting columns from A to T, then divide by div with numpy.log. Last use sum:

print (df['A']/df['sum']*np.log(df['A']/df['sum']))
0         NaN
1         NaN
2         NaN
3   -0.021871
4   -0.015136
5   -0.017144
dtype: float64

print (df.ix[:,'A':'T'].div(df['sum'],axis=0)*np.log(df.ix[:,'A':'T'].div(df['sum'],axis=0)))
          A         B   D         K         T
0       NaN -0.181996 NaN       NaN -0.065191
1       NaN -0.009370 NaN -0.023395 -0.005706
2       NaN -0.302110 NaN -0.010722 -0.156942
3 -0.021871 -0.036835 NaN -0.021871 -0.104303
4 -0.015136 -0.244472 NaN -0.367107 -0.332057
5 -0.017144 -0.096134 NaN -0.230259 -0.120651

print((df.ix[:,'A':'T'].div(df['sum'],axis=0)*np.log(df.ix[:,'A':'T'].div(df['sum'],axis=0)))
         .sum(axis=1))
0   -0.247187
1   -0.038471
2   -0.469774
3   -0.184881
4   -0.958774
5   -0.464188
dtype: float64

Answer 2

df1 = df.iloc[:, :-1]
df2 = df1.div(df1.sum(1), axis=0)
df2.mul(np.log(df2)).sum(1)

word1
na    -0.247187
sva   -0.038471
a     -0.469774
sa    -0.184881
su    -0.958774
waw   -0.464188
dtype: float64

设置

from StringIO import StringIO
import pandas as pd

text = """word1,A,B,D,K,T,sum
na,,63.0,,,870.0,933.0
sva,,1.0,,3.0,695.0,699.0
a,,102.0,,1.0,493.0,596.0
sa,2.0,487.0,,2.0,15.0,506.0
su,1.0,44.0,,136.0,214.0,395.0
waw,1.0,9.0,,34.0,296.0,340.0"""

df = pd.read_csv(StringIO(text), index_col=0)

df

在 pandas 中对具有固定列的多个列应用操作

Applying an operation on multiple columns with a fixed column in pandas

python

sum

multiple-columns

dataframe

pandas

设置