如何使用 IntervalIndex 生成 .cat.codes

Question

我有一个数据集，我将 qcut 并将类别形成为 pandas.core.indexes.interval.IntervalIndex。

如何使用我从原始数据集生成的相同类别对未来的数据集进行分类？

解释码：

>>> import pandas as pd
>>> import numpy as np
>>> np.random.seed(42)

>>> df = pd.DataFrame(np.random.randint(0,100,size=(10, 4)), columns=list('ABCD'))

>>> df
    A   B   C   D
 0  51  92  14  71
 1  60  20  82  86
 2  74  74  87  99
 3  23  2   21  52
 4  1   87  29  37
 5  1   63  59  20
 6  32  75  57  21
 7  88  48  90  58
 8  41  91  59  79
 9  14  61  61  46

>>> categories_a = pd.qcut(df['A'], 4).cat.categories

>>> type(categories_a)
pandas.core.indexes.interval.IntervalIndex

>>> categories_a
IntervalIndex([(0.999, 16.25], (16.25, 36.5], (36.5, 57.75], (57.75, 88.0]]
          closed='right',
          dtype='interval[float64]')

我试过的：

我尝试使用 CategoricalDtype 和 df['B'].astype(categories_a) 之类的东西但没有成功。我运行不知道如何以优雅的方式做到这一点。

预期结果：

鉴于df和categories_a与上面相同，我想将df['B']中的所有元素都按照.cat.codes转换为由 pd.qcut(df['A'], 4).cat.codes 生成。输出看起来像：

df['B'] 
original --> processed # comment
92 --> 3 # (57.75, 88.0] this one actually goes through the roof
20 --> 1 # (16.25, 36.5]
74 --> 3 # (57.75, 88.0]
2  --> 0 # (0.999, 16.25]
87 --> 3 # (57.75, 88.0]
63 --> 3 # (57.75, 88.0]
75 --> 3 # (57.75, 88.0]
48 --> 2 # (36.5, 57.75]
91 --> 3 # (57.75, 88.0]
61 --> 3 # (57.75, 88.0]

我希望已经足够清楚了。

Answer 1

您可以将区间的左端点和右端点拼接在一起，以构建 bins 以供在 pd.cut 中使用。

def cut_by_cats(cats):
    bins = [c[0].left] + [i.right for i in c]
    def cut_(series):
        return pd.cut(series, bins)
    return cut_

cut = cut_by_cats(pd.qcut(df.A, 4).cat.categories)

df.apply(cut)

                A               B               C              D
0   (36.5, 57.75]             NaN  (0.999, 16.25]  (57.75, 88.0]
1   (57.75, 88.0]   (16.25, 36.5]   (57.75, 88.0]  (57.75, 88.0]
2   (57.75, 88.0]   (57.75, 88.0]   (57.75, 88.0]            NaN
3   (16.25, 36.5]  (0.999, 16.25]   (16.25, 36.5]  (36.5, 57.75]
4  (0.999, 16.25]   (57.75, 88.0]   (16.25, 36.5]  (36.5, 57.75]
5  (0.999, 16.25]   (57.75, 88.0]   (57.75, 88.0]  (16.25, 36.5]
6   (16.25, 36.5]   (57.75, 88.0]   (36.5, 57.75]  (16.25, 36.5]
7   (57.75, 88.0]   (36.5, 57.75]             NaN  (57.75, 88.0]
8   (36.5, 57.75]             NaN   (57.75, 88.0]  (57.75, 88.0]
9  (0.999, 16.25]   (57.75, 88.0]   (57.75, 88.0]  (36.5, 57.75]

Answer 2

使用与上述相同的逻辑，但要获取类别而不是已处理数据框中的范围：

np.random.seed(42)
df = pd.DataFrame(np.random.randint(0,100,size=(10, 4)), columns=list('ABCD'))


series, bins = pd.qcut(df["A"], 4, retbins=True, labels=False)
def apply_cut(df):
    for i in df.columns:
        df[i] = pd.cut(df[i], bins=bins, labels=False, include_lowest=True)
    return df

processed = apply_cut(df)

Returns:

>>> processed
   A    B    C    D
0  2  NaN  0.0  3.0
1  3  1.0  3.0  3.0
2  3  3.0  3.0  NaN
3  1  0.0  1.0  2.0
4  0  3.0  1.0  2.0
5  0  3.0  3.0  1.0
6  1  3.0  2.0  1.0
7  3  2.0  NaN  3.0
8  2  NaN  3.0  3.0
9  0  3.0  3.0  2.0

如果您想摆脱 NaN 并将它们强制归入最接近的类别，您可以这样做，但将 -float(np.inf) 和 float(np.inf) 添加到第一个和最后一个 bin：

series, bins = pd.qcut(df["A"], 4, retbins=True, labels=False)
bins[0] = -float(np.inf)
bins[-1] = float(np.inf)
def apply_cut(df):
    for i in df.columns:
        df[i] = pd.cut(df[i], bins=bins, labels=False, include_lowest=True,right=False)
    return df

processed = apply_cut(df)

>>> processed
   A  B  C  D
0  2  3  0  3
1  3  1  3  3
2  3  3  3  3
3  1  0  1  2
4  0  3  1  2
5  0  3  3  1
6  1  3  2  1
7  3  2  3  3
8  2  3  3  3
9  0  3  3  2

piR 编辑

请delete/alter您认为合适。这是你的 post 我打扰了 (-:

bins = pd.qcut(df.A, 4, retbins=True)[1]
bins[0] -= np.finfo(np.float).resolution

df.apply(lambda c: pd.cut(c, bins))

               A              B              C              D
0  (36.5, 57.75]            NaN   (1.0, 16.25]  (57.75, 88.0]
1  (57.75, 88.0]  (16.25, 36.5]  (57.75, 88.0]  (57.75, 88.0]
2  (57.75, 88.0]  (57.75, 88.0]  (57.75, 88.0]            NaN
3  (16.25, 36.5]   (1.0, 16.25]  (16.25, 36.5]  (36.5, 57.75]
4   (1.0, 16.25]  (57.75, 88.0]  (16.25, 36.5]  (36.5, 57.75]
5   (1.0, 16.25]  (57.75, 88.0]  (57.75, 88.0]  (16.25, 36.5]
6  (16.25, 36.5]  (57.75, 88.0]  (36.5, 57.75]  (16.25, 36.5]
7  (57.75, 88.0]  (36.5, 57.75]            NaN  (57.75, 88.0]
8  (36.5, 57.75]            NaN  (57.75, 88.0]  (57.75, 88.0]
9   (1.0, 16.25]  (57.75, 88.0]  (57.75, 88.0]  (36.5, 57.75]

或：

bins = pd.qcut(df.A, 4, retbins=True)[1]
bins[0] = -float(np.inf)
bins[-1] = float(np.inf)

processed = df.apply(lambda c: pd.cut(c, bins, labels=False))

>>> processed
   A  B  C  D
0  2  3  0  3
1  3  1  3  3
2  3  3  3  3
3  1  0  1  2
4  0  3  1  2
5  0  3  3  1
6  1  3  2  1
7  3  2  3  3
8  2  3  3  3
9  0  3  3  2

如何使用 IntervalIndex 生成 .cat.codes

How to use IntervalIndex to generate .cat.codes

python

categories

dataframe

pandas

categorical-data

piR 编辑