pandas DataFrame 多个子串匹配，也将某一行的特定匹配子串放入新列

Question

我正在尝试从调查响应 DF 中提取一些记录。所有这些记录都需要至少包含一些关键词中的一个。例如：现在我有一个数据框 df:

svy_rspns_txt
I like it
I hate it
It's a scam
It's shaddy
Scam!
Good service
Very disappointed

现在如果我运行

kw="hate,scam,shaddy,disappoint"
sensitive_words=[unicode(x,'unicode-escape') for x in kw.lower().split(",")]
df=df[df["svy_rspns_txt"].astype('unicode').str.contains('|'.join(sensitive_words),case=False,na=False)]

我会得到这样的结果

svy_rspns_txt
I hate it
It's a scam
It's shaddy
Scam!
Very disappointed

现在我如何添加一列 "matched_word" 来显示匹配的确切字符串，这样我可以获得如下结果：

svy_rspns_txt            matched_word
I hate it                hate
It's a scam              scam
It's shaddy              shaddy
Scam!                    scam
Very disappointed        disappoint

Answer 1

使用生成器表达式 next:

df = pd.DataFrame({'text': ["I like it", "I hate it", "It's a scam", "It's shaddy",
                            "Scam!", "Good service", "Very disappointed"]})

kw = "hate,scam,shaddy,disappoint"

words = set(kw.split(','))

df['match'] = df['text'].apply(lambda x: next((i for i in words if i in x.lower()), np.nan))

print(df)

                text       match
0          I like it         NaN
1          I hate it        hate
2        It's a scam        scam
3        It's shaddy      shaddy
4              Scam!        scam
5       Good service         NaN
6  Very disappointed  disappoint

您可以通过 pd.Series.notnull 或注释 NaN != NaN:

过滤有效字符串

res = df[df['match'].notnull()]
# or, res = df[df['match'].notna()]
# or, res = df[df['match'] == df['match']]

print(res)

                text       match
1          I hate it        hate
2        It's a scam        scam
3        It's shaddy      shaddy
4              Scam!        scam
6  Very disappointed  disappoint

pandas DataFrame 多个子串匹配，也将某一行的特定匹配子串放入新列

pandas DataFrame multiple substrings match, also put the particular matched substring for a row into a new column

python

substring

match

string-matching

pandas