计算数据集中满足条件的元素 - Python/pandas
Counting elements that satisfy a criteria in a dataset- Python/pandas
在数据集中，我有四列名为 TP1、TP2 等（代表测试阶段 1,2...）。这些列中的信息可以是以下 5 种之一：Not_tested、Not_finished、too_low、介于 150 和 190 之间的值，以及什么都没有。
如果一个元素在 4 列中的任何一列中至少显示一次值：too_low 或介于 150 和 190 之间的值，则该元素被视为经过测试。如果这两个值在任何时候都没有显示在该元素的行中，则视为未测试。
我需要报告如下内容：
“测试元素的数量：43（以及所有元素的百分比）”（等于显示的行数 too_low 或 介于 150 和 190)
之间的值
“新测试元素的数量：# 和 %”（所以从上面找到的元素，在 NEW[=42= 中有一个 YES ] 列？）
“旧测试元素的数量：# 和 %”（取而代之的是 NO）
下面是部分数据集的示例：
+=======+==========+=====+======+==============+=========+==============+=====+
| ID | Location | NEW | YEAR | PT1 | PT2 | PT3 | PT4 |
+=======+==========+=====+======+==============+=========+==============+=====+
| GF342 | Q1 | YES | 2021 | | | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| IF874 | Q3 | NO | 2018 | NOT_TESTED | | TOO_LOW | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| FH386 | Q1 | NO | 2019 | | | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| KJ190 | Q3 | YES | 2021 | NOT_FINISHED | | TOO_LOW | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| TY748 | Q3 | YES | 2021 | | | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| YT947 | Q4 | NO | 2019 | | TOO_LOW | | 165 |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| DF063 | Q3 | NO | 2019 | 180 | | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| ET512 | Q1 | YES | 2021 | | | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| GC714 | Q2 | NO | 2018 | | 160 | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| SD978 | Q3 | NO | 2019 | | TOO_LOW | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| EF472 | Q1 | NO | 2018 | | | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| PL489 | Q2 | YES | 2021 | | | NOT_FINISHED | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| AZ315 | Q1 | NO | 2018 | TOO_LOW | | | 180 |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| OL821 | Q1 | YES | 2021 | | | 185 | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| HN765 | Q3 | YES | 2021 | 155 | | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| ED589 | Q1 | YES | 2021 | | | 165 | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
我解决这个问题的想法是对包含必要值的所有行使用 .sum() ，但是我如何制定一个规则，只有当这些值存在时才被计算在内？我是否可以使用 T 或 F 向数据框添加一个新列，具体取决于它是否满足具有 too_low 或数字 (150-190) 的规则。我对这一切都很陌生，不知道该怎么做。我不一定要直接回答，但如果可能的话，我会提供一些关于问题的指导，谢谢。
数据框：
pd.DataFrame({'ID': {0: 'GF342', 1: 'IF874', 2: 'FH386', 3: 'KJ190', 4: 'TY748', 5: 'YT947', 6: 'DF063', 7: 'ET512', 8: 'GC714', 9: 'SD978', 10: 'EF472', 11: 'PL489', 12: 'AZ315', 13: 'OL821', 14: 'HN765', 15: 'ED589'}, 'Location': {0: 'Q1', 1: 'Q3', 2: 'Q1', 3: 'Q3', 4: 'Q3', 5: 'Q4', 6: 'Q3', 7: 'Q1', 8: 'Q2', 9: 'Q3', 10: 'Q1', 11: 'Q2', 12: 'Q1', 13: 'Q1', 14: 'Q3', 15: 'Q1'}, 'NEW': {0: 'YES', 1: 'NO', 2: 'NO', 3: 'YES', 4: 'YES', 5: 'NO', 6: 'NO', 7: 'YES', 8: 'NO', 9: 'NO', 10: 'NO', 11: 'YES', 12: 'NO', 13: 'YES', 14: 'YES', 15: 'YES'}, 'YEAR': {0: 2021, 1: 2018, 2: 2019, 3: 2021, 4: 2021, 5: 2019, 6: 2019, 7: 2021, 8: 2018, 9: 2019, 10: 2018, 11: 2021, 12: 2018, 13: 2021, 14: 2021, 15: 2021}, 'PT1': {0: '', 1: 'NOT_TESTED', 2: '', 3: 'NOT_FINISHED', 4: '', 5: '', 6: '180', 7: '', 8: '', 9: '', 10: '', 11: '', 12: 'TOO_LOW', 13: '', 14: '155', 15: ''}, 'PT2': {0: '', 1: '', 2: '', 3: '', 4: '', 5: 'TOO_LOW', 6: '', 7: '', 8: '160', 9: 'TOO_LOW', 10: '', 11: '', 12: '', 13: '', 14: '', 15: ''}, 'PT3': {0: '', 1: 'TOO_LOW', 2: '', 3: 'TOO_LOW', 4: '', 5: '', 6: '', 7: '', 8: '', 9: '', 10: '', 11: 'NOT_FINISHED', 12: '', 13: '185', 14: '', 15: '165'}, 'PT4': {0: '', 1: '', 2: '', 3: '', 4: '', 5: 165.0, 6: '', 7: '', 8: '', 9: '', 10: '', 11: '', 12: 180.0, 13: '', 14: '', 15: ''}})
我假设您正在使用库 Pandas 中的 DataFrame。我遇到了类似的问题并遍历了数据框。变量 df 是我使用 Pandas 中的“read_csv”方法从 CSV 中提取的数据帧。我不知道你是如何获得数据集的，但它应该是类似的格式。我希望它对你有用，或者至少你得到了一种方法
tooLow = "TOO_LOW"
elementsCount = 0
index = len(df.columns)
for r in range(len(df.index)):
for c in range(index):
columnValue = df.at[r, df.columns[c]]
if (150 <= columnValue <= 190) or (columnValue == tooLow):
elementsCount += 1
print("Elements satisfying criteria: ", elementsCount)
你可以试试这个
df = pd.DataFrame([['NO', 'NOT_TESTED', None, 'TOO_LOW', None],
['YES', 'NOT_FINISHED ', None, 'TOO_LOW', None],
['YES', None, None, None, None],
['NO', 180, None, None, None],
['NO', 'TOO_LOW', None, None, 180],
['YES', 155.6, None, None, None]], columns=['NEW', 'PT1', 'PT2', 'PT3', 'PT4'])
# Number of tested elements: 43 (and a percentage from all elements)"
# (which equates to how many rows show too_low or a value between 150 and 190)
filter_ = df.loc[:, 'PT1':'PT4']\
.apply(lambda x: x.eq('TOO_LOW') | x.astype(str).str.fullmatch(r'\d+\.*\d*'), axis=0)\
.any(axis=1)
number_of_tested = filter_.sum()
#"Number of new tested elements
new_tested_elements = df[filter_ & df['NEW'].eq('YES')].shape[0]
#"Number of old tested elements
old_tested_elements = df[filter_ & df['NEW'].eq('NO')].shape[0]
在数据集中，我有四列名为 TP1、TP2 等（代表测试阶段 1,2...）。这些列中的信息可以是以下 5 种之一：Not_tested、Not_finished、too_low、介于 150 和 190 之间的值，以及什么都没有。
如果一个元素在 4 列中的任何一列中至少显示一次值：too_low 或介于 150 和 190 之间的值，则该元素被视为经过测试。如果这两个值在任何时候都没有显示在该元素的行中，则视为未测试。
我需要报告如下内容：
“测试元素的数量：43（以及所有元素的百分比）”（等于显示的行数 too_low 或 介于 150 和 190)
之间的值“新测试元素的数量：# 和 %”（所以从上面找到的元素，在 NEW[=42= 中有一个 YES ] 列？）
“旧测试元素的数量：# 和 %”（取而代之的是 NO）
下面是部分数据集的示例：
+=======+==========+=====+======+==============+=========+==============+=====+
| ID | Location | NEW | YEAR | PT1 | PT2 | PT3 | PT4 |
+=======+==========+=====+======+==============+=========+==============+=====+
| GF342 | Q1 | YES | 2021 | | | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| IF874 | Q3 | NO | 2018 | NOT_TESTED | | TOO_LOW | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| FH386 | Q1 | NO | 2019 | | | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| KJ190 | Q3 | YES | 2021 | NOT_FINISHED | | TOO_LOW | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| TY748 | Q3 | YES | 2021 | | | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| YT947 | Q4 | NO | 2019 | | TOO_LOW | | 165 |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| DF063 | Q3 | NO | 2019 | 180 | | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| ET512 | Q1 | YES | 2021 | | | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| GC714 | Q2 | NO | 2018 | | 160 | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| SD978 | Q3 | NO | 2019 | | TOO_LOW | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| EF472 | Q1 | NO | 2018 | | | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| PL489 | Q2 | YES | 2021 | | | NOT_FINISHED | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| AZ315 | Q1 | NO | 2018 | TOO_LOW | | | 180 |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| OL821 | Q1 | YES | 2021 | | | 185 | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| HN765 | Q3 | YES | 2021 | 155 | | | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
| ED589 | Q1 | YES | 2021 | | | 165 | |
+-------+----------+-----+------+--------------+---------+--------------+-----+
我解决这个问题的想法是对包含必要值的所有行使用 .sum() ，但是我如何制定一个规则，只有当这些值存在时才被计算在内？我是否可以使用 T 或 F 向数据框添加一个新列，具体取决于它是否满足具有 too_low 或数字 (150-190) 的规则。我对这一切都很陌生，不知道该怎么做。我不一定要直接回答，但如果可能的话，我会提供一些关于问题的指导，谢谢。
数据框：
pd.DataFrame({'ID': {0: 'GF342', 1: 'IF874', 2: 'FH386', 3: 'KJ190', 4: 'TY748', 5: 'YT947', 6: 'DF063', 7: 'ET512', 8: 'GC714', 9: 'SD978', 10: 'EF472', 11: 'PL489', 12: 'AZ315', 13: 'OL821', 14: 'HN765', 15: 'ED589'}, 'Location': {0: 'Q1', 1: 'Q3', 2: 'Q1', 3: 'Q3', 4: 'Q3', 5: 'Q4', 6: 'Q3', 7: 'Q1', 8: 'Q2', 9: 'Q3', 10: 'Q1', 11: 'Q2', 12: 'Q1', 13: 'Q1', 14: 'Q3', 15: 'Q1'}, 'NEW': {0: 'YES', 1: 'NO', 2: 'NO', 3: 'YES', 4: 'YES', 5: 'NO', 6: 'NO', 7: 'YES', 8: 'NO', 9: 'NO', 10: 'NO', 11: 'YES', 12: 'NO', 13: 'YES', 14: 'YES', 15: 'YES'}, 'YEAR': {0: 2021, 1: 2018, 2: 2019, 3: 2021, 4: 2021, 5: 2019, 6: 2019, 7: 2021, 8: 2018, 9: 2019, 10: 2018, 11: 2021, 12: 2018, 13: 2021, 14: 2021, 15: 2021}, 'PT1': {0: '', 1: 'NOT_TESTED', 2: '', 3: 'NOT_FINISHED', 4: '', 5: '', 6: '180', 7: '', 8: '', 9: '', 10: '', 11: '', 12: 'TOO_LOW', 13: '', 14: '155', 15: ''}, 'PT2': {0: '', 1: '', 2: '', 3: '', 4: '', 5: 'TOO_LOW', 6: '', 7: '', 8: '160', 9: 'TOO_LOW', 10: '', 11: '', 12: '', 13: '', 14: '', 15: ''}, 'PT3': {0: '', 1: 'TOO_LOW', 2: '', 3: 'TOO_LOW', 4: '', 5: '', 6: '', 7: '', 8: '', 9: '', 10: '', 11: 'NOT_FINISHED', 12: '', 13: '185', 14: '', 15: '165'}, 'PT4': {0: '', 1: '', 2: '', 3: '', 4: '', 5: 165.0, 6: '', 7: '', 8: '', 9: '', 10: '', 11: '', 12: 180.0, 13: '', 14: '', 15: ''}})
我假设您正在使用库 Pandas 中的 DataFrame。我遇到了类似的问题并遍历了数据框。变量 df 是我使用 Pandas 中的“read_csv”方法从 CSV 中提取的数据帧。我不知道你是如何获得数据集的，但它应该是类似的格式。我希望它对你有用，或者至少你得到了一种方法
tooLow = "TOO_LOW"
elementsCount = 0
index = len(df.columns)
for r in range(len(df.index)):
for c in range(index):
columnValue = df.at[r, df.columns[c]]
if (150 <= columnValue <= 190) or (columnValue == tooLow):
elementsCount += 1
print("Elements satisfying criteria: ", elementsCount)
你可以试试这个
df = pd.DataFrame([['NO', 'NOT_TESTED', None, 'TOO_LOW', None],
['YES', 'NOT_FINISHED ', None, 'TOO_LOW', None],
['YES', None, None, None, None],
['NO', 180, None, None, None],
['NO', 'TOO_LOW', None, None, 180],
['YES', 155.6, None, None, None]], columns=['NEW', 'PT1', 'PT2', 'PT3', 'PT4'])
# Number of tested elements: 43 (and a percentage from all elements)"
# (which equates to how many rows show too_low or a value between 150 and 190)
filter_ = df.loc[:, 'PT1':'PT4']\
.apply(lambda x: x.eq('TOO_LOW') | x.astype(str).str.fullmatch(r'\d+\.*\d*'), axis=0)\
.any(axis=1)
number_of_tested = filter_.sum()
#"Number of new tested elements
new_tested_elements = df[filter_ & df['NEW'].eq('YES')].shape[0]
#"Number of old tested elements
old_tested_elements = df[filter_ & df['NEW'].eq('NO')].shape[0]