大数据集清洗：如何基于多类别和按行顺序搜索来填充缺失数据
Large data set cleaning: How to fill in missing data based on multiple categories and searching by row order
这是我的第一个 Whosebug post，所以我希望它不会太难理解。
我有一个大型数据集（~14,000）行鸟类观察结果。这些数据是通过站在一个地方（点）并计算您在 3 分钟内看到的鸟类来收集的。在每个点计数中，一个新的鸟类观察结果成为一个新行，因此有许多重复的日期、时间、地点和点（地点内的特定位置）。同样，每个点数为 3 分钟。因此，如果您在第 1 分钟看到 yellow warbler（编码为 YEWA），那么它将与特定的 MINUTE=1 相关联点计数（日期、地点、点和时间）。 ID=观察者首字母和 Number=发现的鸟类数量（此处不一定重要）。
但是，如果看到 NO BIRDS，则 "NOBI" 会进入该特定分钟的数据集。因此，如果整个 3 分钟点计数都有 NOBI，则它们将是三行，具有相同的日期、站点、点和时间，并且在 "BIRD" 列中的 "NOBI" 三行中的每一行行。
所以我有两个主要问题。 第一个是一些观察者在所有三分钟内输入一次"NOBI"，而不是三次（每分钟一次）。 "MINUTE" 任何地方
已留空（变为 NA），AND "BIRD"="NOBI"，我需要添加三行数据，所有列的值都相同，除了"MINUTE"，各行应为 1、2 和 3。
如果看起来像这样：
ID DATE SITE POINT TIME MINUTE BIRD NUMBER
1 BS 5/9/2018 CW2 U125 7:51 NA NOBI NA
2 BS 5/9/2018 CW1 D250 8:12 1 YEWA 2
3 BS 5/9/2018 CW1 D250 8:12 2 NOBI NA
4 BS 5/9/2018 CW1 D250 8:12 3 LABU 1
它应该看起来像这样：
ID DATE SITE POINT TIME MINUTE BIRD NUMBER
1 BS 5/9/2018 CW2 U125 7:51 1 NOBI NA
2 BS 5/9/2018 CW2 U125 7:51 2 NOBI NA
3 BS 5/9/2018 CW2 U125 7:51 3 NOBI NA
4 BS 5/9/2018 CW1 D250 8:12 1 YEWA 2
5 BS 5/9/2018 CW1 D250 8:12 2 NOBI NA
6 BS 5/9/2018 CW1 D250 8:12 3 LABU 1
注意：如果您想将这些数据中的一些输入到您的 R 控制台中，我在这个 post 末尾使用 dput 包含了一些数据，这应该比输入更容易复制粘贴以上内容
我尝试重现具有多个条件的 if 语句失败了（基于：
R multiple conditions in if statement & Ifelse in R with multiple categorical conditions) 我尝试用多种方式编写这些代码，包括使用 dplyr 的管道，但请参阅下面的一些代码示例、注释和错误消息。
>if(PC$BIRD == "NOBI" & PC$MINUTE==NA){PC$Fix<-TRUE}
Error in if (PC$BIRD == "NOBI" & PC$MINUTE == NA) { :
missing value where TRUE/FALSE needed
In addition: Warning message:
In if (PC$BIRD == "NOBI" & PC$MINUTE == NA) { :
the condition has length > 1 and only the first element will be used
## Then I need to do something like this:
>if(PC$Fix<-TRUE){duplicate(row where Fix==TRUE, times=2)} #I know this isn't
### even close, but I want the row to be replicated two more times so
### that there are 3 total rows witht he same values
### Fix indicates that a fix is needed in this example
# Then somehow I need to assign a 1 to PC$MINUTE for the first row (original row),
# a 2 to the next row (with other values from other columns being the same), and a 3
# to the last duplicated row (still other values from other columns being the same)
第二个问题，对我来说似乎更难的是按顺序搜索数据集，或者以某种方式按日期、站点、点和时间搜索数据集。分钟值应始终从 1... 到 2... 再到 3，然后对于下一组日期、时间、地点和点返回到 1。也就是说，每个点计数都应具有所有值 1:3。然而，一次计数可能在 MINUTE=1 内有多次目击，因此在 MINUTE=2 之前有 5 或 6（或 20）个 MINUTE=1。但是，同样，该数据集中的一些观察者在没有 BIRDS (NOBI) 时只是简单地留下一行，而不是每分钟写一行 BIRD="NOBI"。也就是说，如果数据集是：
ID DATE SITE POINT TIME MINUTE BIRD NUMBER
...
4 BS 5/9/2018 CW2 U125 7:54 1 AMRO 1
5 BS 5/9/2018 CW2 U125 7:54 1 SPTO 1
6 BS 5/9/2018 CW2 U125 7:57 1 AMRO 1
7 BS 5/9/2018 CW2 U125 7:57 1 SPTO 1
8 BS 5/9/2018 CW2 U125 7:57 1 AMCR 3
9 BS 5/9/2018 CW2 U125 7:57 2 SPTO 1
10 BS 5/9/2018 CW2 U125 7:57 2 HOWR 1
11 BS 5/9/2018 CW2 U125 7:57 3 UNBI 1
我们可以看到7:57点计数时间完成（有1:3的MINUTE值）。但是，7:54 点计数时间停止在 MINUTE=1。意思是，我需要在下面输入另外两行，这些行具有所有相同的 DATE、SITE、POINT、TIME 信息，但是对于第一个添加的行 MINUTE=2 和 BIRD="NOBI" MINUTE=3 和 BIRD= "NOBI" 对于第二个添加的行。所以它应该是这样的：
ID DATE SITE POINT TIME MINUTE BIRD NUMBER
...
4 BS 5/9/2018 CW2 U125 7:54 1 AMRO 1
5 BS 5/9/2018 CW2 U125 7:54 1 SPTO 1
6 BS 5/9/2018 CW2 U125 7:54 2 NOBI NA
7 BS 5/9/2018 CW2 U125 7:54 3 NOBI NA
8 BS 5/9/2018 CW2 U125 7:57 1 AMRO 1
9 BS 5/9/2018 CW2 U125 7:57 1 SPTO 1
10 BS 5/9/2018 CW2 U125 7:57 1 AMCR 3
11 BS 5/9/2018 CW2 U125 7:57 2 SPTO 1
12 BS 5/9/2018 CW2 U125 7:57 2 HOWR 1
13 BS 5/9/2018 CW2 U125 7:57 3 UNBI 1
最后，我明白这是一个很长很复杂的问题，我可能没有表达清楚。如果需要任何说明，请告诉我，我很乐意听到任何建议，即使它不能完全回答我的问题。提前致谢！
如果您想将我的数据样本输入 R
，此行下方的所有内容仅对您有用
要将我的数据输入 R 控制台，复制并粘贴从 "structure" 函数到代码末尾的所有内容，以使用代码将其作为数据框输入 R 控制台：dataframe<-structure(list...
请参阅 寻求帮助。
PC<-read.csv("PC.csv") ### ORIGINAL FILE
dput(PC)
structure(list(ID = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "BS", class = "factor"),
DATE = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "5/9/2018", class = "factor"),
SITE = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "CW2", class = "factor"),
POINT = structure(c(2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = c("M", "U125"), class = "factor"),
TIME = structure(c(8L, 8L, 8L, 9L, 9L, 10L, 10L, 10L, 10L,
10L, 10L, 11L, 1L, 1L, 1L, 2L, 2L, 2L, 3L, 3L, 4L, 4L, 4L,
4L, 5L, 5L, 5L, 5L, 5L, 5L, 6L, 7L), .Label = c("6:48", "6:51",
"6:54", "6:57", "7:12", "7:15", "7:18", "7:51", "7:54", "7:57",
"8:00"), class = "factor"), MINUTE = c(1L, 2L, 3L, 1L, 1L,
1L, 1L, 1L, 2L, 2L, 3L, 1L, 1L, 2L, 3L, 1L, 2L, 3L, 1L, 1L,
1L, 1L, 2L, 3L, 1L, 1L, 1L, 2L, 3L, 3L, NA, NA), BIRD = structure(c(6L,
6L, 6L, 2L, 7L, 2L, 7L, 1L, 7L, 5L, 8L, 8L, 6L, 6L, 6L, 6L,
6L, 6L, 7L, 7L, 7L, 7L, 6L, 8L, 3L, 7L, 9L, 5L, 4L, 2L, 6L,
6L), .Label = c("AMCR", "AMRO", "BRSP", "DUFL", "HOWR", "NOBI",
"SPTO", "UNBI", "VESP"), class = "factor"), NUMBER = c(NA,
NA, NA, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, NA, NA, NA, NA,
NA, NA, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, NA,
NA)), class = "data.frame", row.names = c(NA, -32L))
PCc<-read.csv("PC_Corrected.csv") #### WHAT I NEED MY DATABASE TO LOOK LIKE
dput(PCc)
structure(list(ID = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L), .Label = "BS", class = "factor"), DATE = structure(c(1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "5/9/2018", class = "factor"),
SITE = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L), .Label = "CW2", class = "factor"), POINT = structure(c(2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = c("M",
"U125"), class = "factor"), TIME = structure(c(8L, 8L, 8L,
9L, 9L, 9L, 9L, 10L, 10L, 10L, 10L, 10L, 10L, 11L, 11L, 11L,
1L, 1L, 1L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 4L, 4L, 4L, 4L, 5L,
5L, 5L, 5L, 5L, 5L, 6L, 6L, 6L, 7L, 7L, 7L), .Label = c("6:48",
"6:51", "6:54", "6:57", "7:12", "7:15", "7:18", "7:51", "7:54",
"7:57", "8:00"), class = "factor"), MINUTE = c(1L, 2L, 3L,
1L, 1L, 2L, 3L, 1L, 1L, 1L, 2L, 2L, 3L, 1L, 2L, 3L, 1L, 2L,
3L, 1L, 2L, 3L, 1L, 1L, 2L, 3L, 1L, 1L, 2L, 3L, 1L, 1L, 1L,
2L, 3L, 3L, 1L, 2L, 3L, 1L, 2L, 3L), BIRD = structure(c(6L,
6L, 6L, 2L, 7L, 6L, 6L, 2L, 7L, 1L, 7L, 5L, 8L, 8L, 6L, 6L,
6L, 6L, 6L, 6L, 6L, 6L, 7L, 7L, 6L, 6L, 7L, 7L, 6L, 8L, 3L,
7L, 9L, 5L, 4L, 2L, 6L, 6L, 6L, 6L, 6L, 6L), .Label = c("AMCR",
"AMRO", "BRSP", "DUFL", "HOWR", "NOBI", "SPTO", "UNBI", "VESP"
), class = "factor"), NUMBER = c(NA, NA, NA, 1L, 1L, NA,
NA, 1L, 1L, 1L, 1L, 1L, 1L, 1L, NA, NA, NA, NA, NA, NA, NA,
NA, 1L, 1L, NA, NA, 1L, 1L, NA, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
NA, NA, NA, NA, NA, NA)), class = "data.frame", row.names = c(NA,
-42L))
这是一种使用 tidyverse 元包中的 dplyr 和 tidyr 的方法。
# Step one - identify missing rows.
# For each DATE, SITE, POINT, TIME, count how many of each minute
library(tidyverse)
# Convert factors to character to make later joining simpler,
# and fix missing ID's by assuming prior line should be used,
# and make NOBI rows have a count of NA
PC_2_clean <- PC %>%
mutate_if(is.factor, as.character) %>%
fill(ID, .direction = "up") %>%
mutate(NUMBER = if_else(BIRD == "NOBI", NA_integer_, NUMBER))
# Create a wide table with spots for each minute. Missing will
# show up as NA's
# All the NA's here in the 1, 2, and 3 columns represent
# missing minutes that we should add.
PC_3_NA_find <- PC_2_clean %>%
count(ID, DATE, SITE, POINT, TIME, MINUTE) %>%
spread(MINUTE, n)
PC_3_NA_find
# A tibble: 11 x 9
# ID DATE SITE POINT TIME `1` `2` `3` `<NA>`
# <chr> <chr> <chr> <chr> <chr> <int> <int> <int> <int>
# 1 BS 5/9/2018 CW2 M 7:12 3 1 2 NA
# 2 BS 5/9/2018 CW2 M 7:15 NA NA NA 1
# 3 BS 5/9/2018 CW2 M 7:18 NA NA NA 1
# 4 BS 5/9/2018 CW2 U125 6:48 1 1 1 NA
# 5 BS 5/9/2018 CW2 U125 6:51 1 1 1 NA
# 6 BS 5/9/2018 CW2 U125 6:54 2 NA NA NA
# 7 BS 5/9/2018 CW2 U125 6:57 2 1 1 NA
# 8 BS 5/9/2018 CW2 U125 7:51 1 1 1 NA
# 9 BS 5/9/2018 CW2 U125 7:54 2 NA NA NA
# 10 BS 5/9/2018 CW2 U125 7:57 3 2 1 NA
# 11 BS 5/9/2018 CW2 U125 8:00 1 NA NA NA
# Take the NA minute entries and make the desired line for each
PC_4_rows_to_add <- PC_3_NA_find %>%
gather(MINUTE, count, `1`:`3`) %>%
filter(is.na(count)) %>%
select(-count, -`<NA>`) %>%
mutate(MINUTE = as.integer(MINUTE),
BIRD = "NOBI",
NUMBER = NA_integer_)
# Add these lines to the original, remove the NA minute rows
# (these have been replaced with minute rows), and sort
PC_5_with_NOBIs <- PC_2_clean %>%
bind_rows(PC_4_rows_to_add) %>%
filter(MINUTE != "NA") %>%
arrange(ID, DATE, SITE, POINT, TIME, MINUTE, BIRD)
# Check result
PC_5_with_NOBIs %>%
count(ID, DATE, SITE, POINT, TIME, MINUTE) %>%
spread(MINUTE, n)
PC_5_with_NOBIs
# Now to confirm it matches your desired output.
# Note, I convert to character to avoid mismatches between factors
PCc_char <- PCc %>%
mutate_if(is.factor, as.character) %>%
arrange(ID, DATE, SITE, POINT, TIME, MINUTE, BIRD)
identical(PC_5_with_NOBIs, PCc_char)
# [1] TRUE
这是我的第一个 Whosebug post，所以我希望它不会太难理解。
我有一个大型数据集（~14,000）行鸟类观察结果。这些数据是通过站在一个地方（点）并计算您在 3 分钟内看到的鸟类来收集的。在每个点计数中，一个新的鸟类观察结果成为一个新行，因此有许多重复的日期、时间、地点和点（地点内的特定位置）。同样，每个点数为 3 分钟。因此，如果您在第 1 分钟看到 yellow warbler（编码为 YEWA），那么它将与特定的 MINUTE=1 相关联点计数（日期、地点、点和时间）。 ID=观察者首字母和 Number=发现的鸟类数量（此处不一定重要）。
但是，如果看到 NO BIRDS，则 "NOBI" 会进入该特定分钟的数据集。因此，如果整个 3 分钟点计数都有 NOBI，则它们将是三行，具有相同的日期、站点、点和时间，并且在 "BIRD" 列中的 "NOBI" 三行中的每一行行。
所以我有两个主要问题。 第一个是一些观察者在所有三分钟内输入一次"NOBI"，而不是三次（每分钟一次）。 "MINUTE" 任何地方 已留空（变为 NA），AND "BIRD"="NOBI"，我需要添加三行数据，所有列的值都相同，除了"MINUTE"，各行应为 1、2 和 3。
如果看起来像这样：
ID DATE SITE POINT TIME MINUTE BIRD NUMBER
1 BS 5/9/2018 CW2 U125 7:51 NA NOBI NA
2 BS 5/9/2018 CW1 D250 8:12 1 YEWA 2
3 BS 5/9/2018 CW1 D250 8:12 2 NOBI NA
4 BS 5/9/2018 CW1 D250 8:12 3 LABU 1
它应该看起来像这样：
ID DATE SITE POINT TIME MINUTE BIRD NUMBER
1 BS 5/9/2018 CW2 U125 7:51 1 NOBI NA
2 BS 5/9/2018 CW2 U125 7:51 2 NOBI NA
3 BS 5/9/2018 CW2 U125 7:51 3 NOBI NA
4 BS 5/9/2018 CW1 D250 8:12 1 YEWA 2
5 BS 5/9/2018 CW1 D250 8:12 2 NOBI NA
6 BS 5/9/2018 CW1 D250 8:12 3 LABU 1
注意：如果您想将这些数据中的一些输入到您的 R 控制台中，我在这个 post 末尾使用 dput 包含了一些数据，这应该比输入更容易复制粘贴以上内容
我尝试重现具有多个条件的 if 语句失败了（基于： R multiple conditions in if statement & Ifelse in R with multiple categorical conditions) 我尝试用多种方式编写这些代码，包括使用 dplyr 的管道，但请参阅下面的一些代码示例、注释和错误消息。
>if(PC$BIRD == "NOBI" & PC$MINUTE==NA){PC$Fix<-TRUE}
Error in if (PC$BIRD == "NOBI" & PC$MINUTE == NA) { :
missing value where TRUE/FALSE needed
In addition: Warning message:
In if (PC$BIRD == "NOBI" & PC$MINUTE == NA) { :
the condition has length > 1 and only the first element will be used
## Then I need to do something like this:
>if(PC$Fix<-TRUE){duplicate(row where Fix==TRUE, times=2)} #I know this isn't
### even close, but I want the row to be replicated two more times so
### that there are 3 total rows witht he same values
### Fix indicates that a fix is needed in this example
# Then somehow I need to assign a 1 to PC$MINUTE for the first row (original row),
# a 2 to the next row (with other values from other columns being the same), and a 3
# to the last duplicated row (still other values from other columns being the same)
第二个问题，对我来说似乎更难的是按顺序搜索数据集，或者以某种方式按日期、站点、点和时间搜索数据集。分钟值应始终从 1... 到 2... 再到 3，然后对于下一组日期、时间、地点和点返回到 1。也就是说，每个点计数都应具有所有值 1:3。然而，一次计数可能在 MINUTE=1 内有多次目击，因此在 MINUTE=2 之前有 5 或 6（或 20）个 MINUTE=1。但是，同样，该数据集中的一些观察者在没有 BIRDS (NOBI) 时只是简单地留下一行，而不是每分钟写一行 BIRD="NOBI"。也就是说，如果数据集是：
ID DATE SITE POINT TIME MINUTE BIRD NUMBER
...
4 BS 5/9/2018 CW2 U125 7:54 1 AMRO 1
5 BS 5/9/2018 CW2 U125 7:54 1 SPTO 1
6 BS 5/9/2018 CW2 U125 7:57 1 AMRO 1
7 BS 5/9/2018 CW2 U125 7:57 1 SPTO 1
8 BS 5/9/2018 CW2 U125 7:57 1 AMCR 3
9 BS 5/9/2018 CW2 U125 7:57 2 SPTO 1
10 BS 5/9/2018 CW2 U125 7:57 2 HOWR 1
11 BS 5/9/2018 CW2 U125 7:57 3 UNBI 1
我们可以看到7:57点计数时间完成（有1:3的MINUTE值）。但是，7:54 点计数时间停止在 MINUTE=1。意思是，我需要在下面输入另外两行，这些行具有所有相同的 DATE、SITE、POINT、TIME 信息，但是对于第一个添加的行 MINUTE=2 和 BIRD="NOBI" MINUTE=3 和 BIRD= "NOBI" 对于第二个添加的行。所以它应该是这样的：
ID DATE SITE POINT TIME MINUTE BIRD NUMBER
...
4 BS 5/9/2018 CW2 U125 7:54 1 AMRO 1
5 BS 5/9/2018 CW2 U125 7:54 1 SPTO 1
6 BS 5/9/2018 CW2 U125 7:54 2 NOBI NA
7 BS 5/9/2018 CW2 U125 7:54 3 NOBI NA
8 BS 5/9/2018 CW2 U125 7:57 1 AMRO 1
9 BS 5/9/2018 CW2 U125 7:57 1 SPTO 1
10 BS 5/9/2018 CW2 U125 7:57 1 AMCR 3
11 BS 5/9/2018 CW2 U125 7:57 2 SPTO 1
12 BS 5/9/2018 CW2 U125 7:57 2 HOWR 1
13 BS 5/9/2018 CW2 U125 7:57 3 UNBI 1
最后，我明白这是一个很长很复杂的问题，我可能没有表达清楚。如果需要任何说明，请告诉我，我很乐意听到任何建议，即使它不能完全回答我的问题。提前致谢！
如果您想将我的数据样本输入 R
，此行下方的所有内容仅对您有用要将我的数据输入 R 控制台，复制并粘贴从 "structure" 函数到代码末尾的所有内容，以使用代码将其作为数据框输入 R 控制台：dataframe<-structure(list...
请参阅
PC<-read.csv("PC.csv") ### ORIGINAL FILE
dput(PC)
structure(list(ID = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "BS", class = "factor"),
DATE = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "5/9/2018", class = "factor"),
SITE = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "CW2", class = "factor"),
POINT = structure(c(2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = c("M", "U125"), class = "factor"),
TIME = structure(c(8L, 8L, 8L, 9L, 9L, 10L, 10L, 10L, 10L,
10L, 10L, 11L, 1L, 1L, 1L, 2L, 2L, 2L, 3L, 3L, 4L, 4L, 4L,
4L, 5L, 5L, 5L, 5L, 5L, 5L, 6L, 7L), .Label = c("6:48", "6:51",
"6:54", "6:57", "7:12", "7:15", "7:18", "7:51", "7:54", "7:57",
"8:00"), class = "factor"), MINUTE = c(1L, 2L, 3L, 1L, 1L,
1L, 1L, 1L, 2L, 2L, 3L, 1L, 1L, 2L, 3L, 1L, 2L, 3L, 1L, 1L,
1L, 1L, 2L, 3L, 1L, 1L, 1L, 2L, 3L, 3L, NA, NA), BIRD = structure(c(6L,
6L, 6L, 2L, 7L, 2L, 7L, 1L, 7L, 5L, 8L, 8L, 6L, 6L, 6L, 6L,
6L, 6L, 7L, 7L, 7L, 7L, 6L, 8L, 3L, 7L, 9L, 5L, 4L, 2L, 6L,
6L), .Label = c("AMCR", "AMRO", "BRSP", "DUFL", "HOWR", "NOBI",
"SPTO", "UNBI", "VESP"), class = "factor"), NUMBER = c(NA,
NA, NA, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, NA, NA, NA, NA,
NA, NA, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, NA,
NA)), class = "data.frame", row.names = c(NA, -32L))
PCc<-read.csv("PC_Corrected.csv") #### WHAT I NEED MY DATABASE TO LOOK LIKE
dput(PCc)
structure(list(ID = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L), .Label = "BS", class = "factor"), DATE = structure(c(1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "5/9/2018", class = "factor"),
SITE = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L), .Label = "CW2", class = "factor"), POINT = structure(c(2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = c("M",
"U125"), class = "factor"), TIME = structure(c(8L, 8L, 8L,
9L, 9L, 9L, 9L, 10L, 10L, 10L, 10L, 10L, 10L, 11L, 11L, 11L,
1L, 1L, 1L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 4L, 4L, 4L, 4L, 5L,
5L, 5L, 5L, 5L, 5L, 6L, 6L, 6L, 7L, 7L, 7L), .Label = c("6:48",
"6:51", "6:54", "6:57", "7:12", "7:15", "7:18", "7:51", "7:54",
"7:57", "8:00"), class = "factor"), MINUTE = c(1L, 2L, 3L,
1L, 1L, 2L, 3L, 1L, 1L, 1L, 2L, 2L, 3L, 1L, 2L, 3L, 1L, 2L,
3L, 1L, 2L, 3L, 1L, 1L, 2L, 3L, 1L, 1L, 2L, 3L, 1L, 1L, 1L,
2L, 3L, 3L, 1L, 2L, 3L, 1L, 2L, 3L), BIRD = structure(c(6L,
6L, 6L, 2L, 7L, 6L, 6L, 2L, 7L, 1L, 7L, 5L, 8L, 8L, 6L, 6L,
6L, 6L, 6L, 6L, 6L, 6L, 7L, 7L, 6L, 6L, 7L, 7L, 6L, 8L, 3L,
7L, 9L, 5L, 4L, 2L, 6L, 6L, 6L, 6L, 6L, 6L), .Label = c("AMCR",
"AMRO", "BRSP", "DUFL", "HOWR", "NOBI", "SPTO", "UNBI", "VESP"
), class = "factor"), NUMBER = c(NA, NA, NA, 1L, 1L, NA,
NA, 1L, 1L, 1L, 1L, 1L, 1L, 1L, NA, NA, NA, NA, NA, NA, NA,
NA, 1L, 1L, NA, NA, 1L, 1L, NA, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
NA, NA, NA, NA, NA, NA)), class = "data.frame", row.names = c(NA,
-42L))
这是一种使用 tidyverse 元包中的 dplyr 和 tidyr 的方法。
# Step one - identify missing rows.
# For each DATE, SITE, POINT, TIME, count how many of each minute
library(tidyverse)
# Convert factors to character to make later joining simpler,
# and fix missing ID's by assuming prior line should be used,
# and make NOBI rows have a count of NA
PC_2_clean <- PC %>%
mutate_if(is.factor, as.character) %>%
fill(ID, .direction = "up") %>%
mutate(NUMBER = if_else(BIRD == "NOBI", NA_integer_, NUMBER))
# Create a wide table with spots for each minute. Missing will
# show up as NA's
# All the NA's here in the 1, 2, and 3 columns represent
# missing minutes that we should add.
PC_3_NA_find <- PC_2_clean %>%
count(ID, DATE, SITE, POINT, TIME, MINUTE) %>%
spread(MINUTE, n)
PC_3_NA_find
# A tibble: 11 x 9
# ID DATE SITE POINT TIME `1` `2` `3` `<NA>`
# <chr> <chr> <chr> <chr> <chr> <int> <int> <int> <int>
# 1 BS 5/9/2018 CW2 M 7:12 3 1 2 NA
# 2 BS 5/9/2018 CW2 M 7:15 NA NA NA 1
# 3 BS 5/9/2018 CW2 M 7:18 NA NA NA 1
# 4 BS 5/9/2018 CW2 U125 6:48 1 1 1 NA
# 5 BS 5/9/2018 CW2 U125 6:51 1 1 1 NA
# 6 BS 5/9/2018 CW2 U125 6:54 2 NA NA NA
# 7 BS 5/9/2018 CW2 U125 6:57 2 1 1 NA
# 8 BS 5/9/2018 CW2 U125 7:51 1 1 1 NA
# 9 BS 5/9/2018 CW2 U125 7:54 2 NA NA NA
# 10 BS 5/9/2018 CW2 U125 7:57 3 2 1 NA
# 11 BS 5/9/2018 CW2 U125 8:00 1 NA NA NA
# Take the NA minute entries and make the desired line for each
PC_4_rows_to_add <- PC_3_NA_find %>%
gather(MINUTE, count, `1`:`3`) %>%
filter(is.na(count)) %>%
select(-count, -`<NA>`) %>%
mutate(MINUTE = as.integer(MINUTE),
BIRD = "NOBI",
NUMBER = NA_integer_)
# Add these lines to the original, remove the NA minute rows
# (these have been replaced with minute rows), and sort
PC_5_with_NOBIs <- PC_2_clean %>%
bind_rows(PC_4_rows_to_add) %>%
filter(MINUTE != "NA") %>%
arrange(ID, DATE, SITE, POINT, TIME, MINUTE, BIRD)
# Check result
PC_5_with_NOBIs %>%
count(ID, DATE, SITE, POINT, TIME, MINUTE) %>%
spread(MINUTE, n)
PC_5_with_NOBIs
# Now to confirm it matches your desired output.
# Note, I convert to character to avoid mismatches between factors
PCc_char <- PCc %>%
mutate_if(is.factor, as.character) %>%
arrange(ID, DATE, SITE, POINT, TIME, MINUTE, BIRD)
identical(PC_5_with_NOBIs, PCc_char)
# [1] TRUE