如何用相应列中的值替换多列中的 NA
How to replace NA in multiple columns with value from corresponding columns
我正在尝试清理数据框，我想用另一列中的相应值替换一列中的 NA。我也想一次对多个列执行此操作。
示例数据框。
set.seed(123)
dates <- seq(as.Date("2016-01-01"), by = "day", length = 10)
names <- rep(c("John Doe", "Jane Smith"), each = 5)
var1_group <- runif(10)
var2_group <- runif(10)
var1_person <- runif(10)
var2_person <- runif(10)
myDF <- data.frame(names, var1_group, var2_group, var1_person, var2_person)
myDF <- cbind(dates, myDF)
在使用 dplyr 进行一些操作后...
myDF <- myDF %>% mutate_each(funs(lag), contains("group"))
myDF <- myDF %>% group_by(names) %>% mutate_each(funs(lag), contains("person"))
我得到了一堆 NA...
dates names var1_group var2_group var1_person var2_person
1 2016-01-01 John Doe NA NA NA NA
2 2016-01-02 John Doe 0.2875775 0.95683335 0.8895393 0.9630242
3 2016-01-03 John Doe 0.7883051 0.45333416 0.6928034 0.9022990
4 2016-01-04 John Doe 0.4089769 0.67757064 0.6405068 0.6907053
5 2016-01-05 John Doe 0.8830174 0.57263340 0.9942698 0.7954674
6 2016-01-06 Jane Smith 0.9404673 0.10292468 NA NA
7 2016-01-07 Jane Smith 0.0455565 0.89982497 0.7085305 0.4777960
8 2016-01-08 Jane Smith 0.5281055 0.24608773 0.5440660 0.7584595
9 2016-01-09 Jane Smith 0.8924190 0.04205953 0.5941420 0.2164079
10 2016-01-10 Jane Smith 0.5514350 0.32792072 0.2891597 0.3181810
我现在想做的是用 *_group 列中的相应值替换 *_person 列中的 NA。 （见第 6 行）
dates names var1_group var2_group var1_person var2_person
1 2016-01-01 John Doe NA NA NA NA
2 2016-01-02 John Doe 0.2875775 0.95683335 0.8895393 0.9630242
3 2016-01-03 John Doe 0.7883051 0.45333416 0.6928034 0.9022990
4 2016-01-04 John Doe 0.4089769 0.67757064 0.6405068 0.6907053
5 2016-01-05 John Doe 0.8830174 0.57263340 0.9942698 0.7954674
6 2016-01-06 Jane Smith 0.9404673 0.10292468 0.9404673 0.1029246
7 2016-01-07 Jane Smith 0.0455565 0.89982497 0.7085305 0.4777960
8 2016-01-08 Jane Smith 0.5281055 0.24608773 0.5440660 0.7584595
9 2016-01-09 Jane Smith 0.8924190 0.04205953 0.5941420 0.2164079
10 2016-01-10 Jane Smith 0.5514350 0.32792072 0.2891597 0.3181810
这适用于一列...
myDF$var1_person <- ifelse(is.na(myDF$var1_person), myDF$var1_group, myDF$var1_person)
但我想一次对所有列执行此操作。在我的实际数据框中，每组大约有 20 列。我已经尝试了很多其他的东西，但我不想用我的废话把这个 post 搞得一团糟。
*如果您可以获得基于列前缀匹配 n 个变量的代码，则可加分。
var1_group > var1_person
var2_group > var2_person
...
varn_group > varn_person
这是一个 "tidyverse" 方法。请注意，正如@Gregor 评论的那样，它有助于整理数据。下面为您处理这个问题，returns 一个有点整洁的数据框。如有必要，我会留给您恢复原始格式。
请注意，我使用了可以在 .
中找到的 mutate_cond() 函数
library(tidyverse)
library(stringr)
myDF %>%
gather(key = col, value = val, -dates, -names) %>%
mutate(col = str_replace(col, "var", "")) %>%
separate(col, into = c("var", "group")) %>%
spread(key = group, value = val) %>%
mutate_cond(is.na(person), person = group)
#> Source: local data frame [20 x 5]
#> Groups: names [2]
#>
#> dates names var group person
#> * <date> <fctr> <chr> <dbl> <dbl>
#> 1 2016-01-01 John Doe 1 NA NA
#> 2 2016-01-01 John Doe 2 NA NA
#> 3 2016-01-02 John Doe 1 0.28757752 0.8895393
#> 4 2016-01-02 John Doe 2 0.95683335 0.9630242
#> 5 2016-01-03 John Doe 1 0.78830514 0.6928034
#> 6 2016-01-03 John Doe 2 0.45333416 0.9022990
#> 7 2016-01-04 John Doe 1 0.40897692 0.6405068
#> 8 2016-01-04 John Doe 2 0.67757064 0.6907053
#> 9 2016-01-05 John Doe 1 0.88301740 0.9942698
#> 10 2016-01-05 John Doe 2 0.57263340 0.7954674
#> 11 2016-01-06 Jane Smith 1 0.94046728 0.9404673
#> 12 2016-01-06 Jane Smith 2 0.10292468 0.1029247
#> 13 2016-01-07 Jane Smith 1 0.04555650 0.7085305
#> 14 2016-01-07 Jane Smith 2 0.89982497 0.4777960
#> 15 2016-01-08 Jane Smith 1 0.52810549 0.5440660
#> 16 2016-01-08 Jane Smith 2 0.24608773 0.7584595
#> 17 2016-01-09 Jane Smith 1 0.89241904 0.5941420
#> 18 2016-01-09 Jane Smith 2 0.04205953 0.2164079
#> 19 2016-01-10 Jane Smith 1 0.55143501 0.2891597
#> 20 2016-01-10 Jane Smith 2 0.32792072 0.3181810
除了最后一行之外的所有内容都是关于整理数据的。最后一行 (mutate_cond()) 处理 NA 值的替换。如果您的列都以这种方式命名，那么这应该扩展到任何 n.
这是一个想法：
tag <- c("person", "group")
# Create a list of 2 elements, persons and groups.
lst <- lapply(tag, function(x) { myDF[grepl(x, colnames(myDF))] })
# Extract everything before the underscore "_" in the column names
ext <- lapply(seq_along(lst), function(x) {
stringi::stri_extract(colnames(lst[[x]]), regex = "^[^_]+(?=_)") })
# Find the common elements between the two
int <- intersect(ext[[1]], ext[[2]])
# Create a new list with only the matching subset
match_list <- lapply(lst, function(x) { select(x, matches(paste(int, collapse = "|"))) })
# Replace all NA values in 'person' by the corresponding values in 'group'
res <- mapply(function(x, y) { replace(x, is.na(x), y[is.na(x)]) },
match_list[[1]], match_list[[2]])
# Assign the result back to the original data.frame
myDF[, colnames(res)] <- res
这应该忽略不匹配的 person/group 对并且只替换匹配的变量
这是一个使用 data.table 中的 set 的选项，它会就地替换
library(data.table)
#convert the data.frame to data.table
setDT(myDF)
#get the column name of 'group' and 'person' columns
nm1 <- grep("group", names(myDF), value = TRUE)
nm2 <- grep("person", names(myDF), value = TRUE)
#loop through the sequence of 'nm1'
for(j in seq_along(nm1)){
#set the elements in the row that are NA for each 'period' column
#with the corresponding row from 'group' column specified in the "value"
set(myDF, i = which(is.na(myDF[[nm2[j]]])), j = nm2[j],
value = myDF[[nm1[j]]][is.na(myDF[[nm2[j]]])])
}
myDF
# dates names var1_group var2_group var1_person var2_person
#1: 2016-01-01 John Doe NA NA NA NA
#2: 2016-01-02 John Doe 0.2875775 0.95683335 0.8895393 0.9630242
#3: 2016-01-03 John Doe 0.7883051 0.45333416 0.6928034 0.9022990
#4: 2016-01-04 John Doe 0.4089769 0.67757064 0.6405068 0.6907053
#5: 2016-01-05 John Doe 0.8830174 0.57263340 0.9942698 0.7954674
#6: 2016-01-06 Jane Smith 0.9404673 0.10292468 0.9404673 0.1029247
#7: 2016-01-07 Jane Smith 0.0455565 0.89982497 0.7085305 0.4777960
#8: 2016-01-08 Jane Smith 0.5281055 0.24608773 0.5440660 0.7584595
#9: 2016-01-09 Jane Smith 0.8924190 0.04205953 0.5941420 0.2164079
#10:2016-01-10 Jane Smith 0.5514350 0.32792072 0.2891597 0.3181810
我正在尝试清理数据框，我想用另一列中的相应值替换一列中的 NA。我也想一次对多个列执行此操作。
示例数据框。
set.seed(123)
dates <- seq(as.Date("2016-01-01"), by = "day", length = 10)
names <- rep(c("John Doe", "Jane Smith"), each = 5)
var1_group <- runif(10)
var2_group <- runif(10)
var1_person <- runif(10)
var2_person <- runif(10)
myDF <- data.frame(names, var1_group, var2_group, var1_person, var2_person)
myDF <- cbind(dates, myDF)
在使用 dplyr 进行一些操作后...
myDF <- myDF %>% mutate_each(funs(lag), contains("group"))
myDF <- myDF %>% group_by(names) %>% mutate_each(funs(lag), contains("person"))
我得到了一堆 NA...
dates names var1_group var2_group var1_person var2_person
1 2016-01-01 John Doe NA NA NA NA
2 2016-01-02 John Doe 0.2875775 0.95683335 0.8895393 0.9630242
3 2016-01-03 John Doe 0.7883051 0.45333416 0.6928034 0.9022990
4 2016-01-04 John Doe 0.4089769 0.67757064 0.6405068 0.6907053
5 2016-01-05 John Doe 0.8830174 0.57263340 0.9942698 0.7954674
6 2016-01-06 Jane Smith 0.9404673 0.10292468 NA NA
7 2016-01-07 Jane Smith 0.0455565 0.89982497 0.7085305 0.4777960
8 2016-01-08 Jane Smith 0.5281055 0.24608773 0.5440660 0.7584595
9 2016-01-09 Jane Smith 0.8924190 0.04205953 0.5941420 0.2164079
10 2016-01-10 Jane Smith 0.5514350 0.32792072 0.2891597 0.3181810
我现在想做的是用 *_group 列中的相应值替换 *_person 列中的 NA。 （见第 6 行）
dates names var1_group var2_group var1_person var2_person
1 2016-01-01 John Doe NA NA NA NA
2 2016-01-02 John Doe 0.2875775 0.95683335 0.8895393 0.9630242
3 2016-01-03 John Doe 0.7883051 0.45333416 0.6928034 0.9022990
4 2016-01-04 John Doe 0.4089769 0.67757064 0.6405068 0.6907053
5 2016-01-05 John Doe 0.8830174 0.57263340 0.9942698 0.7954674
6 2016-01-06 Jane Smith 0.9404673 0.10292468 0.9404673 0.1029246
7 2016-01-07 Jane Smith 0.0455565 0.89982497 0.7085305 0.4777960
8 2016-01-08 Jane Smith 0.5281055 0.24608773 0.5440660 0.7584595
9 2016-01-09 Jane Smith 0.8924190 0.04205953 0.5941420 0.2164079
10 2016-01-10 Jane Smith 0.5514350 0.32792072 0.2891597 0.3181810
这适用于一列...
myDF$var1_person <- ifelse(is.na(myDF$var1_person), myDF$var1_group, myDF$var1_person)
但我想一次对所有列执行此操作。在我的实际数据框中，每组大约有 20 列。我已经尝试了很多其他的东西，但我不想用我的废话把这个 post 搞得一团糟。
*如果您可以获得基于列前缀匹配 n 个变量的代码，则可加分。
var1_group > var1_person
var2_group > var2_person
...
varn_group > varn_person
这是一个 "tidyverse" 方法。请注意，正如@Gregor 评论的那样，它有助于整理数据。下面为您处理这个问题，returns 一个有点整洁的数据框。如有必要，我会留给您恢复原始格式。
请注意，我使用了可以在
mutate_cond() 函数
library(tidyverse)
library(stringr)
myDF %>%
gather(key = col, value = val, -dates, -names) %>%
mutate(col = str_replace(col, "var", "")) %>%
separate(col, into = c("var", "group")) %>%
spread(key = group, value = val) %>%
mutate_cond(is.na(person), person = group)
#> Source: local data frame [20 x 5]
#> Groups: names [2]
#>
#> dates names var group person
#> * <date> <fctr> <chr> <dbl> <dbl>
#> 1 2016-01-01 John Doe 1 NA NA
#> 2 2016-01-01 John Doe 2 NA NA
#> 3 2016-01-02 John Doe 1 0.28757752 0.8895393
#> 4 2016-01-02 John Doe 2 0.95683335 0.9630242
#> 5 2016-01-03 John Doe 1 0.78830514 0.6928034
#> 6 2016-01-03 John Doe 2 0.45333416 0.9022990
#> 7 2016-01-04 John Doe 1 0.40897692 0.6405068
#> 8 2016-01-04 John Doe 2 0.67757064 0.6907053
#> 9 2016-01-05 John Doe 1 0.88301740 0.9942698
#> 10 2016-01-05 John Doe 2 0.57263340 0.7954674
#> 11 2016-01-06 Jane Smith 1 0.94046728 0.9404673
#> 12 2016-01-06 Jane Smith 2 0.10292468 0.1029247
#> 13 2016-01-07 Jane Smith 1 0.04555650 0.7085305
#> 14 2016-01-07 Jane Smith 2 0.89982497 0.4777960
#> 15 2016-01-08 Jane Smith 1 0.52810549 0.5440660
#> 16 2016-01-08 Jane Smith 2 0.24608773 0.7584595
#> 17 2016-01-09 Jane Smith 1 0.89241904 0.5941420
#> 18 2016-01-09 Jane Smith 2 0.04205953 0.2164079
#> 19 2016-01-10 Jane Smith 1 0.55143501 0.2891597
#> 20 2016-01-10 Jane Smith 2 0.32792072 0.3181810
除了最后一行之外的所有内容都是关于整理数据的。最后一行 (mutate_cond()) 处理 NA 值的替换。如果您的列都以这种方式命名，那么这应该扩展到任何 n.
这是一个想法：
tag <- c("person", "group")
# Create a list of 2 elements, persons and groups.
lst <- lapply(tag, function(x) { myDF[grepl(x, colnames(myDF))] })
# Extract everything before the underscore "_" in the column names
ext <- lapply(seq_along(lst), function(x) {
stringi::stri_extract(colnames(lst[[x]]), regex = "^[^_]+(?=_)") })
# Find the common elements between the two
int <- intersect(ext[[1]], ext[[2]])
# Create a new list with only the matching subset
match_list <- lapply(lst, function(x) { select(x, matches(paste(int, collapse = "|"))) })
# Replace all NA values in 'person' by the corresponding values in 'group'
res <- mapply(function(x, y) { replace(x, is.na(x), y[is.na(x)]) },
match_list[[1]], match_list[[2]])
# Assign the result back to the original data.frame
myDF[, colnames(res)] <- res
这应该忽略不匹配的 person/group 对并且只替换匹配的变量
这是一个使用 data.table 中的 set 的选项，它会就地替换
library(data.table)
#convert the data.frame to data.table
setDT(myDF)
#get the column name of 'group' and 'person' columns
nm1 <- grep("group", names(myDF), value = TRUE)
nm2 <- grep("person", names(myDF), value = TRUE)
#loop through the sequence of 'nm1'
for(j in seq_along(nm1)){
#set the elements in the row that are NA for each 'period' column
#with the corresponding row from 'group' column specified in the "value"
set(myDF, i = which(is.na(myDF[[nm2[j]]])), j = nm2[j],
value = myDF[[nm1[j]]][is.na(myDF[[nm2[j]]])])
}
myDF
# dates names var1_group var2_group var1_person var2_person
#1: 2016-01-01 John Doe NA NA NA NA
#2: 2016-01-02 John Doe 0.2875775 0.95683335 0.8895393 0.9630242
#3: 2016-01-03 John Doe 0.7883051 0.45333416 0.6928034 0.9022990
#4: 2016-01-04 John Doe 0.4089769 0.67757064 0.6405068 0.6907053
#5: 2016-01-05 John Doe 0.8830174 0.57263340 0.9942698 0.7954674
#6: 2016-01-06 Jane Smith 0.9404673 0.10292468 0.9404673 0.1029247
#7: 2016-01-07 Jane Smith 0.0455565 0.89982497 0.7085305 0.4777960
#8: 2016-01-08 Jane Smith 0.5281055 0.24608773 0.5440660 0.7584595
#9: 2016-01-09 Jane Smith 0.8924190 0.04205953 0.5941420 0.2164079
#10:2016-01-10 Jane Smith 0.5514350 0.32792072 0.2891597 0.3181810