data.table 使用列的变量名进行分组操作，没有慢速 DT[ mean(get(colName)), by = grp]

Question

我想创建一个使用列变量名和数据变量名的函数。

这个功能是我想要的并且有效:

n <- 1e7
d <- data.table(x = 1:n, grp = sample(1:1e5, n, replace = T))
dataName = "d"
colName = "x"

# Objective :
FOO <- function(dataName = "d",
         colName = "x"){
  get(dataName)[, mean(get(colName)), by = grp]
}

问题是对每个组的 get() 评估非常耗时。在真实数据示例中，它比等效的静态名称长 14 倍。我想达到与列名是静态的相同的执行时间。

我尝试了什么：

(cl <- substitute(mean(eval(parse(text = colName))), list(colName = as.name(colName))))

microbenchmark::microbenchmark(

  # 1) works and quick but does not use variable names of columns (654ms)
  (t1 <- d[, mean(x), by = grp]),

  # 2) works but slow (1006ms)
  (t2 <- get(dataName)[, mean(get(colName)), by = grp]), # works but slow

  # 3) works but slow (4075ms)
  (t3 <- eval(parse(text = dataName))[, mean(eval(parse(text = colName))), by = grp]),

  # 4) works but very slow (37202ms)
  (t4 <- get(dataName)[, eval(cl), by = grp]),

  # 5) double dot syntax doesn't work cause I don't master it
  # (t5 <- get(dataName)[, mean(..colName), by = grp]),

  times = 10)

双点语法在这里合适吗？为什么 4) 这么慢？我从 this post where it was the best option. I adapted the double dot syntax from this post.

拿来的

非常感谢您的帮助！

Answer 1

最好将数据集名称d传递给FOO函数而不是传递字符串"d"。此外，您可以将 lapply 与 .SD 结合使用，这样您就可以从内部优化中获益，而不是使用 mean(get(colName)).

FOO2 = function(dataName=d, colName = "x") { # d instead of "d" passed to the first argument!
  dataName[, lapply(.SD, mean), by=grp, .SDcols=colName]
}

基准：`FOO` 对比 `FOO2`

set.seed(147852)
n <- 1e7
d <- data.table(x = 1:n, grp = sample(1:1e5, n, replace = T))

microbenchmark::microbenchmark(
  FOO(),
  FOO2(),
  times=5L
)

Unit: milliseconds
   expr       min        lq      mean    median        uq       max neval
  FOO() 4632.4014 4672.7781 4787.4958 4707.9023 4846.7081 5077.6893     5
 FOO2()  255.0828  267.1322  297.0389  275.4467  281.9873  405.5456     5

data.table 使用列的变量名进行分组操作，没有慢速 DT[ mean(get(colName)), by = grp]

data.table grouped operations with variable names of columns without slow DT[, mean(get(colName)), by = grp]

r

data.table

基准：FOO 对比 FOO2

基准：`FOO` 对比 `FOO2`