如何用第一个解决方案初始化第二个手套模型？
How to initialize second glove model with solution from first?
我正在尝试实施有关 How to align two GloVe models in text2vec? 问题的解决方案之一。我不明白 GlobalVectors$new(..., init = list(w_i, w_j) 处输入的正确值是什么。如何确保 w_i 和 w_j 的值正确？
这是一个最小的可重现示例。首先，准备一些语料库进行比较，取自 quanteda 教程。我正在使用 dfm_match(all_words) 来尝试确保所有单词都出现在每组中，但这似乎没有达到预期的效果。
library(quanteda)
# from https://quanteda.io/articles/pkgdown/replication/text2vec.html
# get a list of all words in all documents
all_words <-
data_corpus_inaugural %>%
tokens(remove_punct = TRUE,
remove_symbols = TRUE,
remove_numbers = TRUE) %>%
types()
# should expect this mean features in each set
length(all_words)
# these are our three sets that we want to compare, we want to project the
# change in a few key words on a fixed background of other words
corpus_1 <- data_corpus_inaugural[1:19]
corpus_2 <- data_corpus_inaugural[20:39]
corpus_3 <- data_corpus_inaugural[40:58]
my_tokens1 <- texts(corpus_1) %>%
char_tolower() %>%
tokens(remove_punct = TRUE,
remove_symbols = TRUE,
remove_numbers = TRUE)
my_tokens2 <- texts(corpus_2) %>%
char_tolower() %>%
tokens(remove_punct = TRUE,
remove_symbols = TRUE,
remove_numbers = TRUE)
my_tokens3 <- texts(corpus_3) %>%
char_tolower() %>%
tokens(remove_punct = TRUE,
remove_symbols = TRUE,
remove_numbers = TRUE)
my_feats1 <-
dfm(my_tokens1, verbose = TRUE) %>%
dfm_trim(min_termfreq = 5) %>%
dfm_match(all_words) %>%
featnames()
my_feats2 <-
dfm(my_tokens2, verbose = TRUE) %>%
dfm_trim(min_termfreq = 5) %>%
dfm_match(all_words) %>%
featnames()
my_feats3 <-
dfm(my_tokens3, verbose = TRUE) %>%
dfm_trim(min_termfreq = 5) %>%
dfm_match(all_words) %>%
featnames()
# leave the pads so that non-adjacent words will not become adjacent
my_toks1_2 <- tokens_select(my_tokens1, my_feats1, padding = TRUE)
my_toks2_2 <- tokens_select(my_tokens2, my_feats2, padding = TRUE)
my_toks3_2 <- tokens_select(my_tokens3, my_feats3, padding = TRUE)
# Construct the feature co-occurrence matrix
my_fcm1 <- fcm(my_toks1_2, context = "window", tri = TRUE)
my_fcm2 <- fcm(my_toks2_2, context = "window", tri = TRUE)
my_fcm3 <- fcm(my_toks3_2, context = "window", tri = TRUE)
在上述步骤中的某个地方，我相信我需要确保每个集合的 fcm 具有所有集合的所有单词以使矩阵维度相同，但我不确定如何完成那。
现在为第一组拟合词嵌入模型：
library("text2vec")
glove1 <- GlobalVectors$new(rank = 50,
x_max = 10)
my_main1 <- glove1$fit_transform(my_fcm1,
n_iter = 10,
convergence_tol = 0.01,
n_threads = 8)
my_context1 <- glove1$components
word_vectors1 <- my_main1 + t(my_context1)
这就是我卡住的地方，我想用第一个模型初始化第二个模型，这样坐标系就可以在第一个模型和第二个模型之间进行比较。我read认为w_i和w_j是主词和上下文词，b_i和b_j是偏见。我在我的第一个模型对象中找到了那些输出，但我收到错误：
glove2 <- GlobalVectors$new(rank = 50,
x_max = 10,
init = list(w_i = glove1$.__enclos_env__$private$w_i,
b_i = glove1$.__enclos_env__$private$b_i,
w_j = glove1$.__enclos_env__$private$w_j,
b_j = glove1$.__enclos_env__$private$b_j))
my_main2 <- glove2$fit_transform(my_fcm2,
n_iter = 10,
convergence_tol = 0.01,
n_threads = 8)
错误是Error in glove2$fit_transform(my_fcm2, n_iter = 10, convergence_tol = 0.01, :
init values provided in the constructor don't match expected dimensions from the input matrix
假设我已在第一个模型中正确识别 w_i 等，我如何确保它们的尺寸正确？
这是我的会话信息：
sessionInfo()
R version 3.6.0 (2019-04-26)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS 10.15.2
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] text2vec_0.6 quanteda_2.0.0
loaded via a namespace (and not attached):
[1] Rcpp_1.0.4 pillar_1.4.3 compiler_3.6.0 tools_3.6.0 stopwords_1.0
[6] digest_0.6.25 packrat_0.5.0 lifecycle_0.2.0 tibble_3.0.0 gtable_0.3.0
[11] lattice_0.20-40 pkgconfig_2.0.3 rlang_0.4.5 Matrix_1.2-18 fastmatch_1.1-0
[16] cli_2.0.2 rstudioapi_0.11 mlapi_0.1.0 parallel_3.6.0 RhpcBLASctl_0.20-17
[21] dplyr_0.8.5 vctrs_0.2.4 grid_3.6.0 tidyselect_1.0.0.9000 glue_1.3.2
[26] data.table_1.12.8 R6_2.4.1 fansi_0.4.1 lgr_0.3.4 ggplot2_3.3.0
[31] purrr_0.3.3 magrittr_1.5 scales_1.1.0 ellipsis_0.3.0 assertthat_0.2.1
[36] float_0.2-3 rsparse_0.4.0 colorspace_1.4-1 stringi_1.4.6 RcppParallel_5.0.0
[41] munsell_0.5.0 crayon_1.3.4.9000
这是一个工作示例。有关详细信息，请参阅 ?rsparse::GloVe 文档。
library(rsparse)
data("movielens100k")
x = crossprod(sign(movielens100k))
model = GloVe$new(rank = 10, x_max = 5)
w_i = model$fit_transform(x = x, n_iter = 5, n_threads = 1)
w_j = model$components
init = list(w_i = t(w_i), model$bias_i, w_j = w_j, b_j = model$bias_j)
model2 = GloVe$new(rank = 10, x_max = 10, init = init)
w_i2 = model2$fit_transform(x)
我正在尝试实施有关 How to align two GloVe models in text2vec? 问题的解决方案之一。我不明白 GlobalVectors$new(..., init = list(w_i, w_j) 处输入的正确值是什么。如何确保 w_i 和 w_j 的值正确？
这是一个最小的可重现示例。首先，准备一些语料库进行比较，取自 quanteda 教程。我正在使用 dfm_match(all_words) 来尝试确保所有单词都出现在每组中，但这似乎没有达到预期的效果。
library(quanteda)
# from https://quanteda.io/articles/pkgdown/replication/text2vec.html
# get a list of all words in all documents
all_words <-
data_corpus_inaugural %>%
tokens(remove_punct = TRUE,
remove_symbols = TRUE,
remove_numbers = TRUE) %>%
types()
# should expect this mean features in each set
length(all_words)
# these are our three sets that we want to compare, we want to project the
# change in a few key words on a fixed background of other words
corpus_1 <- data_corpus_inaugural[1:19]
corpus_2 <- data_corpus_inaugural[20:39]
corpus_3 <- data_corpus_inaugural[40:58]
my_tokens1 <- texts(corpus_1) %>%
char_tolower() %>%
tokens(remove_punct = TRUE,
remove_symbols = TRUE,
remove_numbers = TRUE)
my_tokens2 <- texts(corpus_2) %>%
char_tolower() %>%
tokens(remove_punct = TRUE,
remove_symbols = TRUE,
remove_numbers = TRUE)
my_tokens3 <- texts(corpus_3) %>%
char_tolower() %>%
tokens(remove_punct = TRUE,
remove_symbols = TRUE,
remove_numbers = TRUE)
my_feats1 <-
dfm(my_tokens1, verbose = TRUE) %>%
dfm_trim(min_termfreq = 5) %>%
dfm_match(all_words) %>%
featnames()
my_feats2 <-
dfm(my_tokens2, verbose = TRUE) %>%
dfm_trim(min_termfreq = 5) %>%
dfm_match(all_words) %>%
featnames()
my_feats3 <-
dfm(my_tokens3, verbose = TRUE) %>%
dfm_trim(min_termfreq = 5) %>%
dfm_match(all_words) %>%
featnames()
# leave the pads so that non-adjacent words will not become adjacent
my_toks1_2 <- tokens_select(my_tokens1, my_feats1, padding = TRUE)
my_toks2_2 <- tokens_select(my_tokens2, my_feats2, padding = TRUE)
my_toks3_2 <- tokens_select(my_tokens3, my_feats3, padding = TRUE)
# Construct the feature co-occurrence matrix
my_fcm1 <- fcm(my_toks1_2, context = "window", tri = TRUE)
my_fcm2 <- fcm(my_toks2_2, context = "window", tri = TRUE)
my_fcm3 <- fcm(my_toks3_2, context = "window", tri = TRUE)
在上述步骤中的某个地方，我相信我需要确保每个集合的 fcm 具有所有集合的所有单词以使矩阵维度相同，但我不确定如何完成那。
现在为第一组拟合词嵌入模型：
library("text2vec")
glove1 <- GlobalVectors$new(rank = 50,
x_max = 10)
my_main1 <- glove1$fit_transform(my_fcm1,
n_iter = 10,
convergence_tol = 0.01,
n_threads = 8)
my_context1 <- glove1$components
word_vectors1 <- my_main1 + t(my_context1)
这就是我卡住的地方，我想用第一个模型初始化第二个模型，这样坐标系就可以在第一个模型和第二个模型之间进行比较。我read认为w_i和w_j是主词和上下文词，b_i和b_j是偏见。我在我的第一个模型对象中找到了那些输出，但我收到错误：
glove2 <- GlobalVectors$new(rank = 50,
x_max = 10,
init = list(w_i = glove1$.__enclos_env__$private$w_i,
b_i = glove1$.__enclos_env__$private$b_i,
w_j = glove1$.__enclos_env__$private$w_j,
b_j = glove1$.__enclos_env__$private$b_j))
my_main2 <- glove2$fit_transform(my_fcm2,
n_iter = 10,
convergence_tol = 0.01,
n_threads = 8)
错误是Error in glove2$fit_transform(my_fcm2, n_iter = 10, convergence_tol = 0.01, :
init values provided in the constructor don't match expected dimensions from the input matrix
假设我已在第一个模型中正确识别 w_i 等，我如何确保它们的尺寸正确？
这是我的会话信息：
sessionInfo()
R version 3.6.0 (2019-04-26)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS 10.15.2
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] text2vec_0.6 quanteda_2.0.0
loaded via a namespace (and not attached):
[1] Rcpp_1.0.4 pillar_1.4.3 compiler_3.6.0 tools_3.6.0 stopwords_1.0
[6] digest_0.6.25 packrat_0.5.0 lifecycle_0.2.0 tibble_3.0.0 gtable_0.3.0
[11] lattice_0.20-40 pkgconfig_2.0.3 rlang_0.4.5 Matrix_1.2-18 fastmatch_1.1-0
[16] cli_2.0.2 rstudioapi_0.11 mlapi_0.1.0 parallel_3.6.0 RhpcBLASctl_0.20-17
[21] dplyr_0.8.5 vctrs_0.2.4 grid_3.6.0 tidyselect_1.0.0.9000 glue_1.3.2
[26] data.table_1.12.8 R6_2.4.1 fansi_0.4.1 lgr_0.3.4 ggplot2_3.3.0
[31] purrr_0.3.3 magrittr_1.5 scales_1.1.0 ellipsis_0.3.0 assertthat_0.2.1
[36] float_0.2-3 rsparse_0.4.0 colorspace_1.4-1 stringi_1.4.6 RcppParallel_5.0.0
[41] munsell_0.5.0 crayon_1.3.4.9000
这是一个工作示例。有关详细信息，请参阅 ?rsparse::GloVe 文档。
library(rsparse)
data("movielens100k")
x = crossprod(sign(movielens100k))
model = GloVe$new(rank = 10, x_max = 5)
w_i = model$fit_transform(x = x, n_iter = 5, n_threads = 1)
w_j = model$components
init = list(w_i = t(w_i), model$bias_i, w_j = w_j, b_j = model$bias_j)
model2 = GloVe$new(rank = 10, x_max = 10, init = init)
w_i2 = model2$fit_transform(x)