R textrecipes step_lda 计算代币的LDA维度估计

step_lda() 创建配方步骤的规范，该步骤将返回文本变量的 lda 维度估计值。

用法

step_lda(
  recipe,
  ...,
  role = "predictor",
  trained = FALSE,
  columns = NULL,
  lda_models = NULL,
  num_topics = 10L,
  prefix = "lda",
  keep_original_cols = FALSE,
  skip = FALSE,
  id = rand_id("lda")
)

来源

https://arxiv.org/abs/1301.3781

参数

recipe: 一个recipe 对象。该步骤将添加到此配方的操作序列中。
...: 一个或多个选择器函数用于选择受该步骤影响的变量。有关更多详细信息，请参阅recipes::selections()。
role: 对于此步骤创建的模型项，应为它们分配什么分析角色？默认情况下，该函数假定由原始变量创建的新列将用作模型中的预测变量。
trained: 指示预处理数量是否已估计的逻辑。
columns: 将由 terms 参数(最终)填充的变量名称字符串。在 recipes::prep.recipe() 训练该步骤之前，这是 NULL 。
lda_models: text2vec 包中的 WarpLDA 模型对象。如果保留为 NULL(默认值)，它将根据训练数据训练其模型。查看示例，了解如何拟合 WarpLDA 模型。
num_topics: 整数所需的潜在主题数。
prefix: 生成的列名称的前缀，默认为"lda"。
keep_original_cols: 将原始变量保留在输出中的逻辑。默认为 FALSE 。
skip: 一个合乎逻辑的。当recipes::bake.recipe() 烘焙食谱时是否应该跳过此步骤？虽然所有操作都是在 recipes::prep.recipe() 运行时烘焙的，但某些操作可能无法对新数据进行(例如处理结果变量)。使用 skip = FALSE 时应小心。
id: 该步骤特有的字符串，用于标识它。

值

recipe 的更新版本，其中新步骤添加到现有步骤(如果有)的序列中。

整理

当您tidy()此步骤时，会出现一个包含列terms(选择的选择器或变量)和num_topics(主题数)的小标题。

箱重

底层操作不允许使用案例权重。

也可以看看

来自标记的数字变量的其他步骤：step_texthash()、step_tfidf()、step_tf()、step_word_embeddings()

例子

library(recipes)
library(modeldata)
data(tate_text)

tate_rec <- recipe(~., data = tate_text) %>%
  step_tokenize(medium) %>%
  step_lda(medium)

tate_obj <- tate_rec %>%
  prep()
#> 'as(<dgTMatrix>, "dgCMatrix")' is deprecated.
#> Use 'as(., "CsparseMatrix")' instead.
#> See help("Deprecated") and help("Matrix-deprecated").

bake(tate_obj, new_data = NULL) %>%
  slice(1:2)
#> # A tibble: 2 × 14
#>      id artist          title  year lda_medium_1 lda_medium_2 lda_medium_3
#>   <dbl> <fct>           <fct> <dbl>        <dbl>        <dbl>        <dbl>
#> 1 21926 Absalon         Prop…  1990          0.7       0.0143       0.0143
#> 2 20472 Auerbach, Frank Mich…  1990          0         0            0     
#> # ℹ 7 more variables: lda_medium_4 <dbl>, lda_medium_5 <dbl>,
#> #   lda_medium_6 <dbl>, lda_medium_7 <dbl>, lda_medium_8 <dbl>,
#> #   lda_medium_9 <dbl>, lda_medium_10 <dbl>
tidy(tate_rec, number = 2)
#> # A tibble: 1 × 3
#>   terms  num_topics id       
#>   <chr>       <int> <chr>    
#> 1 medium         10 lda_UfL6S
tidy(tate_obj, number = 2)
#> # A tibble: 1 × 3
#>   terms  num_topics id       
#>   <chr>       <int> <chr>    
#> 1 medium         10 lda_UfL6S

# Changing the number of topics.
recipe(~., data = tate_text) %>%
  step_tokenize(medium, artist) %>%
  step_lda(medium, artist, num_topics = 20) %>%
  prep() %>%
  bake(new_data = NULL) %>%
  slice(1:2)
#> # A tibble: 2 × 43
#>      id title     year lda_medium_1 lda_medium_2 lda_medium_3 lda_medium_4
#>   <dbl> <fct>    <dbl>        <dbl>        <dbl>        <dbl>        <dbl>
#> 1 21926 Proposa…  1990       0.0286       0.0286            0            0
#> 2 20472 Michael   1990       0            0                 0            0
#> # ℹ 36 more variables: lda_medium_5 <dbl>, lda_medium_6 <dbl>,
#> #   lda_medium_7 <dbl>, lda_medium_8 <dbl>, lda_medium_9 <dbl>,
#> #   lda_medium_10 <dbl>, lda_medium_11 <dbl>, lda_medium_12 <dbl>,
#> #   lda_medium_13 <dbl>, lda_medium_14 <dbl>, lda_medium_15 <dbl>,
#> #   lda_medium_16 <dbl>, lda_medium_17 <dbl>, lda_medium_18 <dbl>,
#> #   lda_medium_19 <dbl>, lda_medium_20 <dbl>, lda_artist_1 <dbl>,
#> #   lda_artist_2 <dbl>, lda_artist_3 <dbl>, lda_artist_4 <dbl>, …

# Supplying A pre-trained LDA model trained using text2vec
library(text2vec)
tokens <- word_tokenizer(tolower(tate_text$medium))
it <- itoken(tokens, ids = seq_along(tate_text$medium))
v <- create_vocabulary(it)
dtm <- create_dtm(it, vocab_vectorizer(v))
lda_model <- LDA$new(n_topics = 15)

recipe(~., data = tate_text) %>%
  step_tokenize(medium, artist) %>%
  step_lda(medium, artist, lda_models = lda_model) %>%
  prep() %>%
  bake(new_data = NULL) %>%
  slice(1:2)
#> # A tibble: 2 × 33
#>      id title     year lda_medium_1 lda_medium_2 lda_medium_3 lda_medium_4
#>   <dbl> <fct>    <dbl>        <dbl>        <dbl>        <dbl>        <dbl>
#> 1 21926 Proposa…  1990       0.0143        0.129            0       0.0143
#> 2 20472 Michael   1990       0             0                0       0     
#> # ℹ 26 more variables: lda_medium_5 <dbl>, lda_medium_6 <dbl>,
#> #   lda_medium_7 <dbl>, lda_medium_8 <dbl>, lda_medium_9 <dbl>,
#> #   lda_medium_10 <dbl>, lda_medium_11 <dbl>, lda_medium_12 <dbl>,
#> #   lda_medium_13 <dbl>, lda_medium_14 <dbl>, lda_medium_15 <dbl>,
#> #   lda_artist_1 <dbl>, lda_artist_2 <dbl>, lda_artist_3 <dbl>,
#> #   lda_artist_4 <dbl>, lda_artist_5 <dbl>, lda_artist_6 <dbl>,
#> #   lda_artist_7 <dbl>, lda_artist_8 <dbl>, lda_artist_9 <dbl>, …

源代码：R/lda.R

相关用法

注：本文由纯净天空筛选整理自等大神的英文原创作品 Calculate LDA Dimension Estimates of Tokens。非经特殊声明，原始代码版权归原作者所有，本译文未经允许或授权，请勿转载或复制。