Python vocab.freqs方法代碼示例

本文整理匯總了Python中torchtext.vocab.freqs方法的典型用法代碼示例。如果您正苦於以下問題：Python vocab.freqs方法的具體用法？Python vocab.freqs怎麽用？Python vocab.freqs使用的例子？那麽, 這裏精選的方法代碼示例或許可以為您提供幫助。您也可以進一步了解該方法所在類torchtext.vocab的用法示例。

在下文中一共展示了vocab.freqs方法的7個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Python代碼示例。

示例1: merge_vocabs

# 需要導入模塊: from torchtext import vocab [as 別名]
# 或者: from torchtext.vocab import freqs [as 別名]
def merge_vocabs(vocabs, vocab_size=None):
    """
    Merge individual vocabularies (assumed to be generated from disjoint
    documents) into a larger vocabulary.

    Args:
        vocabs: `torchtext.vocab.Vocab` vocabularies to be merged
        vocab_size: `int` the final vocabulary size. `None` for no limit.
    Return:
        `torchtext.vocab.Vocab`
    """
    merged = sum([vocab.freqs for vocab in vocabs], Counter())
    return torchtext.vocab.Vocab(merged,
                                 specials=[UNK_WORD, PAD_WORD,
                                           BOS_WORD, EOS_WORD],
                                 max_size=vocab_size)

開發者ID:xiadingZ，項目名稱:video-caption-openNMT.pytorch，代碼行數:18，代碼來源:IO.py

示例2: merge_vocabs

# 需要導入模塊: from torchtext import vocab [as 別名]
# 或者: from torchtext.vocab import freqs [as 別名]
def merge_vocabs(vocabs, min_freq=0, vocab_size=None):
    """
    Merge individual vocabularies (assumed to be generated from disjoint
    documents) into a larger vocabulary.

    Args:
        vocabs: `torchtext.vocab.Vocab` vocabularies to be merged
        vocab_size: `int` the final vocabulary size. `None` for no limit.
    Return:
        `torchtext.vocab.Vocab`
    """
    merged = Counter()
    for vocab in vocabs:
        merged += filter_counter(vocab.freqs, min_freq)
    return torchtext.vocab.Vocab(merged,
                                 specials=list(special_token_list),
                                 max_size=vocab_size, min_freq=min_freq)

開發者ID:donglixp，項目名稱:coarse2fine，代碼行數:19，代碼來源:IO.py

示例3: merge_vocabs

# 需要導入模塊: from torchtext import vocab [as 別名]
# 或者: from torchtext.vocab import freqs [as 別名]
def merge_vocabs(vocabs, vocab_size=None):
    """
    Merge individual vocabularies (assumed to be generated from disjoint
    documents) into a larger vocabulary.

    Args:
        vocabs: `torchtext.vocab.Vocab` vocabularies to be merged
        vocab_size: `int` the final vocabulary size. `None` for no limit.
    Return:
        `torchtext.vocab.Vocab`
    """
    merged = Counter()
    for vocab in vocabs:
        merged += vocab.freqs
    return torchtext.vocab.Vocab(merged,
                                 specials=list(special_token_list),
                                 max_size=vocab_size)

開發者ID:donglixp，項目名稱:coarse2fine，代碼行數:19，代碼來源:IO.py

示例4: merge_vocabs

# 需要導入模塊: from torchtext import vocab [as 別名]
# 或者: from torchtext.vocab import freqs [as 別名]
def merge_vocabs(vocabs, vocab_size=None):
    """
    Merge individual vocabularies (assumed to be generated from disjoint
    documents) into a larger vocabulary.

    Args:
        vocabs: `torchtext.vocab.Vocab` vocabularies to be merged
        vocab_size: `int` the final vocabulary size. `None` for no limit.
    Return:
        `torchtext.vocab.Vocab`
    """
    merged = sum([vocab.freqs for vocab in vocabs], Counter())
    return torchtext.vocab.Vocab(merged,
                                 specials=[PAD_WORD, BOS_WORD, EOS_WORD],
                                 max_size=vocab_size)

開發者ID:matthewmackay，項目名稱:reversible-rnn，代碼行數:17，代碼來源:IO.py

示例5: merge_vocabs

# 需要導入模塊: from torchtext import vocab [as 別名]
# 或者: from torchtext.vocab import freqs [as 別名]
def merge_vocabs(vocabs, vocab_size=None):
    """
    Merge individual vocabularies (assumed to be generated from disjoint
    documents) into a larger vocabulary.

    Args:
        vocabs: `torchtext.vocab.Vocab` vocabularies to be merged
        vocab_size: `int` the final vocabulary size. `None` for no limit.
    Return:
        `torchtext.vocab.Vocab`
    """
    merged = sum([vocab.freqs for vocab in vocabs], Counter())
    return torchtext.vocab.Vocab(merged,
                                 specials=list(special_token_list),
                                 max_size=vocab_size)

開發者ID:prezaei85，項目名稱:nl2sql，代碼行數:17，代碼來源:IO.py

示例6: merge_vocabs

# 需要導入模塊: from torchtext import vocab [as 別名]
# 或者: from torchtext.vocab import freqs [as 別名]
def merge_vocabs(vocabs, vocab_size=None):
    """
    Merge individual vocabularies (assumed to be generated from disjoint
    documents) into a larger vocabulary.

    Args:
        vocabs: `torchtext.vocab.Vocab` vocabularies to be merged
        vocab_size: `int` the final vocabulary size. `None` for no limit.
    Return:
        `torchtext.vocab.Vocab`
    """
    merged = Counter(chain(*[vocab.freqs for vocab in vocabs]))
    return torchtext.vocab.Vocab(merged,
                                 specials=[PAD_WORD, BOS_WORD, EOS_WORD],
                                 max_size=vocab_size)

開發者ID:antspy，項目名稱:quantized_distillation，代碼行數:17，代碼來源:IO.py

示例7: filter_counter

# 需要導入模塊: from torchtext import vocab [as 別名]
# 或者: from torchtext.vocab import freqs [as 別名]
def filter_counter(freqs, min_freq):
    cnt = Counter()
    for k, v in freqs.items():
        if (min_freq is None) or (v >= min_freq):
            cnt[k] = v
    return cnt

開發者ID:donglixp，項目名稱:coarse2fine，代碼行數:8，代碼來源:IO.py

注：本文中的torchtext.vocab.freqs方法示例由純淨天空整理自Github/MSDocs等開源代碼及文檔管理平台，相關代碼片段篩選自各路編程大神貢獻的開源項目，源碼版權歸原作者所有，傳播和使用請參考對應項目的License；未經允許，請勿轉載。