本文整理汇总了Python中gensim.corpora.dictionary.Dictionary.token2id[word]方法的典型用法代码示例。如果您正苦于以下问题：Python Dictionary.token2id[word]方法的具体用法？Python Dictionary.token2id[word]怎么用？Python Dictionary.token2id[word]使用的例子？那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类gensim.corpora.dictionary.Dictionary的用法示例。
在下文中一共展示了Dictionary.token2id[word]方法的1个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Python代码示例。
示例1: loadDictionary
# 需要导入模块: from gensim.corpora.dictionary import Dictionary [as 别名]
# 或者: from gensim.corpora.dictionary.Dictionary import token2id[word] [as 别名]
def loadDictionary(fname, mapping_only=True):
"""
Load previously stored mapping between words and their ids.
The result can be used as the `id2word` parameter for input to transformations.
"""
if mapping_only:
result = {}
for lineNo, line in enumerate(open(fname)):
cols = line[:-1].split('\t')
if len(cols) == 2:
wordId, word = cols
elif len(cols) == 3:
wordId, word, dfs = cols
else:
raise ValueError("invalid line in dictionary file %s: %s" % (fname, line.strip()))
result[int(wordId)] = word # dfs not used
else:
result = Dictionary()
for lineNo, line in enumerate(open(fname)):
cols = line[:-1].split('\t')
if len(cols) == 3:
wordId, word, dfs = cols
else:
raise ValueError("invalid line in dictionary file %s: %s" % (fname, line.strip()))
wordId = int(wordId)
result.token2id[word] = wordId
result.dfs[wordId] = int(dfs)
return result