本文整理汇总了Python中gensim.corpora.TextCorpus方法的典型用法代码示例。如果您正苦于以下问题：Python corpora.TextCorpus方法的具体用法？Python corpora.TextCorpus怎么用？Python corpora.TextCorpus使用的例子？那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类gensim.corpora的用法示例。
在下文中一共展示了corpora.TextCorpus方法的3个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Python代码示例。
示例1: test_save_load_ability
# 需要导入模块: from gensim import corpora [as 别名]
# 或者: from gensim.corpora import TextCorpus [as 别名]
def test_save_load_ability(self):
"""
Make sure we can save and load (un/pickle) TextCorpus objects (as long
as the underlying input isn't a file-like object; we cannot pickle those).
"""
# construct corpus from file
corpusname = datapath('miIslita.cor')
miislita = CorpusMiislita(corpusname)
# pickle to disk
tmpf = get_tmpfile('tc_test.cpickle')
miislita.save(tmpf)
miislita2 = CorpusMiislita.load(tmpf)
self.assertEqual(len(miislita), len(miislita2))
self.assertEqual(miislita.dictionary.token2id, miislita2.dictionary.token2id) 示例2: test_textcorpus
# 需要导入模块: from gensim import corpora [as 别名]
# 或者: from gensim.corpora import TextCorpus [as 别名]
def test_textcorpus(self):
"""Make sure TextCorpus can be serialized to disk. """
# construct corpus from file
miislita = CorpusMiislita(datapath('head500.noblanks.cor.bz2'))
# make sure serializing works
ftmp = get_tmpfile('test_textcorpus.mm')
corpora.MmCorpus.save_corpus(ftmp, miislita)
self.assertTrue(os.path.exists(ftmp))
# make sure deserializing gives the same result
miislita2 = corpora.MmCorpus(ftmp)
self.assertEqual(list(miislita), list(miislita2)) 示例3: __init__
# 需要导入模块: from gensim import corpora [as 别名]
# 或者: from gensim.corpora import TextCorpus [as 别名]
def __init__(self, corpusPath, tokenRegexStr):
if tokenRegexStr is None:
tokenRegexStr = GensimTermiteCorpusReader.DEFAULT_TOKEN_REGEX
self.tokenRegexStr = tokenRegexStr
self.tokenRegex = re.compile(self.tokenRegexStr)
self.docIds = []
corpora.TextCorpus.__init__(self, corpusPath)