本文整理匯總了Python中gensim.corpora.TextCorpus方法的典型用法代碼示例。如果您正苦於以下問題：Python corpora.TextCorpus方法的具體用法？Python corpora.TextCorpus怎麽用？Python corpora.TextCorpus使用的例子？那麽, 這裏精選的方法代碼示例或許可以為您提供幫助。您也可以進一步了解該方法所在類gensim.corpora的用法示例。
在下文中一共展示了corpora.TextCorpus方法的3個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Python代碼示例。
示例1: test_save_load_ability
# 需要導入模塊: from gensim import corpora [as 別名]
# 或者: from gensim.corpora import TextCorpus [as 別名]
def test_save_load_ability(self):
"""
Make sure we can save and load (un/pickle) TextCorpus objects (as long
as the underlying input isn't a file-like object; we cannot pickle those).
"""
# construct corpus from file
corpusname = datapath('miIslita.cor')
miislita = CorpusMiislita(corpusname)
# pickle to disk
tmpf = get_tmpfile('tc_test.cpickle')
miislita.save(tmpf)
miislita2 = CorpusMiislita.load(tmpf)
self.assertEqual(len(miislita), len(miislita2))
self.assertEqual(miislita.dictionary.token2id, miislita2.dictionary.token2id) 示例2: test_textcorpus
# 需要導入模塊: from gensim import corpora [as 別名]
# 或者: from gensim.corpora import TextCorpus [as 別名]
def test_textcorpus(self):
"""Make sure TextCorpus can be serialized to disk. """
# construct corpus from file
miislita = CorpusMiislita(datapath('head500.noblanks.cor.bz2'))
# make sure serializing works
ftmp = get_tmpfile('test_textcorpus.mm')
corpora.MmCorpus.save_corpus(ftmp, miislita)
self.assertTrue(os.path.exists(ftmp))
# make sure deserializing gives the same result
miislita2 = corpora.MmCorpus(ftmp)
self.assertEqual(list(miislita), list(miislita2)) 示例3: __init__
# 需要導入模塊: from gensim import corpora [as 別名]
# 或者: from gensim.corpora import TextCorpus [as 別名]
def __init__(self, corpusPath, tokenRegexStr):
if tokenRegexStr is None:
tokenRegexStr = GensimTermiteCorpusReader.DEFAULT_TOKEN_REGEX
self.tokenRegexStr = tokenRegexStr
self.tokenRegex = re.compile(self.tokenRegexStr)
self.docIds = []
corpora.TextCorpus.__init__(self, corpusPath)