本文整理汇总了Python中nltk.tokenize.punkt.PunktSentenceTokenizer.train方法的典型用法代码示例。如果您正苦于以下问题：Python PunktSentenceTokenizer.train方法的具体用法？Python PunktSentenceTokenizer.train怎么用？Python PunktSentenceTokenizer.train使用的例子？那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类nltk.tokenize.punkt.PunktSentenceTokenizer的用法示例。
在下文中一共展示了PunktSentenceTokenizer.train方法的1个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Python代码示例。
示例1: train_sentence_splitter
# 需要导入模块: from nltk.tokenize.punkt import PunktSentenceTokenizer [as 别名]
# 或者: from nltk.tokenize.punkt.PunktSentenceTokenizer import train [as 别名]
def train_sentence_splitter(lang):
"""
Train an NLTK punkt tokenizer for sentence splitting.
http://www.nltk.org
"""
# Read in trainings corpus
plain_file = "%s.plain" % (lang)
text = codecs.open(plain_file, "Ur", "utf-8").read()
# Train tokenizer
tokenizer = PunktSentenceTokenizer()
tokenizer.train(text)
# Dump pickled tokenizer
pickle_file = "%s.pickle" % (lang)
out = open(pickle_file, "wb")
pickle.dump(tokenizer, out)
out.close()