借助nltk.tokenize.StanfordTokenizer()
方法,我们可以使用以下方法从字符串或数字字符串中提取标记tokenize.StanfordTokenizer()
方法。它遵循斯坦福标准生成令牌。
用法: tokenize.StanfordTokenizer()
返回: Return the tokens from a string of characters or numbers.
范例1:
在这个例子中,我们可以通过使用tokenize.SExprTokenizer()
方法,我们能够使用斯坦福标准从字符或数字流中提取令牌。
# import StanfordTokenizer() method from nltk
from nltk.tokenize.stanford import StanfordTokenizer
# Create a reference variable for Class StanfordTokenizer
tk = StanfordTokenizer()
# Create a string input
gfg = "Geeks f o r Geeks"
# Use tokenize method
geek = tk.tokenize(gfg)
print(geek)
输出:
[‘Geeks’, ‘f’, ‘o’, ‘r’, ‘Geeks’]
范例2:
# import StanfordTokenizer() method from nltk
from nltk.tokenize.stanford import StanfordTokenizer
# Create a reference variable for Class StanfordTokenizer
tk = StanfordTokenizer()
# Create a string input
gfg = "This is your great author."
# Use tokenize method
geek = tk.tokenize(gfg)
print(geek)
输出:
[‘This’, ‘is’, ‘your’, ‘great’, ‘author’, ‘.’]
相关用法
- Python nltk.tokenizer.word_tokenize()用法及代码示例
- Python NLTK nltk.tokenize.ConditionalFreqDist()用法及代码示例
- Python nltk.tokenize.LineTokenizer用法及代码示例
- Python nltk.tokenize.SExprTokenizer()用法及代码示例
- Python nltk.tokenize.TabTokenizer()用法及代码示例
- Python nltk.tokenize.SpaceTokenizer()用法及代码示例
- Python nltk.WhitespaceTokenizer用法及代码示例
- Python nltk.TweetTokenizer()用法及代码示例
- Python nltk.tokenize.mwe()用法及代码示例
- Python NLTK tokenize.regexp()用法及代码示例
- Python NLTK tokenize.WordPunctTokenizer()用法及代码示例
注:本文由纯净天空筛选整理自Jitender_1998大神的英文原创作品 Python NLTK | nltk.tokenize.StanfordTokenizer()。非经特殊声明,原始代码版权归原作者所有,本译文未经允许或授权,请勿转载或复制。