本文整理汇总了Java中org.apache.lucene.analysis.ja.JapaneseTokenizer.Mode.EXTENDED属性的典型用法代码示例。如果您正苦于以下问题：Java Mode.EXTENDED属性的具体用法？Java Mode.EXTENDED怎么用？Java Mode.EXTENDED使用的例子？那么恭喜您, 这里精选的属性代码示例或许可以为您提供帮助。您也可以进一步了解该属性所在类org.apache.lucene.analysis.ja.JapaneseTokenizer.Mode的用法示例。
在下文中一共展示了Mode.EXTENDED属性的5个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。
示例1: tokenizationMode
@Nonnull
private static Mode tokenizationMode(@Nonnull final ObjectInspector oi)
throws UDFArgumentException {
final String arg = HiveUtils.getConstString(oi);
if (arg == null) {
return Mode.NORMAL;
}
final Mode mode;
if ("NORMAL".equalsIgnoreCase(arg)) {
mode = Mode.NORMAL;
} else if ("SEARCH".equalsIgnoreCase(arg)) {
mode = Mode.SEARCH;
} else if ("EXTENDED".equalsIgnoreCase(arg)) {
mode = Mode.EXTENDED;
} else if ("DEFAULT".equalsIgnoreCase(arg)) {
mode = JapaneseTokenizer.DEFAULT_MODE;
} else {
throw new UDFArgumentException(
"Expected NORMAL|SEARCH|EXTENDED|DEFAULT but got an unexpected mode: " + arg);
}
return mode;
}
示例2: createComponents
@Override
protected TokenStreamComponents createComponents(String fieldName, Reader reader) {
Tokenizer tokenizer = new JapaneseTokenizer(newAttributeFactory(), reader, readDict(), true, Mode.EXTENDED);
return new TokenStreamComponents(tokenizer, tokenizer);
}
示例3: createComponents
@Override
protected TokenStreamComponents createComponents(String fieldName, Reader reader) {
Tokenizer tokenizer = new JapaneseTokenizer(newAttributeFactory(), reader, null, true, Mode.EXTENDED);
return new TokenStreamComponents(tokenizer, tokenizer);
}
示例4: createComponents
@Override
protected TokenStreamComponents createComponents(String fieldName, Reader reader) {
Tokenizer tokenizer = new JapaneseTokenizer(reader, readDict(), true, Mode.EXTENDED);
return new TokenStreamComponents(tokenizer, tokenizer);
}
示例5: createComponents
@Override
protected TokenStreamComponents createComponents(String fieldName, Reader reader) {
Tokenizer tokenizer = new JapaneseTokenizer(reader, null, true, Mode.EXTENDED);
return new TokenStreamComponents(tokenizer, tokenizer);
}