當前位置: 首頁>>代碼示例>>Java>>正文

Java StandardFilter類代碼示例

本文整理匯總了Java中org.apache.lucene.analysis.standard.StandardFilter類的典型用法代碼示例。如果您正苦於以下問題：Java StandardFilter類的具體用法？Java StandardFilter怎麽用？Java StandardFilter使用的例子？那麽, 這裏精選的類代碼示例或許可以為您提供幫助。


StandardFilter類屬於org.apache.lucene.analysis.standard包，在下文中一共展示了StandardFilter類的15個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Java代碼示例。

示例1: reusableTokenStream

import org.apache.lucene.analysis.standard.StandardFilter; //導入依賴的package包/類
@Override
public TokenStream reusableTokenStream(String fieldName, Reader reader) throws IOException {
    class SavedStreams {
        StandardTokenizer tokenStream;
        TokenStream filteredTokenStream;
    }

    SavedStreams streams = (SavedStreams) getPreviousTokenStream();
    if (streams == null) {
        streams = new SavedStreams();
        setPreviousTokenStream(streams);
        streams.tokenStream = new StandardTokenizer(LUCENE_VERSION, reader);
        streams.filteredTokenStream = new StandardFilter(streams.tokenStream);
        streams.filteredTokenStream = new LowerCaseFilter(streams.filteredTokenStream);
        streams.filteredTokenStream = new StopFilter(true, streams.filteredTokenStream, STOP_WORDS_SET);
        streams.filteredTokenStream = new ASCIIFoldingFilter(streams.filteredTokenStream);
    } else {
        streams.tokenStream.reset(reader);
    }
    streams.tokenStream.setMaxTokenLength(DEFAULT_MAX_TOKEN_LENGTH);

    return streams.filteredTokenStream;
}
開發者ID:airsonic，項目名稱:airsonic，代碼行數:24，代碼來源:SearchService.java

示例2: testStandardTokenizer

import org.apache.lucene.analysis.standard.StandardFilter; //導入依賴的package包/類
public void testStandardTokenizer() throws Exception {

        String source = "우리나라라면에서부터 일본라면이 파생되었잖니?";
        source = "너는 너는 다시 내게 돌아 올거야. school is a good place 嗬嗬大笑 嗬嗬大笑";

        long start = System.currentTimeMillis();

        StandardAnalyzer analyzer = new StandardAnalyzer(Version.LUCENE_36);
        TokenStream stream = analyzer.tokenStream("s", new StringReader(source));
        TokenStream tok = new StandardFilter(Version.LUCENE_36, stream);

        while (tok.incrementToken()) {
            CharTermAttribute termAttr = stream.getAttribute(CharTermAttribute.class);
            OffsetAttribute offAttr = stream.getAttribute(OffsetAttribute.class);
            PositionIncrementAttribute posAttr = stream.getAttribute(PositionIncrementAttribute.class);
            TypeAttribute typeAttr = stream.getAttribute(TypeAttribute.class);

            System.out.println(new String(termAttr.buffer(), 0, termAttr.length()));
        }

        System.out.println((System.currentTimeMillis() - start) + "ms");
    }
開發者ID:debop，項目名稱:lucene-korean，代碼行數:23，代碼來源:KoreanAnalyzerTest.java

示例3: tokenize

import org.apache.lucene.analysis.standard.StandardFilter; //導入依賴的package包/類
public List<String> tokenize(Analyzer analyzer, String data) {
    List<String> terms = Lists.newArrayList();
    try {
        TokenStream tokens = new StandardFilter(analyzer.tokenStream(null, new StringReader(data)));
        tokens.reset();

        while (tokens.incrementToken()) {
            CharTermAttribute termAttribute = tokens.getAttribute(CharTermAttribute.class);
            String term = trimToNull(termAttribute.toString());
            if (term != null) {
                terms.add(term);
            }
        }

        tokens.end();
        tokens.close();
    } catch (IOException ioe) {
        LOG.warn("Unable to tokenize data. cause: {}", new Object[] { ioe.getMessage() }, ioe);
    }
    return terms;
}
開發者ID:jivesoftware，項目名稱:miru，代碼行數:22，代碼來源:TermTokenizer.java

示例4: createComponents

import org.apache.lucene.analysis.standard.StandardFilter; //導入依賴的package包/類
@Override
	protected TokenStreamComponents createComponents(String fieldName) {
		
//		Tokenizer _IKTokenizer = new IKTokenizer(in , this.useSmart());
		final Tokenizer src= new IKTokenizer(new StringReader(""), this.useSmart());
        TokenStream tok = new StandardFilter(src);
        return new TokenStreamComponents(src, tok) {
            @Override
            protected void setReader(final Reader reader) {
                super.setReader(reader);
            }
        };
	}
開發者ID:TFdream，項目名稱:lucene-analyzer-ik，代碼行數:14，代碼來源:IKAnalyzer.java

示例5: createComponents

import org.apache.lucene.analysis.standard.StandardFilter; //導入依賴的package包/類
@Override
protected TokenStreamComponents createComponents(String fieldName, Reader reader) {
    final AutocompleteTokenizer tokenizer = new AutocompleteTokenizer(reader);

    TokenFilter filter = new StandardFilter(tokenizer);

    return new TokenStreamComponents(tokenizer, filter);
}
開發者ID:gncloud，項目名稱:fastcatsearch3，代碼行數:9，代碼來源:AutocompleteAnalyzer.java

示例6: tokenStream

import org.apache.lucene.analysis.standard.StandardFilter; //導入依賴的package包/類
/**
 * @param fieldName ignored param
 * @param reader contains data to parse
 * @return TokenStream of ngrams
 */
public TokenStream tokenStream(String fieldName, Reader reader) {
    return new NGramTokenFilter(
            new LowerCaseFilter(
                new StandardFilter(
                    new StandardTokenizer(reader))), min_ngram, max_ngram);
}
開發者ID:spacewalkproject，項目名稱:spacewalk，代碼行數:12，代碼來源:NGramAnalyzer.java

示例7: createComponents

import org.apache.lucene.analysis.standard.StandardFilter; //導入依賴的package包/類
@Override
protected TokenStreamComponents createComponents(String fieldName, Reader reader) {

  Version matchVersion = Version.LUCENE_45;

  final Tokenizer source = new WhitespaceTokenizer(matchVersion, reader);
  TokenStream result = new StandardFilter(matchVersion, source);
  result = new DiarienummerTokenFilter(result);
  return new TokenStreamComponents(source, result);

}
開發者ID:Helsingborg，項目名稱:solarie，代碼行數:12，代碼來源:DiarienummerAnalyzer.java

示例8: createComponents

import org.apache.lucene.analysis.standard.StandardFilter; //導入依賴的package包/類
@Override
protected TokenStreamComponents createComponents(String fieldName, Reader reader) {
    Tokenizer source = new StandardTokenizer(LuceneConfig.USED_VERSION, reader);
    TokenStream result = new StandardFilter(LuceneConfig.USED_VERSION, source);
    TokenStreamComponents components = new TokenStreamComponents(source, result);
    return components;
}
開發者ID:Tietoarkisto，項目名稱:metka，代碼行數:8，代碼來源:DefaultAnalyzer.java

示例9: tokenStream

import org.apache.lucene.analysis.standard.StandardFilter; //導入依賴的package包/類
public TokenStream tokenStream(String fieldName, Reader reader) {
  TokenStream result = new SynonymFilter(
                        new StopFilter(true,
                          new LowerCaseFilter(
                            new StandardFilter(
                              new StandardTokenizer(
                               Version.LUCENE_41, reader))),
                          StopAnalyzer.ENGLISH_STOP_WORDS_SET),
                        engine
                       );
  return result;
}
開發者ID:xuzhikethinker，項目名稱:t4f-data，代碼行數:13，代碼來源:SynonymAnalyzer.java

示例10: create

import org.apache.lucene.analysis.standard.StandardFilter; //導入依賴的package包/類
@Override
public TokenStream create(TokenStream tokenStream) {
    return new StandardFilter(tokenStream);
}
開發者ID:justor，項目名稱:elasticsearch_my，代碼行數:5，代碼來源:StandardTokenFilterFactory.java

示例11: create

import org.apache.lucene.analysis.standard.StandardFilter; //導入依賴的package包/類
@Override
public StandardFilter create(TokenStream input) {
  return new StandardFilter(input);
}
開發者ID:lamsfoundation，項目名稱:lams，代碼行數:5，代碼來源:StandardFilterFactory.java

示例12: createComponents

import org.apache.lucene.analysis.standard.StandardFilter; //導入依賴的package包/類
@Override
protected TokenStreamComponents createComponents(String paramString) {
	Tokenizer source = new NGramTokenizer(n, n);
	TokenStream result =  new StandardFilter(source);
	return new TokenStreamComponents(source, result);
}
開發者ID:ksgwr，項目名稱:LuceneDB，代碼行數:7，代碼來源:NgramAnalyzer.java

示例13: create

import org.apache.lucene.analysis.standard.StandardFilter; //導入依賴的package包/類
@Override
public StandardFilter create(TokenStream input) {
  return new StandardFilter(luceneMatchVersion, input);
}
開發者ID:pkarmstr，項目名稱:NYBC，代碼行數:5，代碼來源:StandardFilterFactory.java

示例14: StdTermFilter

import org.apache.lucene.analysis.standard.StandardFilter; //導入依賴的package包/類
/** Construct the rewriter */
public StdTermFilter() {
  dribble = new DribbleStream();
  filter = new StandardFilter(new LowerCaseFilter(dribble));
}
開發者ID:CDLUC3，項目名稱:dash-xtf，代碼行數:6，代碼來源:StdTermFilter.java

示例15: createComponents

import org.apache.lucene.analysis.standard.StandardFilter; //導入依賴的package包/類
@Override
protected TokenStreamComponents createComponents(String fieldName, Reader reader) {

	final NGramWordTokenizer tokenizer = new NGramWordTokenizer(reader, 2, 3);

	TokenFilter filter = new StandardFilter(tokenizer);

	return new TokenStreamComponents(tokenizer, filter);
}
開發者ID:gncloud，項目名稱:fastcatsearch3，代碼行數:10，代碼來源:NGramWordAnalyzer.java


注：本文中的org.apache.lucene.analysis.standard.StandardFilter類示例由純淨天空整理自Github/MSDocs等開源代碼及文檔管理平台，相關代碼片段篩選自各路編程大神貢獻的開源項目，源碼版權歸原作者所有，傳播和使用請參考對應項目的License；未經允許，請勿轉載。