Java StopFilter類代碼示例

本文整理匯總了Java中org.apache.lucene.analysis.core.StopFilter類的典型用法代碼示例。如果您正苦於以下問題：Java StopFilter類的具體用法？Java StopFilter怎麽用？Java StopFilter使用的例子？那麽, 這裏精選的類代碼示例或許可以為您提供幫助。

StopFilter類屬於org.apache.lucene.analysis.core包，在下文中一共展示了StopFilter類的15個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Java代碼示例。

示例1: getWordSet

import org.apache.lucene.analysis.core.StopFilter; //導入依賴的package包/類
/**
 * Returns as {@link CharArraySet} from wordFiles, which
 * can be a comma-separated list of filenames
 */
protected final CharArraySet getWordSet(ResourceLoader loader,
    String wordFiles, boolean ignoreCase) throws IOException {
  assureMatchVersion();
  List<String> files = splitFileNames(wordFiles);
  CharArraySet words = null;
  if (files.size() > 0) {
    // default stopwords list has 35 or so words, but maybe don't make it that
    // big to start
    words = new CharArraySet(luceneMatchVersion,
        files.size() * 10, ignoreCase);
    for (String file : files) {
      List<String> wlist = getLines(loader, file.trim());
      words.addAll(StopFilter.makeStopSet(luceneMatchVersion, wlist,
          ignoreCase));
    }
  }
  return words;
}

開發者ID:lamsfoundation，項目名稱:lams，代碼行數:23，代碼來源:AbstractAnalysisFactory.java

示例2: createComponents

import org.apache.lucene.analysis.core.StopFilter; //導入依賴的package包/類
/**
 * Creates a token stream that tokenizes the given string into token terms
 * (aka words).
 * 
 * @param fieldName
 *            the name of the field to tokenize (currently ignored).
 * @param reader
 *            reader (e.g. charfilter) of the original text. can be null.
 * @param text
 *            the string to tokenize
 * @return a new token stream
 */
public TokenStreamComponents createComponents(String fieldName, Reader reader, String text) {
  // Ideally the Analyzer superclass should have a method with the same signature, 
  // with a default impl that simply delegates to the StringReader flavour. 
  if (reader == null) 
    reader = new FastStringReader(text);
  
  if (pattern == NON_WORD_PATTERN) { // fast path
    return new TokenStreamComponents(new FastStringTokenizer(reader, true, toLowerCase, stopWords));
  } else if (pattern == WHITESPACE_PATTERN) { // fast path
    return new TokenStreamComponents(new FastStringTokenizer(reader, false, toLowerCase, stopWords));
  }

  Tokenizer tokenizer = new PatternTokenizer(reader, pattern, toLowerCase);
  TokenStream result = (stopWords != null) ? new StopFilter(matchVersion, tokenizer, stopWords) : tokenizer;
  return new TokenStreamComponents(tokenizer, result);
}

開發者ID:lamsfoundation，項目名稱:lams，代碼行數:29，代碼來源:PatternAnalyzer.java

示例3: testEndingHole

import org.apache.lucene.analysis.core.StopFilter; //導入依賴的package包/類
public void testEndingHole() throws Exception {
  // Just deletes "of"
  Analyzer a = new Analyzer() {
      @Override
      public TokenStreamComponents createComponents(String field, Reader reader) {
        Tokenizer tokenizer = new MockTokenizer(reader);
        CharArraySet stopSet = StopFilter.makeStopSet("of");
        return new TokenStreamComponents(tokenizer, new StopFilter(tokenizer, stopSet));
      }
    };

  Iterable<Input> keys = AnalyzingSuggesterTest.shuffle(
      new Input("wizard of oz", 50)
  );
  FreeTextSuggester sug = new FreeTextSuggester(a, a, 3, (byte) 0x20);
  sug.build(new InputArrayIterator(keys));
  assertEquals("wizard _ oz/1.00",
               toString(sug.lookup("wizard of", 10)));

  // Falls back to unigram model, with backoff 0.4 times
  // prop 0.5:
  assertEquals("oz/0.20",
               toString(sug.lookup("wizard o", 10)));
}

開發者ID:europeana，項目名稱:search，代碼行數:25，代碼來源:TestFreeTextSuggester.java

示例4: testTwoEndingHoles

import org.apache.lucene.analysis.core.StopFilter; //導入依賴的package包/類
public void testTwoEndingHoles() throws Exception {
  // Just deletes "of"
  Analyzer a = new Analyzer() {
      @Override
      public TokenStreamComponents createComponents(String field, Reader reader) {
        Tokenizer tokenizer = new MockTokenizer(reader);
        CharArraySet stopSet = StopFilter.makeStopSet("of");
        return new TokenStreamComponents(tokenizer, new StopFilter(tokenizer, stopSet));
      }
    };

  Iterable<Input> keys = AnalyzingSuggesterTest.shuffle(
      new Input("wizard of of oz", 50)
  );
  FreeTextSuggester sug = new FreeTextSuggester(a, a, 3, (byte) 0x20);
  sug.build(new InputArrayIterator(keys));
  assertEquals("",
               toString(sug.lookup("wizard of of", 10)));
}

開發者ID:europeana，項目名稱:search，代碼行數:20，代碼來源:TestFreeTextSuggester.java