Java StopFilter类代码示例

本文整理汇总了Java中org.apache.lucene.analysis.core.StopFilter类的典型用法代码示例。如果您正苦于以下问题：Java StopFilter类的具体用法？Java StopFilter怎么用？Java StopFilter使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。

StopFilter类属于org.apache.lucene.analysis.core包，在下文中一共展示了StopFilter类的15个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: getWordSet

import org.apache.lucene.analysis.core.StopFilter; //导入依赖的package包/类
/**
 * Returns as {@link CharArraySet} from wordFiles, which
 * can be a comma-separated list of filenames
 */
protected final CharArraySet getWordSet(ResourceLoader loader,
    String wordFiles, boolean ignoreCase) throws IOException {
  assureMatchVersion();
  List<String> files = splitFileNames(wordFiles);
  CharArraySet words = null;
  if (files.size() > 0) {
    // default stopwords list has 35 or so words, but maybe don't make it that
    // big to start
    words = new CharArraySet(luceneMatchVersion,
        files.size() * 10, ignoreCase);
    for (String file : files) {
      List<String> wlist = getLines(loader, file.trim());
      words.addAll(StopFilter.makeStopSet(luceneMatchVersion, wlist,
          ignoreCase));
    }
  }
  return words;
}

开发者ID:lamsfoundation，项目名称:lams，代码行数:23，代码来源:AbstractAnalysisFactory.java

示例2: createComponents

import org.apache.lucene.analysis.core.StopFilter; //导入依赖的package包/类
/**
 * Creates a token stream that tokenizes the given string into token terms
 * (aka words).
 * 
 * @param fieldName
 *            the name of the field to tokenize (currently ignored).
 * @param reader
 *            reader (e.g. charfilter) of the original text. can be null.
 * @param text
 *            the string to tokenize
 * @return a new token stream
 */
public TokenStreamComponents createComponents(String fieldName, Reader reader, String text) {
  // Ideally the Analyzer superclass should have a method with the same signature, 
  // with a default impl that simply delegates to the StringReader flavour. 
  if (reader == null) 
    reader = new FastStringReader(text);
  
  if (pattern == NON_WORD_PATTERN) { // fast path
    return new TokenStreamComponents(new FastStringTokenizer(reader, true, toLowerCase, stopWords));
  } else if (pattern == WHITESPACE_PATTERN) { // fast path
    return new TokenStreamComponents(new FastStringTokenizer(reader, false, toLowerCase, stopWords));
  }

  Tokenizer tokenizer = new PatternTokenizer(reader, pattern, toLowerCase);
  TokenStream result = (stopWords != null) ? new StopFilter(matchVersion, tokenizer, stopWords) : tokenizer;
  return new TokenStreamComponents(tokenizer, result);
}

开发者ID:lamsfoundation，项目名称:lams，代码行数:29，代码来源:PatternAnalyzer.java

示例3: testEndingHole

import org.apache.lucene.analysis.core.StopFilter; //导入依赖的package包/类
public void testEndingHole() throws Exception {
  // Just deletes "of"
  Analyzer a = new Analyzer() {
      @Override
      public TokenStreamComponents createComponents(String field, Reader reader) {
        Tokenizer tokenizer = new MockTokenizer(reader);
        CharArraySet stopSet = StopFilter.makeStopSet("of");
        return new TokenStreamComponents(tokenizer, new StopFilter(tokenizer, stopSet));
      }
    };

  Iterable<Input> keys = AnalyzingSuggesterTest.shuffle(
      new Input("wizard of oz", 50)
  );
  FreeTextSuggester sug = new FreeTextSuggester(a, a, 3, (byte) 0x20);
  sug.build(new InputArrayIterator(keys));
  assertEquals("wizard _ oz/1.00",
               toString(sug.lookup("wizard of", 10)));

  // Falls back to unigram model, with backoff 0.4 times
  // prop 0.5:
  assertEquals("oz/0.20",
               toString(sug.lookup("wizard o", 10)));
}

开发者ID:europeana，项目名称:search，代码行数:25，代码来源:TestFreeTextSuggester.java

示例4: testTwoEndingHoles

import org.apache.lucene.analysis.core.StopFilter; //导入依赖的package包/类
public void testTwoEndingHoles() throws Exception {
  // Just deletes "of"
  Analyzer a = new Analyzer() {
      @Override
      public TokenStreamComponents createComponents(String field, Reader reader) {
        Tokenizer tokenizer = new MockTokenizer(reader);
        CharArraySet stopSet = StopFilter.makeStopSet("of");
        return new TokenStreamComponents(tokenizer, new StopFilter(tokenizer, stopSet));
      }
    };

  Iterable<Input> keys = AnalyzingSuggesterTest.shuffle(
      new Input("wizard of of oz", 50)
  );
  FreeTextSuggester sug = new FreeTextSuggester(a, a, 3, (byte) 0x20);
  sug.build(new InputArrayIterator(keys));
  assertEquals("",
               toString(sug.lookup("wizard of of", 10)));
}

开发者ID:europeana，项目名称:search，代码行数:20，代码来源:TestFreeTextSuggester.java