當前位置: 首頁>>代碼示例>>Java>>正文

Java TokenFilterFactory類代碼示例

本文整理匯總了Java中org.elasticsearch.index.analysis.TokenFilterFactory類的典型用法代碼示例。如果您正苦於以下問題：Java TokenFilterFactory類的具體用法？Java TokenFilterFactory怎麽用？Java TokenFilterFactory使用的例子？那麽, 這裏精選的類代碼示例或許可以為您提供幫助。


TokenFilterFactory類屬於org.elasticsearch.index.analysis包，在下文中一共展示了TokenFilterFactory類的15個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Java代碼示例。

示例1: getTokenFilterFactory

import org.elasticsearch.index.analysis.TokenFilterFactory; //導入依賴的package包/類
public synchronized TokenFilterFactory getTokenFilterFactory(final Version version) {
TokenFilterFactory factory = cache.get(version);
if (factory == null) {
    final String finalName = name().toLowerCase(Locale.ROOT);
    if (isMultiTermAware()) {
        factory = new MultiTermAwareTokenFilterFactory() {
            @Override
            public String name() {
                return finalName;
            }

            @Override
            public TokenStream create(TokenStream tokenStream) {
                return PreBuiltTokenFilters.this.create(tokenStream, version);
            }

            @Override
            public Object getMultiTermComponent() {
                return this;
            }
        };
    } else {
        factory = new TokenFilterFactory() {
開發者ID:justor，項目名稱:elasticsearch_my，代碼行數:24，代碼來源:PreBuiltTokenFilters.java

示例2: getShingleFilterFactory

import org.elasticsearch.index.analysis.TokenFilterFactory; //導入依賴的package包/類
private static ShingleTokenFilterFactory.Factory getShingleFilterFactory(Analyzer analyzer) {
    if (analyzer instanceof NamedAnalyzer) {
        analyzer = ((NamedAnalyzer)analyzer).analyzer();
    }
    if (analyzer instanceof CustomAnalyzer) {
        final CustomAnalyzer a = (CustomAnalyzer) analyzer;
        final TokenFilterFactory[] tokenFilters = a.tokenFilters();
        for (TokenFilterFactory tokenFilterFactory : tokenFilters) {
            if (tokenFilterFactory instanceof ShingleTokenFilterFactory) {
                return ((ShingleTokenFilterFactory)tokenFilterFactory).getInnerFactory();
            } else if (tokenFilterFactory instanceof ShingleTokenFilterFactory.Factory) {
                return (ShingleTokenFilterFactory.Factory) tokenFilterFactory;
            }
        }
    }
    return null;
}
開發者ID:justor，項目名稱:elasticsearch_my，代碼行數:18，代碼來源:PhraseSuggestionBuilder.java

示例3: containsBrokenAnalysis

import org.elasticsearch.index.analysis.TokenFilterFactory; //導入依賴的package包/類
private static boolean containsBrokenAnalysis(Analyzer analyzer) {
    // TODO maybe we need a getter on Namedanalyzer that tells if this uses broken Analysis
    if (analyzer instanceof NamedAnalyzer) {
        analyzer = ((NamedAnalyzer) analyzer).analyzer();
    }
    if (analyzer instanceof CustomAnalyzer) {
        final CustomAnalyzer a = (CustomAnalyzer) analyzer;
        TokenFilterFactory[] tokenFilters = a.tokenFilters();
        for (TokenFilterFactory tokenFilterFactory : tokenFilters) {
            if (tokenFilterFactory instanceof WordDelimiterTokenFilterFactory
                    || tokenFilterFactory instanceof EdgeNGramTokenFilterFactory) {
                return true;
            }
        }
    }
    return false;
}
開發者ID:justor，項目名稱:elasticsearch_my，代碼行數:18，代碼來源:FragmentBuilderHelper.java

示例4: getShingleFilterFactory

import org.elasticsearch.index.analysis.TokenFilterFactory; //導入依賴的package包/類
public static ShingleTokenFilterFactory.Factory getShingleFilterFactory(Analyzer analyzer) {
    if (analyzer instanceof NamedAnalyzer) {
        analyzer = ((NamedAnalyzer)analyzer).analyzer();
    }
    if (analyzer instanceof CustomAnalyzer) {
        final CustomAnalyzer a = (CustomAnalyzer) analyzer;
        final TokenFilterFactory[] tokenFilters = a.tokenFilters();
        for (TokenFilterFactory tokenFilterFactory : tokenFilters) {
            if (tokenFilterFactory instanceof ShingleTokenFilterFactory) {
                return ((ShingleTokenFilterFactory)tokenFilterFactory).getInnerFactory();
            } else if (tokenFilterFactory instanceof ShingleTokenFilterFactory.Factory) {
                return (ShingleTokenFilterFactory.Factory) tokenFilterFactory;
            }
        }
    }
    return null;
}
開發者ID:baidu，項目名稱:Elasticsearch，代碼行數:18，代碼來源:SuggestUtils.java

示例5: testTokenizerFilter

import org.elasticsearch.index.analysis.TokenFilterFactory; //導入依賴的package包/類
@Test
public void testTokenizerFilter() throws IOException {

    String source = "eins zwei drei";

    String[] expected = {
            "d800000000001214",
            "4a00000000000000",
            "c0000000000a100"
    };
    AnalysisService analysisService = MapperTestUtils.analysisService("eudex_analysis.json");
    TokenFilterFactory tokenFilterFactory = analysisService.tokenFilter("my_phonetic");
    Tokenizer tokenizer = analysisService.tokenizer("my_phonetic").create();
    tokenizer.setReader(new StringReader(source));
    assertSimpleTSOutput(tokenFilterFactory.create(tokenizer), expected);
}
開發者ID:jprante，項目名稱:elasticsearch-analysis-phonetic-eudex，代碼行數:17，代碼來源:EudexAnalyzerTests.java

示例6: testOne

import org.elasticsearch.index.analysis.TokenFilterFactory; //導入依賴的package包/類
@Test
public void testOne() throws IOException {

    String source = "Das ist ein Bindestrich-Wort.";

    String[] expected = {
            "Das",
            "ist",
            "ein",
            "Bindestrich-Wort",
            "BindestrichWort",
            "Wort",
            "Bindestrich"
    };
    String resource = "org/xbib/elasticsearch/index/analysis/hyphen/hyphen_tokenizer.json";
    Tokenizer tokenizer = tokenizerFactory(resource, "my_hyphen_tokenizer").create();
    tokenizer.setReader(new StringReader(source));
    TokenFilterFactory tokenFilter = tokenFilterFactory(resource, "hyphen");
    TokenStream tokenStream = tokenFilter.create(tokenizer);
    assertSimpleTSOutput(tokenStream, expected);
}
開發者ID:jprante，項目名稱:elasticsearch-analysis-hyphen，代碼行數:22，代碼來源:HyphenTokenizerTests.java

示例7: testTwo

import org.elasticsearch.index.analysis.TokenFilterFactory; //導入依賴的package包/類
@Test
public void testTwo() throws IOException {

    String source = "Das E-Book muss dringend zum Buchbinder.";

    String[] expected = {
            "Das",
            "E-Book",
            "EBook",
            "Book",
            "muss",
            "dringend",
            "zum",
            "Buchbinder"
    };
    String resource = "org/xbib/elasticsearch/index/analysis/hyphen/hyphen_tokenizer.json";
    Tokenizer tokenizer = tokenizerFactory(resource, "my_hyphen_tokenizer").create();
    tokenizer.setReader(new StringReader(source));
    TokenFilterFactory tokenFilter = tokenFilterFactory(resource, "hyphen");
    assertSimpleTSOutput(tokenFilter.create(tokenizer), expected);
}
開發者ID:jprante，項目名稱:elasticsearch-analysis-hyphen，代碼行數:22，代碼來源:HyphenTokenizerTests.java

示例8: testFour

import org.elasticsearch.index.analysis.TokenFilterFactory; //導入依賴的package包/類
@Test
public void testFour() throws IOException {

    String source = "So wird's was: das Elasticsearch-Buch erscheint beim O'Reilly-Verlag.";

    String[] expected = {
            "So",
            "wird's",
            "was",
            "das",
            "Elasticsearch-Buch",
            "ElasticsearchBuch",
            "Buch",
            "Elasticsearch",
            "erscheint",
            "beim",
            "O'Reilly-Verlag"
    };
    String resource = "org/xbib/elasticsearch/index/analysis/hyphen/hyphen_tokenizer.json";
    Tokenizer tokenizer = tokenizerFactory(resource, "my_hyphen_tokenizer").create();
    tokenizer.setReader(new StringReader(source));
    TokenFilterFactory tokenFilter = tokenFilterFactory(resource, "hyphen");
    assertSimpleTSOutput(tokenFilter.create(tokenizer), expected);
}
開發者ID:jprante，項目名稱:elasticsearch-analysis-hyphen，代碼行數:25，代碼來源:HyphenTokenizerTests.java

示例9: testSix

import org.elasticsearch.index.analysis.TokenFilterFactory; //導入依賴的package包/類
@Test
public void testSix() throws IOException {

    String source = "E-Book";

    String[] expected = {
            "E-Book",
            "EBook",
            "Book"
    };

    String resource = "org/xbib/elasticsearch/index/analysis/hyphen/hyphen_tokenizer.json";
    Tokenizer tokenizer = tokenizerFactory(resource, "my_hyphen_tokenizer").create();
    tokenizer.setReader(new StringReader(source));
    TokenFilterFactory tokenFilter = tokenFilterFactory(resource, "hyphen");
    assertSimpleTSOutput(tokenFilter.create(tokenizer), expected);
}
開發者ID:jprante，項目名稱:elasticsearch-analysis-hyphen，代碼行數:18，代碼來源:HyphenTokenizerTests.java

示例10: testSeven

import org.elasticsearch.index.analysis.TokenFilterFactory; //導入依賴的package包/類
@Test
public void testSeven() throws IOException {
    String source = "Procter & Gamble ist Procter&Gamble. Schwarz - weiss ist schwarz-weiss";

    String[] expected = {
            "Procter",
            "Gamble",
            "ist",
            "Procter&Gamble",
            "Schwarz",
            "weiss",
            "ist",
            "schwarz-weiss",
            "schwarzweiss",
            "weiss",
            "schwarz"
    };

    String resource = "org/xbib/elasticsearch/index/analysis/hyphen/hyphen_tokenizer.json";
    Tokenizer tokenizer = tokenizerFactory(resource, "my_hyphen_tokenizer").create();
    tokenizer.setReader(new StringReader(source));
    TokenFilterFactory tokenFilter = tokenFilterFactory(resource, "hyphen");
    assertSimpleTSOutput(tokenFilter.create(tokenizer), expected);
}
開發者ID:jprante，項目名稱:elasticsearch-analysis-hyphen，代碼行數:25，代碼來源:HyphenTokenizerTests.java

示例11: testOpenNLPAnalysis

import org.elasticsearch.index.analysis.TokenFilterFactory; //導入依賴的package包/類
@Test
public void testOpenNLPAnalysis() throws Exception {
    AnalysisService analysisService = MapperTestUtils.analysisService(getResource());
    TokenizerFactory tokenizerFactory = analysisService.tokenizer("opennlp");
    MatcherAssert.assertThat(tokenizerFactory, instanceOf(OpenNLPTokenizerFactory.class));
    this.tokenizerFactory = (OpenNLPTokenizerFactory) tokenizerFactory;
    TokenFilterFactory filterFactory = analysisService.tokenFilter("opennlp");
    MatcherAssert.assertThat(filterFactory, instanceOf(OpenNLPTokenFilterFactory.class));
    this.filterFactory = (OpenNLPTokenFilterFactory) filterFactory;

    TokenFilterFactory filterpayloadsfilterFactory = analysisService.tokenFilter("filterpayloads");
    MatcherAssert.assertThat(filterpayloadsfilterFactory, instanceOf(FilterPayloadsTokenFilterFactory.class));
    TokenFilterFactory strippayloadsfilterFactory = analysisService.tokenFilter("strippayloads");
    MatcherAssert.assertThat(strippayloadsfilterFactory, instanceOf(StripPayloadsTokenFilterFactory.class));

    analyze();
}
開發者ID:jprante，項目名稱:elasticsearch-analysis-opennlp，代碼行數:18，代碼來源:BaseOpenNLPAnalysis.java

示例12: JapaneseBaseFormFilterFactory

import org.elasticsearch.index.analysis.TokenFilterFactory; //導入依賴的package包/類
public JapaneseBaseFormFilterFactory(final IndexSettings indexSettings, final Environment env, final String name,
        final Settings settings, final FessAnalysisService fessAnalysisService) {
    super(indexSettings, name, settings);

    for (final String factoryClass : FACTORIES) {
        final Class<?> tokenFilterFactoryClass = fessAnalysisService.loadClass(factoryClass);
        if (tokenFilterFactoryClass != null) {
            if (logger.isInfoEnabled()) {
                logger.info("{} is found.", factoryClass);
            }
            tokenFilterFactory = AccessController.doPrivileged((PrivilegedAction<TokenFilterFactory>) () -> {
                try {
                    final Constructor<?> constructor = tokenFilterFactoryClass.getConstructor(IndexSettings.class, Environment.class,
                            String.class, Settings.class);
                    return (TokenFilterFactory) constructor.newInstance(indexSettings, env, name, settings);
                } catch (final Exception e) {
                    throw new ElasticsearchException("Failed to load " + factoryClass, e);
                }
            });
            break;
        } else if (logger.isDebugEnabled()) {
            logger.debug("{} is not found.", factoryClass);
        }
    }
}
開發者ID:codelibs，項目名稱:elasticsearch-analysis-fess，代碼行數:26，代碼來源:JapaneseBaseFormFilterFactory.java

示例13: JapanesePartOfSpeechFilterFactory

import org.elasticsearch.index.analysis.TokenFilterFactory; //導入依賴的package包/類
public JapanesePartOfSpeechFilterFactory(final IndexSettings indexSettings, final Environment env, final String name,
        final Settings settings, final FessAnalysisService fessAnalysisService) {
    super(indexSettings, name, settings);

    for (final String factoryClass : FACTORIES) {
        final Class<?> TokenFilterFactoryClass = fessAnalysisService.loadClass(factoryClass);
        if (TokenFilterFactoryClass != null) {
            if (logger.isInfoEnabled()) {
                logger.info("{} is found.", factoryClass);
            }
            tokenFilterFactory = AccessController.doPrivileged((PrivilegedAction<TokenFilterFactory>) () -> {
                try {
                    final Constructor<?> constructor = TokenFilterFactoryClass.getConstructor(IndexSettings.class, Environment.class,
                            String.class, Settings.class);
                    return (TokenFilterFactory) constructor.newInstance(indexSettings, env, name, settings);
                } catch (final Exception e) {
                    throw new ElasticsearchException("Failed to load " + factoryClass, e);
                }
            });
            break;
        } else if (logger.isDebugEnabled()) {
            logger.debug("{} is not found.", factoryClass);
        }
    }
}
開發者ID:codelibs，項目名稱:elasticsearch-analysis-fess，代碼行數:26，代碼來源:JapanesePartOfSpeechFilterFactory.java

示例14: JapaneseKatakanaStemmerFactory

import org.elasticsearch.index.analysis.TokenFilterFactory; //導入依賴的package包/類
public JapaneseKatakanaStemmerFactory(final IndexSettings indexSettings, final Environment env, final String name,
        final Settings settings, final FessAnalysisService fessAnalysisService) {
    super(indexSettings, name, settings);

    for (final String factoryClass : FACTORIES) {
        final Class<?> tokenizerFactoryClass = fessAnalysisService.loadClass(factoryClass);
        if (tokenizerFactoryClass != null) {
            if (logger.isInfoEnabled()) {
                logger.info("{} is found.", factoryClass);
            }
            tokenFilterFactory = AccessController.doPrivileged((PrivilegedAction<TokenFilterFactory>) () -> {
                try {
                    final Constructor<?> constructor =
                            tokenizerFactoryClass.getConstructor(IndexSettings.class, Environment.class, String.class, Settings.class);
                    return (TokenFilterFactory) constructor.newInstance(indexSettings, env, name, settings);
                } catch (final Exception e) {
                    throw new ElasticsearchException("Failed to load " + factoryClass, e);
                }

            });
            break;
        } else if (logger.isDebugEnabled()) {
            logger.debug("{} is not found.", factoryClass);
        }
    }
}
開發者ID:codelibs，項目名稱:elasticsearch-analysis-fess，代碼行數:27，代碼來源:JapaneseKatakanaStemmerFactory.java

示例15: JapaneseReadingFormFilterFactory

import org.elasticsearch.index.analysis.TokenFilterFactory; //導入依賴的package包/類
public JapaneseReadingFormFilterFactory(final IndexSettings indexSettings, final Environment env, final String name,
        final Settings settings, final FessAnalysisService fessAnalysisService) {
    super(indexSettings, name, settings);

    for (final String factoryClass : FACTORIES) {
        final Class<?> tokenFilterFactoryClass = fessAnalysisService.loadClass(factoryClass);
        if (tokenFilterFactoryClass != null) {
            if (logger.isInfoEnabled()) {
                logger.info("{} is found.", factoryClass);
            }
            tokenFilterFactory = AccessController.doPrivileged((PrivilegedAction<TokenFilterFactory>) () -> {
                try {
                    final Constructor<?> constructor = tokenFilterFactoryClass.getConstructor(IndexSettings.class, Environment.class,
                            String.class, Settings.class);
                    return (TokenFilterFactory) constructor.newInstance(indexSettings, env, name, settings);
                } catch (final Exception e) {
                    throw new ElasticsearchException("Failed to load " + factoryClass, e);
                }
            });
            break;
        } else if (logger.isDebugEnabled()) {
            logger.debug("{} is not found.", factoryClass);
        }
    }
}
開發者ID:codelibs，項目名稱:elasticsearch-analysis-fess，代碼行數:26，代碼來源:JapaneseReadingFormFilterFactory.java


注：本文中的org.elasticsearch.index.analysis.TokenFilterFactory類示例由純淨天空整理自Github/MSDocs等開源代碼及文檔管理平台，相關代碼片段篩選自各路編程大神貢獻的開源項目，源碼版權歸原作者所有，傳播和使用請參考對應項目的License；未經允許，請勿轉載。