本文整理汇总了Java中org.galagosearch.tupleflow.NullProcessor类的典型用法代码示例。如果您正苦于以下问题：Java NullProcessor类的具体用法？Java NullProcessor怎么用？Java NullProcessor使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
NullProcessor类属于org.galagosearch.tupleflow包，在下文中一共展示了NullProcessor类的1个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。
示例1: testTokenize
import org.galagosearch.tupleflow.NullProcessor; //导入依赖的package包/类
/**
* Test of tokenize method, of class galago.parse.TagTokenizer.
*/
public void testTokenize() throws IOException, IncompatibleProcessorException {
TagTokenizer tokenizer = new TagTokenizer();
Document document = new Document();
tokenizer.setProcessor(new NullProcessor(Document.class));
document.text = "<html> <a href=\"http://www.yahoo.com\">Yahoo</a> this is some text " +
"<title>title text</title> \n" +
"That's all folks!" +
"</html>" +
"ciir.cs.umass.edu " +
"m.b.a.";
tokenizer.process(document);
// first, check tokens
String[] tokens = {"yahoo", "this", "is", "some", "text", "title", "text", "thats", "all", "folks", "ciir", "cs", "umass", "edu", "mba"};
assertEquals("Token length", tokens.length, document.terms.size());
for (int i = 0; i < tokens.length; i++) {
assertEquals("Token text", tokens[i], document.terms.get(i));
}
// then, check tags
assertEquals(document.tags.size(), 3);
// html tag
Tag html = document.tags.get(0);
assertEquals(html.name, "html");
assertEquals(html.attributes.size(), 0);
assertEquals(html.begin, 0);
assertEquals(html.end, 10);
// a tag
Tag a = document.tags.get(1);
assertEquals(a.name, "a");
assertEquals(a.attributes.size(), 1);
assertEquals(a.attributes.get("href"), "http://www.yahoo.com");
assertEquals(a.begin, 0);
assertEquals(a.end, 1);
// title tag
Tag title = document.tags.get(2);
assertEquals(title.name, "title");
assertEquals(title.attributes.size(), 0);
assertEquals(title.begin, 5);
assertEquals(title.end, 7);
}