本文整理匯總了Java中org.antlr.v4.runtime.CommonTokenStream.fill方法的典型用法代碼示例。如果您正苦於以下問題:Java CommonTokenStream.fill方法的具體用法?Java CommonTokenStream.fill怎麽用?Java CommonTokenStream.fill使用的例子?那麽, 這裏精選的方法代碼示例或許可以為您提供幫助。您也可以進一步了解該方法所在類org.antlr.v4.runtime.CommonTokenStream
的用法示例。
在下文中一共展示了CommonTokenStream.fill方法的1個代碼示例,這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚,您的評價將有助於係統推薦出更棒的Java代碼示例。
示例1: getTokenList
import org.antlr.v4.runtime.CommonTokenStream; //導入方法依賴的package包/類
/**
* Extract the token list from the Cypher input. Uses ANTLR classes to perform this. Some tokens
* are excluded, such as EOF and semi colons.
*
* @param cyp Cypher input as text.
* @param DEBUG_PRINT Print out debug statements or not.
* @return A list of tokens as deciphered by the ANTLR classes, based on the openCypher grammar.
*/
public static ArrayList<String> getTokenList(String cyp, boolean DEBUG_PRINT) {
CypherLexer lexer = new CypherLexer(new ANTLRInputStream(cyp));
CommonTokenStream tokens = new CommonTokenStream(lexer);
tokens.fill();
CypherParser parser = new CypherParser(tokens);
// dangerous - comment out if something is going wrong.
parser.removeErrorListeners();
ParseTree tree = parser.cypher();
ParseTreeWalker walker = new ParseTreeWalker();
cypherWalker = null;
cypherWalker = new CypherWalker();
walker.walk(cypherWalker, tree);
if (DEBUG_PRINT) cypherWalker.printInformation();
ArrayList<String> tokenList = new ArrayList<>();
for (Object t : tokens.getTokens()) {
CommonToken tok = (CommonToken) t;
String s = tok.getText().toLowerCase();
// exclude some tokens from the list of tokens. This includes the EOF pointer,
// semi-colons, and alias artifacts.
if (!" ".equals(s) && !"<eof>".equals(s) && !";".equals(s) && !"as".equals(s) &&
!cypherWalker.getAlias().contains(s)) {
tokenList.add(s);
}
}
return tokenList;
}