當前位置: 首頁>>代碼示例>>Java>>正文


Java CommonTokenStream.getTokens方法代碼示例

本文整理匯總了Java中org.antlr.v4.runtime.CommonTokenStream.getTokens方法的典型用法代碼示例。如果您正苦於以下問題:Java CommonTokenStream.getTokens方法的具體用法?Java CommonTokenStream.getTokens怎麽用?Java CommonTokenStream.getTokens使用的例子?那麽, 這裏精選的方法代碼示例或許可以為您提供幫助。您也可以進一步了解該方法所在org.antlr.v4.runtime.CommonTokenStream的用法示例。


在下文中一共展示了CommonTokenStream.getTokens方法的1個代碼示例,這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚,您的評價將有助於係統推薦出更棒的Java代碼示例。

示例1: getTokenList

import org.antlr.v4.runtime.CommonTokenStream; //導入方法依賴的package包/類
/**
 * Extract the token list from the Cypher input. Uses ANTLR classes to perform this. Some tokens
 * are excluded, such as EOF and semi colons.
 *
 * @param cyp         Cypher input as text.
 * @param DEBUG_PRINT Print out debug statements or not.
 * @return A list of tokens as deciphered by the ANTLR classes, based on the openCypher grammar.
 */
public static ArrayList<String> getTokenList(String cyp, boolean DEBUG_PRINT) {
    CypherLexer lexer = new CypherLexer(new ANTLRInputStream(cyp));

    CommonTokenStream tokens = new CommonTokenStream(lexer);
    tokens.fill();
    CypherParser parser = new CypherParser(tokens);

    // dangerous - comment out if something is going wrong.
    parser.removeErrorListeners();

    ParseTree tree = parser.cypher();
    ParseTreeWalker walker = new ParseTreeWalker();

    cypherWalker = null;
    cypherWalker = new CypherWalker();
    walker.walk(cypherWalker, tree);

    if (DEBUG_PRINT) cypherWalker.printInformation();

    ArrayList<String> tokenList = new ArrayList<>();

    for (Object t : tokens.getTokens()) {
        CommonToken tok = (CommonToken) t;
        String s = tok.getText().toLowerCase();

        // exclude some tokens from the list of tokens. This includes the EOF pointer,
        // semi-colons, and alias artifacts.
        if (!" ".equals(s) && !"<eof>".equals(s) && !";".equals(s) && !"as".equals(s) &&
                !cypherWalker.getAlias().contains(s)) {
            tokenList.add(s);
        }
    }
    return tokenList;
}
 
開發者ID:DTG-FRESCO,項目名稱:cyp2sql,代碼行數:43,代碼來源:CypherTokenizer.java


注:本文中的org.antlr.v4.runtime.CommonTokenStream.getTokens方法示例由純淨天空整理自Github/MSDocs等開源代碼及文檔管理平台,相關代碼片段篩選自各路編程大神貢獻的開源項目,源碼版權歸原作者所有,傳播和使用請參考對應項目的License;未經允許,請勿轉載。