本文整理汇总了Java中org.eclipse.jface.text.rules.ITokenScanner.nextToken方法的典型用法代码示例。如果您正苦于以下问题：Java ITokenScanner.nextToken方法的具体用法？Java ITokenScanner.nextToken怎么用？Java ITokenScanner.nextToken使用的例子？那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类org.eclipse.jface.text.rules.ITokenScanner的用法示例。
在下文中一共展示了ITokenScanner.nextToken方法的4个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。
示例1: configure
import org.eclipse.jface.text.rules.ITokenScanner; //导入方法依赖的package包/类
public static void configure(TextLayout textLayout) {
String text = textLayout.getText();
Document doc = new Document(text);
ITokenScanner scanner = getRecipeScanner(doc);
scanner.setRange(doc, 0, doc.getLength());
IToken token;
while ((token = scanner.nextToken()) != Token.EOF) {
int offset = scanner.getTokenOffset();
int length = scanner.getTokenLength();
Object data = token.getData();
if (data != null && data instanceof TextStyle) {
TextStyle textStyle = (TextStyle) data;
textLayout.setStyle(textStyle, offset, offset + length - 1);
}
}
}
示例2: configure
import org.eclipse.jface.text.rules.ITokenScanner; //导入方法依赖的package包/类
public static void configure(TextLayout textLayout) {
String text = textLayout.getText();
Document doc = new Document(text);
ITokenScanner scanner = getRecipeScanner(doc);
scanner.setRange(doc, 0, doc.getLength());
IToken token;
while ((token = scanner.nextToken()) != Token.EOF) {
int offset = scanner.getTokenOffset();
int length = scanner.getTokenLength();
Object data = token.getData();
if (data != null && data instanceof TextStyle) {
TextStyle textStyle = (TextStyle) data;
textLayout.setStyle(textStyle, offset, offset + length - 1);
}
}
scanner = null;
doc = null;
}
示例3: scan
import org.eclipse.jface.text.rules.ITokenScanner; //导入方法依赖的package包/类
public static String scan(ITokenScanner scanner, IDocument document) {
scanner.setRange(document, 0, document.getLength());
ArrayList<String> found = new ArrayList<String>();
FastStringBuffer buf = new FastStringBuffer();
IToken token = scanner.nextToken();
while (!token.isEOF()) {
Object data = token.getData();
if (data != null) {
buf.clear();
buf.append(data.toString()).append(":");
buf.append(scanner.getTokenOffset()).append(":");
buf.append(scanner.getTokenLength());
found.add(buf.toString());
} else {
buf.clear();
buf.append("null").append(":");
buf.append(scanner.getTokenOffset()).append(":");
buf.append(scanner.getTokenLength());
found.add(buf.toString());
}
token = scanner.nextToken();
}
return listToExpected(found);
}
示例4: index
import org.eclipse.jface.text.rules.ITokenScanner; //导入方法依赖的package包/类
/**
* We index based on what we want to search later on!
*
* We have to index giving the path for the file (workspace-relative path).
*
* The project is not expected to be passed because the idea is having one index
* for each project.
*
* The scanner and the mapper work together: the scanner generates the tokens
* and the mapper maps the token from the scanner to the mapping used for indexing.
*/
public void index(Path filepath, long modifiedTime, ITokenScanner tokenScanner, IFields mapper)
throws IOException {
if (this.writer == null) {
return;
}
IContentsScanner contentsScanner = (IContentsScanner) tokenScanner;
Document doc = createDocument(filepath, modifiedTime, null);
FastStringBuffer buf = new FastStringBuffer();
IToken nextToken = tokenScanner.nextToken();
while (!nextToken.isEOF()) {
if (!nextToken.isUndefined() && !nextToken.isWhitespace()) {
int offset = tokenScanner.getTokenOffset();
int length = tokenScanner.getTokenLength();
contentsScanner.getContents(offset, length, buf.clear());
String fieldName = mapper.getTokenFieldName(nextToken);
if (fieldName != null) {
//Note: TextField should be analyzed/normalized in Analyzer.createComponents(String)
doc.add(new TextField(fieldName, buf.toString(), Field.Store.NO));
}
}
nextToken = tokenScanner.nextToken();
}
this.writer.addDocument(doc);
}