本文整理汇总了C++中StringTokenizer::setSeparators方法的典型用法代码示例。如果您正苦于以下问题：C++ StringTokenizer::setSeparators方法的具体用法？C++ StringTokenizer::setSeparators怎么用？C++ StringTokenizer::setSeparators使用的例子？那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类StringTokenizer的用法示例。
在下文中一共展示了StringTokenizer::setSeparators方法的1个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的C++代码示例。
示例1: tokenize
//...................................................... tokenize ...
bool FileTokenizer::tokenize() {
ifstream currentFile; //file to be tokenized
string inputLine; //line that will be read from the file
StringTokenizer * tkn; //String tokenizer
int i; //index variable
string token; //token currently extracted
bool res = true;
tkn = new StringTokenizer();
tkn->setSeparators(currentSeparators);
// [1] Initialize position and open the file to be tokenized
tokenPos = 0;
tokens.clear();
currentFile.open(currentFileName.c_str(), std::ifstream::in);
if (currentFile.fail()) {
std::cerr << "Can't open file [" << currentFileName << "]" << std::endl;
res = false;
} else {
// [2] Read input while the file has not been finished, and tokenize
while (currentFile.good()) {
//Get next line
std::getline(currentFile, inputLine);
#ifndef _WIN32
inputLine = StringUtilities::removeAtEnd(inputLine, __CRLF);
#endif
//Tokenize
tkn->setLine(inputLine);
tkn->tokenize(true);
/*
* Add tokens to the token buffer.
*
* Tokens are added while they are "correct" tokens. A correct token is a
* token that is not empty, does not contain "#", and does not appear
* after a "#" character in the line.
*/
i = 0;
while (i < tkn->getNumTokens()) {
token = tkn->getToken(i);
if (token == "") {
//The token is blank. Do not add but check next token
i = i + 1;
} else {
//The token is not blank, and has no comments inside it
tokens.push_back(token);
i = i + 1;
}
}
}
}
currentFile.close();
//Free memory of the ATMStringTokenizer object
delete tkn;
return res;
}