本文整理汇总了Java中us.codecraft.webmagic.Request.getUrl方法的典型用法代码示例。如果您正苦于以下问题：Java Request.getUrl方法的具体用法？Java Request.getUrl怎么用？Java Request.getUrl使用的例子？那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类us.codecraft.webmagic.Request的用法示例。
在下文中一共展示了Request.getUrl方法的3个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。
示例1: covertRequest
import us.codecraft.webmagic.Request; //导入方法依赖的package包/类
public static Seed covertRequest(Request request) {
if (StringUtils.isNotEmpty(request.getMethod()) && !StringUtils.equalsIgnoreCase(request.getMethod(), "get")) {
log.warn("vscrawler can not support webmagic get method,this request {} will be ignore", request.getUrl());
return null;
}
Seed seed = new Seed(request.getUrl());
seed.setExt(Maps.transformEntries(request.getExtras(), new Maps.EntryTransformer<String, Object, String>() {
@Override
public String transformEntry(String key, Object value) {
if (value instanceof String) {
return (String) value;
}
return JSONObject.toJSONString(value);
}
}));
return seed;
}
示例2: getPage
import us.codecraft.webmagic.Request; //导入方法依赖的package包/类
protected String getPage(Request request) {
try {
String url = request.getUrl();
Runtime runtime = Runtime.getRuntime();
Process process = runtime.exec(phantomJsCommand + " " + crawlJsPath + " " + url);
InputStream is = process.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
StringBuffer stringBuffer = new StringBuffer();
String line;
while ((line = br.readLine()) != null) {
stringBuffer.append(line).append("\n");
}
return stringBuffer.toString();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
示例3: getUrl
import us.codecraft.webmagic.Request; //导入方法依赖的package包/类
protected String getUrl(Request request) {
return request.getUrl();
}