本文整理汇总了Java中nl.surfsara.warcutils.WarcInputFormat类的典型用法代码示例。如果您正苦于以下问题：Java WarcInputFormat类的具体用法？Java WarcInputFormat怎么用？Java WarcInputFormat使用的例子？那么, 这里精选的类代码示例或许可以为您提供帮助。
WarcInputFormat类属于nl.surfsara.warcutils包，在下文中一共展示了WarcInputFormat类的4个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。
示例1: run
import nl.surfsara.warcutils.WarcInputFormat; //导入依赖的package包/类
@Override
public int run(String[] args) throws Exception {
Configuration conf = this.getConf();
Job job = Job.getInstance(conf, "Torrent Web");
job.setJarByClass(TorrentWeb.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
job.setMapperClass(TorrentWebExtracter.class);
job.setReducerClass(TorrentWebReducer.class);
job.setInputFormatClass(WarcInputFormat.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
// Execute job and return status
return job.waitForCompletion(true) ? 0 : 1;
}
示例2: run
import nl.surfsara.warcutils.WarcInputFormat; //导入依赖的package包/类
@Override
public int run(String[] args) throws Exception {
Configuration conf = this.getConf();
Job job = Job.getInstance(conf, "Name entities");
job.setJarByClass(NER.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
job.setMapperClass(NERMapper.class);
job.setReducerClass(Reducer.class);
job.setInputFormatClass(WarcInputFormat.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
// Execute job and return status
return job.waitForCompletion(true) ? 0 : 1;
}
示例3: run
import nl.surfsara.warcutils.WarcInputFormat; //导入依赖的package包/类
@Override
public int run(String[] args) throws Exception {
Configuration conf = this.getConf();
Job job = Job.getInstance(conf, "Extract server type");
job.setJarByClass(ServerType.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
job.setMapperClass(ServerTypeExtracter.class);
job.setReducerClass(LongSumReducer.class);
job.setInputFormatClass(WarcInputFormat.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(LongWritable.class);
// Execute job and return status
return job.waitForCompletion(true) ? 0 : 1;
}
示例4: getInputFormat
import nl.surfsara.warcutils.WarcInputFormat; //导入依赖的package包/类
@Override
public InputFormat<LongWritable, WarcRecord> getInputFormat() throws IOException {
return new WarcInputFormat();
}