本文整理汇总了Java中org.apache.spark.SparkConf.getInt方法的典型用法代码示例。如果您正苦于以下问题：Java SparkConf.getInt方法的具体用法？Java SparkConf.getInt怎么用？Java SparkConf.getInt使用的例子？那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类org.apache.spark.SparkConf的用法示例。
在下文中一共展示了SparkConf.getInt方法的2个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。
示例1: main
import org.apache.spark.SparkConf; //导入方法依赖的package包/类
public static void main(String[] args) throws Exception {
if (args.length != 1) {
System.err.println("Usage: JavaSleep <seconds>");
System.exit(1);
}
SparkConf sparkConf = new SparkConf().setAppName("JavaSleep");
JavaSparkContext ctx = new JavaSparkContext(sparkConf);
Integer parallel = sparkConf.getInt("spark.default.parallelism", ctx.defaultParallelism());
Integer seconds = Integer.parseInt(args[0]);
Integer[] init_val = new Integer[parallel];
Arrays.fill(init_val, seconds);
JavaRDD<Integer> workload = ctx.parallelize(Arrays.asList(init_val), parallel).map(new Function<Integer, Integer>() {
@Override
public Integer call(Integer s) throws InterruptedException {
Thread.sleep(s * 1000);
return 0;
}
});
List<Integer> output = workload.collect();
ctx.stop();
}
示例2: main
import org.apache.spark.SparkConf; //导入方法依赖的package包/类
public static void main(String[] args) throws Exception {
if (args.length < 2) {
System.err.println("Usage: JavaTeraSort <HDFS_INPUT> <HDFS_OUTPUT>");
System.exit(1);
}
SparkConf sparkConf = new SparkConf().setAppName("JavaTeraSort");
JavaSparkContext ctx = new JavaSparkContext(sparkConf);
JavaRDD<String> lines = ctx.textFile(args[0], 1);
Integer parallel = sparkConf.getInt("spark.default.parallelism", ctx.defaultParallelism());
Integer reducer = Integer.parseInt(IOCommon.getProperty("hibench.default.shuffle.parallelism").get());
JavaPairRDD<String, String> words = lines.mapToPair(new PairFunction<String, String, String>() {
@Override
public Tuple2<String, String> call(String s) throws Exception {
return new Tuple2<String, String>(s.substring(0, 10), s.substring(10));
}
});
JavaPairRDD<String, String> sorted = words.sortByKey(true, reducer);
JavaRDD<String> result = sorted.map(new Function<Tuple2<String, String>, String>() {
@Override
public String call(Tuple2<String, String> e) throws Exception {
return e._1() + e._2();
}
});
result.saveAsTextFile(args[1]);
ctx.stop();
}