当前位置: 首页>>代码示例>>Java>>正文

Java GiraphConfiguration.setMasterComputeClass方法代码示例

本文整理汇总了Java中org.apache.giraph.conf.GiraphConfiguration.setMasterComputeClass方法的典型用法代码示例。如果您正苦于以下问题：Java GiraphConfiguration.setMasterComputeClass方法的具体用法？Java GiraphConfiguration.setMasterComputeClass怎么用？Java GiraphConfiguration.setMasterComputeClass使用的例子？那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类org.apache.giraph.conf.GiraphConfiguration的用法示例。


在下文中一共展示了GiraphConfiguration.setMasterComputeClass方法的15个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: prepareConfiguration

import org.apache.giraph.conf.GiraphConfiguration; //导入方法依赖的package包/类
@Override
protected void prepareConfiguration(GiraphConfiguration conf,
    CommandLine cmd) {
  conf.setVertexClass(RandomMessageVertex.class);
  conf.setVertexInputFormatClass(PseudoRandomVertexInputFormat.class);
  conf.setWorkerContextClass(RandomMessageBenchmarkWorkerContext.class);
  conf.setMasterComputeClass(RandomMessageBenchmarkMasterCompute.class);
  conf.setLong(PseudoRandomInputFormatConstants.AGGREGATE_VERTICES,
      BenchmarkOption.VERTICES.getOptionLongValue(cmd));
  conf.setLong(PseudoRandomInputFormatConstants.EDGES_PER_VERTEX,
      BenchmarkOption.EDGES_PER_VERTEX.getOptionLongValue(cmd));
  conf.setInt(SUPERSTEP_COUNT,
      BenchmarkOption.SUPERSTEPS.getOptionIntValue(cmd));
  conf.setInt(RandomMessageBenchmark.NUM_BYTES_PER_MESSAGE,
      BYTES_PER_MESSAGE.getOptionIntValue(cmd));
  conf.setInt(RandomMessageBenchmark.NUM_MESSAGES_PER_EDGE,
      MESSAGES_PER_EDGE.getOptionIntValue(cmd));
  if (FLUSH_THREADS.optionTurnedOn(cmd)) {
    conf.setInt(GiraphConstants.MSG_NUM_FLUSH_THREADS,
        FLUSH_THREADS.getOptionIntValue(cmd));
  }
}
开发者ID:zfighter，项目名称:giraph-research，代码行数:23，代码来源:RandomMessageBenchmark.java

示例2: testBspMasterCompute

import org.apache.giraph.conf.GiraphConfiguration; //导入方法依赖的package包/类
/**
 * Run a sample BSP job locally and test MasterCompute.
 *
 * @throws IOException
 * @throws ClassNotFoundException
 * @throws InterruptedException
 */
@Test
public void testBspMasterCompute()
    throws IOException, InterruptedException, ClassNotFoundException {
  GiraphConfiguration conf = new GiraphConfiguration();
  conf.setVertexClass(SimpleMasterComputeVertex.class);
  conf.setVertexInputFormatClass(SimplePageRankVertexInputFormat.class);
  conf.setMasterComputeClass(
      SimpleMasterComputeVertex.SimpleMasterCompute.class);
  conf.setWorkerContextClass(
      SimpleMasterComputeVertex.SimpleMasterComputeWorkerContext.class);
  GiraphJob job = prepareJob(getCallingMethodName(), conf);
  assertTrue(job.run(true));
  if (!runningInDistributedMode()) {
    double finalSum =
        SimpleMasterComputeVertex.SimpleMasterComputeWorkerContext.getFinalSum();
    System.out.println("testBspMasterCompute: finalSum=" + finalSum);
    assertEquals(32.5, finalSum, 0d);
  }
}
开发者ID:zfighter，项目名称:giraph-research，代码行数:27，代码来源:TestBspBasic.java

示例3: prepareConfiguration

import org.apache.giraph.conf.GiraphConfiguration; //导入方法依赖的package包/类
@Override
protected void prepareConfiguration(GiraphConfiguration conf,
    CommandLine cmd) {
  conf.setComputationClass(RandomMessageComputation.class);
  conf.setVertexInputFormatClass(PseudoRandomVertexInputFormat.class);
  conf.setWorkerContextClass(RandomMessageBenchmarkWorkerContext.class);
  conf.setMasterComputeClass(RandomMessageBenchmarkMasterCompute.class);
  conf.setLong(PseudoRandomInputFormatConstants.AGGREGATE_VERTICES,
      BenchmarkOption.VERTICES.getOptionLongValue(cmd));
  conf.setLong(PseudoRandomInputFormatConstants.EDGES_PER_VERTEX,
      BenchmarkOption.EDGES_PER_VERTEX.getOptionLongValue(cmd));
  conf.setInt(SUPERSTEP_COUNT,
      BenchmarkOption.SUPERSTEPS.getOptionIntValue(cmd));
  conf.setInt(RandomMessageBenchmark.NUM_BYTES_PER_MESSAGE,
      BYTES_PER_MESSAGE.getOptionIntValue(cmd));
  conf.setInt(RandomMessageBenchmark.NUM_MESSAGES_PER_EDGE,
      MESSAGES_PER_EDGE.getOptionIntValue(cmd));
  if (FLUSH_THREADS.optionTurnedOn(cmd)) {
    conf.setInt(GiraphConstants.MSG_NUM_FLUSH_THREADS,
        FLUSH_THREADS.getOptionIntValue(cmd));
  }
}
开发者ID:renato2099，项目名称:giraph-gora，代码行数:23，代码来源:RandomMessageBenchmark.java

示例4: run

import org.apache.giraph.conf.GiraphConfiguration; //导入方法依赖的package包/类
@Override
public int run(String[] args) throws Exception {
    if (args.length != 3) {
        throw new IllegalArgumentException(
            "Syntax error: Must have 3 arguments <numbersOfWorkers> <inputLocaiton> <outputLocation>");
    }

    int numberOfWorkers = Integer.parseInt(args[0]);
    String inputLocation = args[1];
    String outputLocation = args[2];

    GiraphJob job = new GiraphJob(getConf(), getClass().getName());
    GiraphConfiguration gconf = job.getConfiguration();
    gconf.setWorkerConfiguration(numberOfWorkers, numberOfWorkers, 100.0f);

    GiraphFileInputFormat.addVertexInputPath(gconf, new Path(inputLocation));
    FileOutputFormat.setOutputPath(job.getInternalJob(), new Path(outputLocation));

    gconf.setComputationClass(ZombieComputation.class);
    gconf.setMasterComputeClass(ZombieMasterCompute.class);
    gconf.setVertexInputFormatClass(ZombieTextVertexInputFormat.class);
    gconf.setVertexOutputFormatClass(ZombieTextVertexOutputFormat.class);
    gconf.setWorkerContextClass(ZombieWorkerContext.class);

    boolean verbose = true;
    if (job.run(verbose)) {
        return 0;
    } else {
        return -1;
    }
}
开发者ID:amitchmca，项目名称:hadooparchitecturebook，代码行数:32，代码来源:ZombieBiteJob.java

示例5: getConfiguration

import org.apache.giraph.conf.GiraphConfiguration; //导入方法依赖的package包/类
private GiraphConfiguration getConfiguration() {
  GiraphConfiguration conf = new GiraphConfiguration();
  conf.setComputationClass(DiffusionComputation.class);
  conf.setMasterComputeClass(DiffusionMasterComputation.class);
  conf.setVertexInputFormatClass(DiffusionTextVertexInputFormat.class);
  conf.setVertexOutputFormatClass(DiffusionTextVertexOutputFormat.class);
  conf.setBoolean(DiffusionTextVertexOutputFormat.TEST_OUTPUT, true);
  return conf;
}
开发者ID:galpha，项目名称:giraph-didic，代码行数:10，代码来源:DiffusionComputationTest.java

示例6: getConfiguration

import org.apache.giraph.conf.GiraphConfiguration; //导入方法依赖的package包/类
private GiraphConfiguration getConfiguration() {
  GiraphConfiguration conf = new GiraphConfiguration();
  conf.setComputationClass(ARPComputation.class);
  conf.setMasterComputeClass(ARPMasterComputation.class);
  conf.setVertexInputFormatClass(ARPTextVertexInputFormat.class);
  conf.setVertexOutputFormatClass(ARPTextVertexOutputFormat.class);
  return conf;
}
开发者ID:dbs-leipzig，项目名称:giraph-algorithms，代码行数:9，代码来源:ARPComputationTest.java

示例7: getConfiguration

import org.apache.giraph.conf.GiraphConfiguration; //导入方法依赖的package包/类
private GiraphConfiguration getConfiguration() {
  GiraphConfiguration conf = new GiraphConfiguration();
  conf.setComputationClass(LPComputation.class);
  conf.setMasterComputeClass(LPMasterComputation.class);
  conf.setVertexInputFormatClass(LPTextVertexInputFormat.class);
  conf.setVertexOutputFormatClass(LPTextVertexOutputFormat.class);
  return conf;
}
开发者ID:dbs-leipzig，项目名称:giraph-algorithms，代码行数:9，代码来源:LPComputationTest.java

示例8: getConf

import org.apache.giraph.conf.GiraphConfiguration; //导入方法依赖的package包/类
public GiraphConfiguration getConf() {
    GiraphConfiguration conf = new GiraphConfiguration();
    conf.setMasterComputeClass(PageRankMasterCompute.class);
    conf.setComputationClass(PageRankComputation.class);
    conf.setVertexOutputFormatClass(InMemoryVertexOutputFormat.class);
    //conf.setInt("mapreduce.job.counters.limit", 1000);
    return conf;
}
开发者ID:Sotera，项目名称:distributed-graph-analytics，代码行数:9，代码来源:PageRankTest.java

示例9: getConf

import org.apache.giraph.conf.GiraphConfiguration; //导入方法依赖的package包/类
public GiraphConfiguration getConf() {
    GiraphConfiguration conf = new GiraphConfiguration();
    conf.setMasterComputeClass(LouvainMasterCompute.class);
    conf.setComputationClass(LouvainComputation.class);
    conf.setVertexOutputFormatClass(InMemoryVertexOutputFormat.class);
    System.setProperty("giraph.useSuperstepCounters", "false");
    conf.set("actual.Q.aggregators", "1");
    conf.set("minimum.progress", "2000");
    conf.set("progress.tries", "1");
    conf.set("mapred.output.dir", "tmp/giraph_0");
    return conf;
}
开发者ID:Sotera，项目名称:distributed-graph-analytics，代码行数:13，代码来源:LouvainTests.java

示例10: getConf

import org.apache.giraph.conf.GiraphConfiguration; //导入方法依赖的package包/类
public GiraphConfiguration getConf() {
    GiraphConfiguration conf = new GiraphConfiguration();
    conf.setMasterComputeClass(HBSEMasterCompute.class);
    conf.setComputationClass(HBSEComputation.class);
    conf.setVertexOutputFormatClass(InMemoryVertexOutputFormat.class);
    conf.set(HBSEConfigurationConstants.BETWEENNESS_OUTPUT_DIR, "tmp/output");
    conf.set(HBSEConfigurationConstants.BETWEENNESS_SET_STABILITY, "1");
    conf.set(HBSEConfigurationConstants.BETWEENNESS_SET_MAX_SIZE, "10");
    conf.set(HBSEConfigurationConstants.BETWEENNESS_SET_STABILITY_COUNTER, "3");
    conf.set(HBSEConfigurationConstants.PIVOT_BATCH_SIZE, "2");
    conf.set(HBSEConfigurationConstants.TOTAL_PIVOT_COUNT, "6");
    return conf;
}
开发者ID:Sotera，项目名称:distributed-graph-analytics，代码行数:14，代码来源:HBSEComputeTest.java

示例11: testToyData2

import org.apache.giraph.conf.GiraphConfiguration; //导入方法依赖的package包/类
@Test
public void testToyData2() throws Exception {

  // A small graph
  String[] vertices = new String[]{
          "a 1",
          "b 1",
  };

  String[] edges = new String[]{
          "a b",
          "b a",
  };

  GiraphConfiguration conf = new GiraphConfiguration();
  conf.setComputationClass(LinkRankComputation.class);
  conf.setOutEdgesClass(ByteArrayEdges.class);

  conf.setVertexInputFormatClass(LinkRankVertexInputFormat.class);
  conf.setVertexOutputFormatClass(
          LinkRankVertexOutputFormat.class);
  conf.setEdgeInputFormatClass(LinkRankEdgeInputFormat.class);
  conf.setInt("giraph.linkRank.superstepCount", 10);
  conf.setInt("giraph.linkRank.scale", 10);
  conf.setMasterComputeClass(LinkRankVertexMasterCompute.class);
  // Run internally
  Iterable<String> results = InternalVertexRunner.run(conf, vertices, edges);

  HashMap<String, Double> hm = new HashMap();
  for (String result : results) {
    String[] tokens = result.split("\t");
    hm.put(tokens[0], Double.parseDouble(tokens[1]));
    LOG.info(result);
  }

  assertEquals("a scores are not the same", hm.get("a"), 5.0d, DELTA);
  assertEquals("b scores are not the same", hm.get("b"), 5.0d, DELTA);
}
开发者ID:AGMLab，项目名称:giranking，代码行数:39，代码来源:LinkRankComputationTest.java

示例12: testToyData

import org.apache.giraph.conf.GiraphConfiguration; //导入方法依赖的package包/类
/**
 * A local integration test on toy data
 */
@Test
public void testToyData() throws Exception {

  // A small graph
  String[] graph = new String[] {
    "1 4 2 3",
    "2 1",
    "4 3 2",
    "5 2 4"
  };

  GiraphConfiguration conf = new GiraphConfiguration();
  conf.setInt(RandomWalkWithRestartComputation.MAX_SUPERSTEPS, 50);
  conf.setFloat(
      RandomWalkWithRestartComputation.TELEPORTATION_PROBABILITY, 0.15f);
  conf.setComputationClass(PageRankComputation.class);
  conf.setOutEdgesClass(ByteArrayEdges.class);
  conf.setVertexInputFormatClass(LongDoubleNullTextInputFormat.class);
  conf.setVertexOutputFormatClass(
      VertexWithDoubleValueNullEdgeTextOutputFormat.class);
  conf.setWorkerContextClass(RandomWalkWorkerContext.class);
  conf.setMasterComputeClass(RandomWalkVertexMasterCompute.class);
  // Run internally
  Iterable<String> results = InternalVertexRunner.run(conf, graph);

  Map<Long, Double> steadyStateProbabilities =
      RandomWalkTestUtils.parseSteadyStateProbabilities(results);

  assertEquals(0.28159076008518047, steadyStateProbabilities.get(1l),
      RandomWalkTestUtils.EPSILON);
  assertEquals(0.2514648601529863, steadyStateProbabilities.get(2l),
      RandomWalkTestUtils.EPSILON);
  assertEquals(0.22262961972286327, steadyStateProbabilities.get(3l),
      RandomWalkTestUtils.EPSILON);
  assertEquals(0.17646783276703806, steadyStateProbabilities.get(4l),
      RandomWalkTestUtils.EPSILON);
  assertEquals(0.06784692727193153, steadyStateProbabilities.get(5l),
      RandomWalkTestUtils.EPSILON);
}
开发者ID:renato2099，项目名称:giraph-gora，代码行数:43，代码来源:PageRankComputationTest.java

示例13: testToyData

import org.apache.giraph.conf.GiraphConfiguration; //导入方法依赖的package包/类
/**
 * A local integration test on toy data
 */
@Test
public void testToyData() throws Exception {

  // A small graph
  String[] graph = new String[] {
    "1 4 2 3",
    "2 1",
    "4 3 2",
    "5 2 4"
  };

  GiraphConfiguration conf = new GiraphConfiguration();
  conf.setInt(RandomWalkWithRestartVertex.MAX_SUPERSTEPS, 50);
  conf.setFloat(RandomWalkWithRestartVertex.TELEPORTATION_PROBABILITY, 0.15f);
  conf.setVertexClass(PageRankVertex.class);
  conf.setOutEdgesClass(ByteArrayEdges.class);
  conf.setVertexInputFormatClass(LongDoubleNullTextInputFormat.class);
  conf.setVertexOutputFormatClass(
      VertexWithDoubleValueNullEdgeTextOutputFormat.class);
  conf.setWorkerContextClass(RandomWalkWorkerContext.class);
  conf.setMasterComputeClass(RandomWalkVertexMasterCompute.class);
  // Run internally
  Iterable<String> results = InternalVertexRunner.run(conf, graph);

  Map<Long, Double> steadyStateProbabilities =
      RandomWalkTestUtils.parseSteadyStateProbabilities(results);

  assertEquals(0.28159076008518047, steadyStateProbabilities.get(1l),
      RandomWalkTestUtils.EPSILON);
  assertEquals(0.2514648601529863, steadyStateProbabilities.get(2l),
      RandomWalkTestUtils.EPSILON);
  assertEquals(0.22262961972286327, steadyStateProbabilities.get(3l),
      RandomWalkTestUtils.EPSILON);
  assertEquals(0.17646783276703806, steadyStateProbabilities.get(4l),
      RandomWalkTestUtils.EPSILON);
  assertEquals(0.06784692727193153, steadyStateProbabilities.get(5l),
      RandomWalkTestUtils.EPSILON);
}
开发者ID:zfighter，项目名称:giraph-research，代码行数:42，代码来源:PageRankVertexTest.java

示例14: testSingleFault

import org.apache.giraph.conf.GiraphConfiguration; //导入方法依赖的package包/类
/**
 * Run a job that requires checkpointing and will have a worker crash
 * and still recover from a previous checkpoint.
 *
 * @throws IOException
 * @throws ClassNotFoundException
 * @throws InterruptedException
 */
@Test
public void testSingleFault()
  throws IOException, InterruptedException, ClassNotFoundException {
  if (!runningInDistributedMode()) {
    System.out.println(
        "testSingleFault: Ignore this test in local mode.");
    return;
  }
  Path outputPath = getTempPath(getCallingMethodName());
  GiraphConfiguration conf = new GiraphConfiguration();
  conf.setComputationClass(
      SimpleCheckpoint.SimpleCheckpointComputation.class);
  conf.setWorkerContextClass(
      SimpleCheckpoint.SimpleCheckpointVertexWorkerContext.class);
  conf.setMasterComputeClass(
      SimpleCheckpoint.SimpleCheckpointVertexMasterCompute.class);
  conf.setVertexInputFormatClass(SimpleSuperstepVertexInputFormat.class);
  conf.setVertexOutputFormatClass(SimpleSuperstepVertexOutputFormat.class);
  conf.setBoolean(SimpleCheckpoint.ENABLE_FAULT, true);
  conf.setInt("mapred.map.max.attempts", 4);
  // Trigger failure faster
  conf.setInt("mapred.task.timeout", 10000);
  conf.setMaxMasterSuperstepWaitMsecs(10000);
  conf.setEventWaitMsecs(1000);
  conf.setCheckpointFrequency(2);
  GiraphConstants.CHECKPOINT_DIRECTORY.set(conf,
      getTempPath("_singleFaultCheckpoints").toString());
  GiraphConstants.CLEANUP_CHECKPOINTS_AFTER_SUCCESS.set(conf, false);
  GiraphConstants.ZOOKEEPER_SESSION_TIMEOUT.set(conf, 10000);
  GiraphConstants.ZOOKEEPER_MIN_SESSION_TIMEOUT.set(conf, 10000);
  GiraphJob job = prepareJob(getCallingMethodName(), conf, outputPath);
  assertTrue(job.run(true));
}
开发者ID:renato2099，项目名称:giraph-gora，代码行数:42，代码来源:TestAutoCheckpoint.java

示例15: prepareConfiguration

import org.apache.giraph.conf.GiraphConfiguration; //导入方法依赖的package包/类
@Override
protected void prepareConfiguration(GiraphConfiguration conf,
    CommandLine cmd) {
  conf.setComputationClass(AggregatorsBenchmarkComputation.class);
  conf.setMasterComputeClass(AggregatorsBenchmarkMasterCompute.class);
  conf.setVertexInputFormatClass(PseudoRandomVertexInputFormat.class);
  conf.setWorkerContextClass(AggregatorsBenchmarkWorkerContext.class);
  conf.setLong(PseudoRandomInputFormatConstants.AGGREGATE_VERTICES,
      BenchmarkOption.VERTICES.getOptionLongValue(cmd));
  conf.setLong(PseudoRandomInputFormatConstants.EDGES_PER_VERTEX, 1);
  conf.setInt(AGGREGATORS_NUM, AGGREGATORS.getOptionIntValue(cmd));
  conf.setInt("workers", conf.getInt(GiraphConstants.MAX_WORKERS, -1));
}
开发者ID:renato2099，项目名称:giraph-gora，代码行数:14，代码来源:AggregatorsBenchmark.java


注：本文中的org.apache.giraph.conf.GiraphConfiguration.setMasterComputeClass方法示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。