当前位置: 首页>>代码示例>>Java>>正文


Java EdgeReader.close方法代码示例

本文整理汇总了Java中org.apache.giraph.io.EdgeReader.close方法的典型用法代码示例。如果您正苦于以下问题:Java EdgeReader.close方法的具体用法?Java EdgeReader.close怎么用?Java EdgeReader.close使用的例子?那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在org.apache.giraph.io.EdgeReader的用法示例。


在下文中一共展示了EdgeReader.close方法的1个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: readInputSplit

import org.apache.giraph.io.EdgeReader; //导入方法依赖的package包/类
/**
 * Read edges from input split.  If testing, the user may request a
 * maximum number of edges to be read from an input split.
 *
 * @param inputSplit Input split to process with edge reader
 * @param graphState Current graph state
 * @return Edges loaded from this input split
 * @throws IOException
 * @throws InterruptedException
 */
@Override
protected VertexEdgeCount readInputSplit(
    InputSplit inputSplit,
    GraphState<I, V, E, M> graphState) throws IOException,
    InterruptedException {
  EdgeInputFormat<I, E> edgeInputFormat =
      configuration.createEdgeInputFormat();
  EdgeReader<I, E> edgeReader =
      edgeInputFormat.createEdgeReader(inputSplit, context);
  edgeReader.setConf(
      (ImmutableClassesGiraphConfiguration<I, Writable, E, Writable>)
          configuration);
  edgeReader.initialize(inputSplit, context);
  long inputSplitEdgesLoaded = 0;
  while (edgeReader.nextEdge()) {
    I sourceId = edgeReader.getCurrentSourceId();
    Edge<I, E> readerEdge = edgeReader.getCurrentEdge();
    if (sourceId == null) {
      throw new IllegalArgumentException(
          "readInputSplit: Edge reader returned an edge " +
              "without a source vertex id!  - " + readerEdge);
    }
    if (readerEdge.getTargetVertexId() == null) {
      throw new IllegalArgumentException(
          "readInputSplit: Edge reader returned an edge " +
              "without a target vertex id!  - " + readerEdge);
    }
    if (readerEdge.getValue() == null) {
      throw new IllegalArgumentException(
          "readInputSplit: Edge reader returned an edge " +
              "without a value!  - " + readerEdge);
    }

    graphState.getWorkerClientRequestProcessor().sendEdgeRequest(
        sourceId, readerEdge);
    context.progress(); // do this before potential data transfer
    ++inputSplitEdgesLoaded;

    // Update status every VERTICES_UPDATE_PERIOD edges
    if (inputSplitEdgesLoaded % VERTICES_UPDATE_PERIOD == 0) {
      totalEdgesMeter.mark(VERTICES_UPDATE_PERIOD);
      LoggerUtils.setStatusAndLog(context, LOG, Level.INFO,
          "readEdgeInputSplit: Loaded " +
              totalEdgesMeter.count() + " edges at " +
              totalEdgesMeter.meanRate() + " edges/sec " +
              MemoryUtils.getRuntimeMemoryStats());
    }

    // For sampling, or to limit outlier input splits, the number of
    // records per input split can be limited
    if (inputSplitMaxEdges > 0 &&
        inputSplitEdgesLoaded >= inputSplitMaxEdges) {
      if (LOG.isInfoEnabled()) {
        LOG.info("readInputSplit: Leaving the input " +
            "split early, reached maximum edges " +
            inputSplitEdgesLoaded);
      }
      break;
    }
  }
  edgeReader.close();
  return new VertexEdgeCount(0, inputSplitEdgesLoaded);
}
 
开发者ID:zfighter,项目名称:giraph-research,代码行数:74,代码来源:EdgeInputSplitsCallable.java


注:本文中的org.apache.giraph.io.EdgeReader.close方法示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。