當前位置: 首頁>>代碼示例>>Java>>正文

Java Reader類代碼示例

本文整理匯總了Java中org.apache.tez.runtime.api.Reader類的典型用法代碼示例。如果您正苦於以下問題：Java Reader類的具體用法？Java Reader怎麽用？Java Reader使用的例子？那麽, 這裏精選的類代碼示例或許可以為您提供幫助。


Reader類屬於org.apache.tez.runtime.api包，在下文中一共展示了Reader類的14個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Java代碼示例。

示例1: attachInputs

import org.apache.tez.runtime.api.Reader; //導入依賴的package包/類
@Override
public void attachInputs(Map<String, LogicalInput> inputs,
        Configuration conf)
        throws ExecException {
    this.conf = conf;
    LogicalInput input = inputs.get(inputKey);
    if (input == null) {
        throw new ExecException("Input from vertex " + inputKey + " is missing");
    }

    try {
        Reader r = input.getReader();
        if (r instanceof KeyValueReader) {
            reader = (KeyValueReader) r;
        } else {
            shuffleInput = true;
            shuffleReader = (KeyValuesReader) r;
            hasNext = shuffleReader.next();
        }
        LOG.info("Attached input from vertex " + inputKey + " : input=" + input + ", reader=" + r);
    } catch (Exception e) {
        throw new ExecException(e);
    }
}
開發者ID:sigmoidanalytics，項目名稱:spork，代碼行數:25，代碼來源:POValueInputTez.java

示例2: attachInputs

import org.apache.tez.runtime.api.Reader; //導入依賴的package包/類
@Override
public void attachInputs(Map<String, LogicalInput> inputs,
        Configuration conf) throws ExecException {
    LogicalInput input = inputs.get(inputKey);
    if (input == null) {
        throw new ExecException("Input from vertex " + inputKey + " is missing");
    }
    try {
        Reader r = input.getReader();
        if (r instanceof KeyValueReader) {
            reader = (KeyValueReader) r;
        } else {
            shuffleInput = true;
            shuffleReader = (KeyValuesReader) r;
        }
        LOG.info("Attached input from vertex " + inputKey + " : input=" + input + ", reader=" + r);
    } catch (Exception e) {
        throw new ExecException(e);
    }
}
開發者ID:sigmoidanalytics，項目名稱:spork，代碼行數:21，代碼來源:POIdentityInOutTez.java

示例3: run

import org.apache.tez.runtime.api.Reader; //導入依賴的package包/類
@Override
public void run() throws Exception {
  Preconditions.checkState(getInputs().size() == 1);
  Preconditions.checkState(getOutputs().size() == 1);
  LogicalInput input = getInputs().values().iterator().next();
  Reader rawReader = input.getReader();
  Preconditions.checkState(rawReader instanceof KeyValueReader);
  LogicalOutput output = getOutputs().values().iterator().next();

  KeyValueReader reader = (KeyValueReader) rawReader;
  KeyValueWriter writer = (KeyValueWriter) output.getWriter();

  while (reader.next()) {
    Object val = reader.getCurrentValue();
    writer.write(val, NullWritable.get());
  }
}
開發者ID:apache，項目名稱:incubator-tez，代碼行數:18，代碼來源:IntersectExample.java

示例4: run

import org.apache.tez.runtime.api.Reader; //導入依賴的package包/類
@Override
public void run() throws Exception {
  Preconditions.checkState(getInputs().size() == 2);
  Preconditions.checkState(getOutputs().size() == 1);
  // Get the input data for the 2 sides of the join from the 2 inputs
  LogicalInput logicalInput1 = getInputs().get(input1);
  LogicalInput logicalInput2 = getInputs().get(input2);
  Reader inputReader1 = logicalInput1.getReader();
  Reader inputReader2 = logicalInput2.getReader();
  Preconditions.checkState(inputReader1 instanceof KeyValuesReader);
  Preconditions.checkState(inputReader2 instanceof KeyValuesReader);
  LogicalOutput lo = getOutputs().get(joinOutput);
  Preconditions.checkState(lo.getWriter() instanceof KeyValueWriter);
  KeyValueWriter writer = (KeyValueWriter) lo.getWriter();

  join((KeyValuesReader) inputReader1, (KeyValuesReader) inputReader2,
      writer);
}
開發者ID:apache，項目名稱:tez，代碼行數:19，代碼來源:SortMergeJoinExample.java

示例5: run

import org.apache.tez.runtime.api.Reader; //導入依賴的package包/類
@Override
public void run() throws Exception {
  Preconditions.checkState(getInputs().size() == 1);
  Preconditions.checkState(getOutputs().size() == 1);
  // not looking up inputs and outputs by name because there is just one
  // instance and this processor is used in many different DAGs with
  // different names for inputs and outputs
  LogicalInput input = getInputs().values().iterator().next();
  Reader rawReader = input.getReader();
  Preconditions.checkState(rawReader instanceof KeyValueReader);
  LogicalOutput output = getOutputs().values().iterator().next();

  KeyValueReader reader = (KeyValueReader) rawReader;
  KeyValueWriter writer = (KeyValueWriter) output.getWriter();

  while (reader.next()) {
    Object val = reader.getCurrentValue();
    // The data value itself is the join key. Simply write it out as the
    // key.
    // The output value is null.
    writer.write(val, NullWritable.get());
  }
}
開發者ID:apache，項目名稱:tez，代碼行數:24，代碼來源:HashJoinExample.java

示例6: run

import org.apache.tez.runtime.api.Reader; //導入依賴的package包/類
@Override
public void run() throws Exception {
  Preconditions.checkState(getInputs().size() == 2);
  Preconditions.checkState(getOutputs().size() == 0);
  LogicalInput lhsInput = getInputs().get(LHS_INPUT_NAME);
  LogicalInput rhsInput = getInputs().get(RHS_INPUT_NAME);
  Reader lhsReaderRaw = lhsInput.getReader();
  Reader rhsReaderRaw = rhsInput.getReader();
  Preconditions.checkState(lhsReaderRaw instanceof KeyValuesReader);
  Preconditions.checkState(rhsReaderRaw instanceof KeyValuesReader);
  KeyValuesReader lhsReader = (KeyValuesReader) lhsReaderRaw;
  KeyValuesReader rhsReader = (KeyValuesReader) rhsReaderRaw;

  TezCounter lhsMissingKeyCounter = getContext().getCounters().findCounter(COUNTER_GROUP_NAME,
      MISSING_KEY_COUNTER_NAME);

  while (lhsReader.next()) {
    if (rhsReader.next()) {
      if (!lhsReader.getCurrentKey().equals(rhsReader.getCurrentKey())) {
        LOG.info("MismatchedKeys: " + "lhs=" + lhsReader.getCurrentKey() + ", rhs=" + rhsReader.getCurrentKey());
        lhsMissingKeyCounter.increment(1);
      }
    } else {
      lhsMissingKeyCounter.increment(1);
      LOG.info("ExtraKey in lhs: " + lhsReader.getClass());
      break;
    }
  }
  if (rhsReader.next()) {
    lhsMissingKeyCounter.increment(1);
    LOG.info("ExtraKey in rhs: " + lhsReader.getClass());
  }
}
開發者ID:apache，項目名稱:incubator-tez，代碼行數:34，代碼來源:IntersectValidate.java

示例7: run

import org.apache.tez.runtime.api.Reader; //導入依賴的package包/類
@Override
public void run() throws Exception {
  Preconditions.checkState(getInputs().size() == 2);
  Preconditions.checkState(getOutputs().size() == 0);
  LogicalInput lhsInput = getInputs().get(LHS_INPUT_NAME);
  LogicalInput rhsInput = getInputs().get(RHS_INPUT_NAME);
  Reader lhsReaderRaw = lhsInput.getReader();
  Reader rhsReaderRaw = rhsInput.getReader();
  Preconditions.checkState(lhsReaderRaw instanceof KeyValuesReader);
  Preconditions.checkState(rhsReaderRaw instanceof KeyValuesReader);
  KeyValuesReader lhsReader = (KeyValuesReader) lhsReaderRaw;
  KeyValuesReader rhsReader = (KeyValuesReader) rhsReaderRaw;
  boolean rhsReaderEnd = false;

  TezCounter lhsMissingKeyCounter = getContext().getCounters().findCounter(COUNTER_GROUP_NAME,
      MISSING_KEY_COUNTER_NAME);

  while (lhsReader.next()) {
    if (rhsReader.next()) {
      if (!lhsReader.getCurrentKey().equals(rhsReader.getCurrentKey())) {
        LOG.info("MismatchedKeys: " + "lhs=" + lhsReader.getCurrentKey() + ", rhs=" + rhsReader.getCurrentKey());
        lhsMissingKeyCounter.increment(1);
      }
    } else {
      lhsMissingKeyCounter.increment(1);
      LOG.info("ExtraKey in lhs: " + lhsReader.getClass());
      rhsReaderEnd = true;
      break;
    }
  }
  if (!rhsReaderEnd && rhsReader.next()) {
    lhsMissingKeyCounter.increment(1);
    LOG.info("ExtraKey in rhs: " + rhsReader.getClass());
  }
}
開發者ID:apache，項目名稱:tez，代碼行數:36，代碼來源:JoinValidate.java

示例8: getReader

import org.apache.tez.runtime.api.Reader; //導入依賴的package包/類
@Override
public Reader getReader() throws Exception {
  return new KeyValueReader() {
    String[] keys = new String[numRecordPerSrc];

    int i = -1;

    @Override
    public boolean next() throws IOException {
      if (i == -1) {
        for (int j = 0; j < numRecordPerSrc; j++) {
          keys[j] = ""+j;
        }
      }
      i++;
      return i < keys.length;
    }

    @Override
    public Object getCurrentKey() throws IOException {
      return keys[i];
    }

    @Override
    public Object getCurrentValue() throws IOException {
      return keys[i];
    }
  };
}
開發者ID:apache，項目名稱:tez，代碼行數:30，代碼來源:CartesianProduct.java

示例9: getReader

import org.apache.tez.runtime.api.Reader; //導入依賴的package包/類
@Override
public Reader getReader() throws Exception {
  if (this.exLocation == ExceptionLocation.INPUT_HANDLE_EVENTS) {
    synchronized (condition) {
      // wait for exception thrown from handleEvents. Otherwise,
      // processor may exit before the exception from handleEvents is
      // caught.
      condition.wait();
    }
  }
  if (this.exLocation == ExceptionLocation.INPUT_GET_READER) {
    throw new Exception(this.exLocation.name());
  }
  return null;
}
開發者ID:apache，項目名稱:tez，代碼行數:16，代碼來源:TestExceptionPropagation.java

示例10: getReader

import org.apache.tez.runtime.api.Reader; //導入依賴的package包/類
@Override
public Reader getReader() throws Exception {
  return null;
}
開發者ID:apache，項目名稱:incubator-tez，代碼行數:5，代碼來源:TestInput.java

示例11: getReader

import org.apache.tez.runtime.api.Reader; //導入依賴的package包/類
@Override
public Reader getReader() throws Exception {
  return new ConcatenatedMergedKeyValueReader();
}
開發者ID:apache，項目名稱:incubator-tez，代碼行數:5，代碼來源:ConcatenatedMergedKeyValueInput.java

示例12: getReader

import org.apache.tez.runtime.api.Reader; //導入依賴的package包/類
@Override
public Reader getReader() throws Exception {
  return new ConcatenatedMergedKeyValuesReader();
}
開發者ID:apache，項目名稱:incubator-tez，代碼行數:5，代碼來源:ConcatenatedMergedKeyValuesInput.java

示例13: getReader

import org.apache.tez.runtime.api.Reader; //導入依賴的package包/類
@Override
public Reader getReader() throws Exception {
  throw new UnsupportedOperationException("getReader not supported. use getKeyValueReaders");
}
開發者ID:apache，項目名稱:incubator-tez，代碼行數:5，代碼來源:MultiMRInput.java

示例14: getReader

import org.apache.tez.runtime.api.Reader; //導入依賴的package包/類
@Override
public Reader getReader() throws Exception {
  return reader;
}
開發者ID:apache，項目名稱:tez，代碼行數:5，代碼來源:TestSortedGroupedMergedInput.java


注：本文中的org.apache.tez.runtime.api.Reader類示例由純淨天空整理自Github/MSDocs等開源代碼及文檔管理平台，相關代碼片段篩選自各路編程大神貢獻的開源項目，源碼版權歸原作者所有，傳播和使用請參考對應項目的License；未經允許，請勿轉載。