本文整理匯總了Java中org.apache.hadoop.mapred.InputSplit.getLength方法的典型用法代碼示例。如果您正苦於以下問題：Java InputSplit.getLength方法的具體用法？Java InputSplit.getLength怎麽用？Java InputSplit.getLength使用的例子？那麽, 這裏精選的方法代碼示例或許可以為您提供幫助。您也可以進一步了解該方法所在類org.apache.hadoop.mapred.InputSplit的用法示例。
在下文中一共展示了InputSplit.getLength方法的3個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Java代碼示例。
示例1: getScanStats
import org.apache.hadoop.mapred.InputSplit; //導入方法依賴的package包/類
@Override
public ScanStats getScanStats() {
try {
long data =0;
for (final InputSplit split : inputSplits) {
data += split.getLength();
}
long estRowCount = rowCount;
if (estRowCount == 0) {
// having a rowCount of 0 can mean the statistics were never computed
estRowCount = data/1024;
}
// Hive's native reader is neither memory efficient nor fast. Increase the CPU cost
// by a factor to let the planner choose HiveDrillNativeScan over HiveScan with SerDes.
float cpuCost = 1 * getSerDeOverheadFactor();
logger.debug("estimated row count = {}, stats row count = {}", estRowCount, rowCount);
return new ScanStats(GroupScanProperty.NO_EXACT_ROW_COUNT, estRowCount, cpuCost, data);
} catch (final IOException e) {
throw new DrillRuntimeException(e);
}
}
示例2: add
import org.apache.hadoop.mapred.InputSplit; //導入方法依賴的package包/類
/**
* Add an InputSplit to this collection.
* @throws IOException If capacity was not specified during construction
* or if capacity has been reached.
*/
public void add(InputSplit s) throws IOException {
if (null == splits) {
throw new IOException("Uninitialized InputSplit");
}
if (fill == splits.length) {
throw new IOException("Too many splits");
}
splits[fill++] = s;
totsize += s.getLength();
}
示例3: getLength
import org.apache.hadoop.mapred.InputSplit; //導入方法依賴的package包/類
/**
* @return returns total length of all stored input splits
*/
public long getLength() throws IOException {
long length = 0L;
for (InputSplit inputSplit: inputSplits) {
length += inputSplit.getLength();
}
return length;
}