本文整理汇总了Java中org.apache.hadoop.mapred.InputSplit.getLength方法的典型用法代码示例。如果您正苦于以下问题：Java InputSplit.getLength方法的具体用法？Java InputSplit.getLength怎么用？Java InputSplit.getLength使用的例子？那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类org.apache.hadoop.mapred.InputSplit的用法示例。
在下文中一共展示了InputSplit.getLength方法的3个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。
示例1: getScanStats
import org.apache.hadoop.mapred.InputSplit; //导入方法依赖的package包/类
@Override
public ScanStats getScanStats() {
try {
long data =0;
for (final InputSplit split : inputSplits) {
data += split.getLength();
}
long estRowCount = rowCount;
if (estRowCount == 0) {
// having a rowCount of 0 can mean the statistics were never computed
estRowCount = data/1024;
}
// Hive's native reader is neither memory efficient nor fast. Increase the CPU cost
// by a factor to let the planner choose HiveDrillNativeScan over HiveScan with SerDes.
float cpuCost = 1 * getSerDeOverheadFactor();
logger.debug("estimated row count = {}, stats row count = {}", estRowCount, rowCount);
return new ScanStats(GroupScanProperty.NO_EXACT_ROW_COUNT, estRowCount, cpuCost, data);
} catch (final IOException e) {
throw new DrillRuntimeException(e);
}
}
示例2: add
import org.apache.hadoop.mapred.InputSplit; //导入方法依赖的package包/类
/**
* Add an InputSplit to this collection.
* @throws IOException If capacity was not specified during construction
* or if capacity has been reached.
*/
public void add(InputSplit s) throws IOException {
if (null == splits) {
throw new IOException("Uninitialized InputSplit");
}
if (fill == splits.length) {
throw new IOException("Too many splits");
}
splits[fill++] = s;
totsize += s.getLength();
}
示例3: getLength
import org.apache.hadoop.mapred.InputSplit; //导入方法依赖的package包/类
/**
* @return returns total length of all stored input splits
*/
public long getLength() throws IOException {
long length = 0L;
for (InputSplit inputSplit: inputSplits) {
length += inputSplit.getLength();
}
return length;
}