本文整理汇总了Java中org.apache.pig.LoadFunc.getInputFormat方法的典型用法代码示例。如果您正苦于以下问题：Java LoadFunc.getInputFormat方法的具体用法？Java LoadFunc.getInputFormat怎么用？Java LoadFunc.getInputFormat使用的例子？那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类org.apache.pig.LoadFunc的用法示例。
在下文中一共展示了LoadFunc.getInputFormat方法的2个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。
示例1: createRecordReader
import org.apache.pig.LoadFunc; //导入方法依赖的package包/类
@Override
public org.apache.hadoop.mapreduce.RecordReader<Text, Tuple> createRecordReader(
org.apache.hadoop.mapreduce.InputSplit split,
TaskAttemptContext context) throws IOException,
InterruptedException {
// We need to create a TaskAttemptContext based on the Configuration which
// was used in the getSplits() to produce the split supplied here. For
// this, let's find out the input of the script which produced the split
// supplied here and then get the corresponding Configuration and setup
// TaskAttemptContext based on it and then call the real InputFormat's
// createRecordReader() method
PigSplit pigSplit = (PigSplit)split;
activeSplit = pigSplit;
// XXX hadoop 20 new API integration: get around a hadoop 20 bug by
// passing total # of splits to each split so it can be retrieved
// here and set it to the configuration object. This number is needed
// by PoissonSampleLoader to compute the number of samples
int n = pigSplit.getTotalSplits();
context.getConfiguration().setInt("pig.mapsplits.count", n);
Configuration conf = context.getConfiguration();
LoadFunc loadFunc = getLoadFunc(pigSplit.getInputIndex(), conf);
// Pass loader signature to LoadFunc and to InputFormat through
// the conf
passLoadSignature(loadFunc, pigSplit.getInputIndex(), conf);
// merge entries from split specific conf into the conf we got
PigInputFormat.mergeSplitSpecificConf(loadFunc, pigSplit, conf);
// for backward compatibility
PigInputFormat.sJob = conf;
InputFormat inputFormat = loadFunc.getInputFormat();
List<Long> inpLimitLists =
(ArrayList<Long>)ObjectSerializer.deserialize(
conf.get("pig.inpLimits"));
return new PigRecordReader(inputFormat, pigSplit, loadFunc, context, inpLimitLists.get(pigSplit.getInputIndex()));
}
示例2: createRecordReader
import org.apache.pig.LoadFunc; //导入方法依赖的package包/类
@SuppressWarnings({ "rawtypes", "unchecked" })
@Override
public org.apache.hadoop.mapreduce.RecordReader<Text, Tuple> createRecordReader(
org.apache.hadoop.mapreduce.InputSplit split,
TaskAttemptContext context) throws IOException,
InterruptedException {
// We need to create a TaskAttemptContext based on the Configuration which
// was used in the getSplits() to produce the split supplied here. For
// this, let's find out the input of the script which produced the split
// supplied here and then get the corresponding Configuration and setup
// TaskAttemptContext based on it and then call the real InputFormat's
// createRecordReader() method
PigSplit pigSplit = (PigSplit)split;
activeSplit = pigSplit;
// XXX hadoop 20 new API integration: get around a hadoop 20 bug by
// passing total # of splits to each split so it can be retrieved
// here and set it to the configuration object. This number is needed
// by PoissonSampleLoader to compute the number of samples
int n = pigSplit.getTotalSplits();
context.getConfiguration().setInt("pig.mapsplits.count", n);
Configuration conf = context.getConfiguration();
PigContext.setPackageImportList((ArrayList<String>) ObjectSerializer
.deserialize(conf.get("udf.import.list")));
MapRedUtil.setupUDFContext(conf);
LoadFunc loadFunc = getLoadFunc(pigSplit.getInputIndex(), conf);
// Pass loader signature to LoadFunc and to InputFormat through
// the conf
passLoadSignature(loadFunc, pigSplit.getInputIndex(), conf);
// merge entries from split specific conf into the conf we got
PigInputFormat.mergeSplitSpecificConf(loadFunc, pigSplit, conf);
// for backward compatibility
PigInputFormat.sJob = conf;
InputFormat inputFormat = loadFunc.getInputFormat();
List<Long> inpLimitLists =
(ArrayList<Long>)ObjectSerializer.deserialize(
conf.get("pig.inpLimits"));
return new PigRecordReader(inputFormat, pigSplit, loadFunc, context, inpLimitLists.get(pigSplit.getInputIndex()));
}