本文整理汇总了Java中org.apache.tez.mapreduce.hadoop.MRHelpers.doJobClientMagic方法的典型用法代码示例。如果您正苦于以下问题：Java MRHelpers.doJobClientMagic方法的具体用法？Java MRHelpers.doJobClientMagic怎么用？Java MRHelpers.doJobClientMagic使用的例子？那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类org.apache.tez.mapreduce.hadoop.MRHelpers的用法示例。
在下文中一共展示了MRHelpers.doJobClientMagic方法的2个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。
示例1: createUserPayload
import org.apache.tez.mapreduce.hadoop.MRHelpers; //导入方法依赖的package包/类
/**
* Helper API to generate the user payload for the MRInput and
* MRInputAMSplitGenerator (if used). The InputFormat will be invoked by Tez
* at DAG runtime to generate the input splits.
*
* @param conf
* Configuration for the InputFormat
* @param inputFormatClassName
* Name of the class of the InputFormat
* @param useNewApi
* use new mapreduce API or old mapred API
* @param groupSplitsInAM
* do grouping of splits in the AM. If true then splits generated by
* the InputFormat will be grouped in the AM based on available
* resources, locality etc. This option may be set to true only when
* using MRInputAMSplitGenerator as the initializer class in
* {@link Vertex#addInput(String, org.apache.tez.dag.api.InputDescriptor, Class)}
* @return returns the user payload to be set on the InputDescriptor of MRInput
* @throws IOException
*/
public static byte[] createUserPayload(Configuration conf,
String inputFormatClassName, boolean useNewApi, boolean groupSplitsInAM)
throws IOException {
Configuration inputConf = new JobConf(conf);
String wrappedInputFormatClassName = null;
String configInputFormatClassName = null;
if (groupSplitsInAM) {
wrappedInputFormatClassName = inputFormatClassName;
configInputFormatClassName = TezGroupedSplitsInputFormat.class.getName();
} else {
wrappedInputFormatClassName = null;
configInputFormatClassName = inputFormatClassName;
}
inputConf.set(MRJobConfig.INPUT_FORMAT_CLASS_ATTR,
configInputFormatClassName);
inputConf.setBoolean("mapred.mapper.new-api", useNewApi);
MRHelpers.translateVertexConfToTez(inputConf);
MRHelpers.doJobClientMagic(inputConf);
if (groupSplitsInAM) {
return MRHelpers.createMRInputPayloadWithGrouping(inputConf,
wrappedInputFormatClassName);
} else {
return MRHelpers.createMRInputPayload(inputConf, null);
}
}
示例2: createUserPayload
import org.apache.tez.mapreduce.hadoop.MRHelpers; //导入方法依赖的package包/类
/**
* Creates the user payload to be set on the OutputDescriptor for MROutput
* @param conf Configuration for the OutputFormat
* @param outputFormatName Name of the class of the OutputFormat
* @param useNewApi Use new mapreduce API or old mapred API
* @return
* @throws IOException
*/
public static byte[] createUserPayload(Configuration conf,
String outputFormatName, boolean useNewApi) throws IOException {
Configuration outputConf = new JobConf(conf);
outputConf.set(MRJobConfig.OUTPUT_FORMAT_CLASS_ATTR, outputFormatName);
outputConf.setBoolean("mapred.mapper.new-api", useNewApi);
MRHelpers.translateVertexConfToTez(outputConf);
MRHelpers.doJobClientMagic(outputConf);
return TezUtils.createUserPayloadFromConf(outputConf);
}