当前位置: 首页>>代码示例>>Java>>正文


Java InputSplit.readFields方法代码示例

本文整理汇总了Java中org.apache.hadoop.mapred.InputSplit.readFields方法的典型用法代码示例。如果您正苦于以下问题:Java InputSplit.readFields方法的具体用法?Java InputSplit.readFields怎么用?Java InputSplit.readFields使用的例子?那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在org.apache.hadoop.mapred.InputSplit的用法示例。


在下文中一共展示了InputSplit.readFields方法的5个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: deserializeInputSplit

import org.apache.hadoop.mapred.InputSplit; //导入方法依赖的package包/类
public static InputSplit deserializeInputSplit(String base64, String className) throws IOException, ReflectiveOperationException{
  Constructor<?> constructor = Class.forName(className).getDeclaredConstructor();
  if (constructor == null) {
    throw new ReflectiveOperationException("Class " + className + " does not implement a default constructor.");
  }
  constructor.setAccessible(true);
  InputSplit split = (InputSplit) constructor.newInstance();
  ByteArrayDataInput byteArrayDataInput = ByteStreams.newDataInput(Base64.decodeBase64(base64));
  split.readFields(byteArrayDataInput);
  return split;
}
 
开发者ID:skhalifa,项目名称:QDrill,代码行数:12,代码来源:HiveSubScan.java

示例2: readFields

import org.apache.hadoop.mapred.InputSplit; //导入方法依赖的package包/类
@SuppressWarnings("unchecked")
public void readFields(DataInput in) throws IOException {
  inputSplitClass = (Class<? extends InputSplit>) readClass(in);
  inputSplit = (InputSplit) ReflectionUtils
     .newInstance(inputSplitClass, conf);
  inputSplit.readFields(in);
  inputFormatClass = (Class<? extends InputFormat>) readClass(in);
  mapperClass = (Class<? extends Mapper>) readClass(in);
}
 
开发者ID:naver,项目名称:hadoop,代码行数:10,代码来源:TaggedInputSplit.java

示例3: deserializeInputSplit

import org.apache.hadoop.mapred.InputSplit; //导入方法依赖的package包/类
public static InputSplit deserializeInputSplit(SerializedInputSplit split) throws IOException, ReflectiveOperationException{
  Constructor<?> constructor = Class.forName(split.getInputSplitClass()).getDeclaredConstructor();
  if (constructor == null) {
    throw new ReflectiveOperationException("Class " + split.getInputSplitClass() + " does not implement a default constructor.");
  }
  constructor.setAccessible(true);
  InputSplit deserializedSplit = (InputSplit) constructor.newInstance();
  deserializedSplit.readFields(ByteStreams.newDataInput(split.getInputSplit().toByteArray()));
  return deserializedSplit;
}
 
开发者ID:dremio,项目名称:dremio-oss,代码行数:11,代码来源:HiveAbstractReader.java

示例4: readFields

import org.apache.hadoop.mapred.InputSplit; //导入方法依赖的package包/类
@Override
public void readFields(DataInput in) throws IOException {
  String inputSplitClassName = in.readUTF();
  try {
    inputSplit = (InputSplit) ReflectionUtil.newInstance(conf
        .getClassByName(inputSplitClassName), conf);
  } catch (Exception e) {
    throw new IOException(
        "Cannot create an instance of InputSplit class = "
        + inputSplitClassName + ":" + e.getMessage(), e);
  }
  inputSplit.readFields(in);
  inputFormatClassName = in.readUTF();
}
 
开发者ID:mini666,项目名称:hive-phoenix-handler,代码行数:15,代码来源:HiveInputFormat.java

示例5: deserializeInputSplit

import org.apache.hadoop.mapred.InputSplit; //导入方法依赖的package包/类
public static List<InputSplit> deserializeInputSplit(List<String> base64, String className) throws IOException, ReflectiveOperationException{
  Constructor<?> constructor = Class.forName(className).getDeclaredConstructor();
  if (constructor == null) {
    throw new ReflectiveOperationException("Class " + className + " does not implement a default constructor.");
  }
  constructor.setAccessible(true);
  List<InputSplit> splits = new ArrayList<>();
  for (String str : base64) {
    InputSplit split = (InputSplit) constructor.newInstance();
    ByteArrayDataInput byteArrayDataInput = ByteStreams.newDataInput(Base64.decodeBase64(str));
    split.readFields(byteArrayDataInput);
    splits.add(split);
  }
  return splits;
}
 
开发者ID:axbaretto,项目名称:drill,代码行数:16,代码来源:HiveSubScan.java


注:本文中的org.apache.hadoop.mapred.InputSplit.readFields方法示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。