当前位置: 首页>>代码示例>>Java>>正文

Java ResourceFieldSchema.getType方法代码示例

本文整理汇总了Java中org.apache.pig.ResourceSchema.ResourceFieldSchema.getType方法的典型用法代码示例。如果您正苦于以下问题：Java ResourceFieldSchema.getType方法的具体用法？Java ResourceFieldSchema.getType怎么用？Java ResourceFieldSchema.getType使用的例子？那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类org.apache.pig.ResourceSchema.ResourceFieldSchema的用法示例。


在下文中一共展示了ResourceFieldSchema.getType方法的13个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: convert

import org.apache.pig.ResourceSchema.ResourceFieldSchema; //导入方法依赖的package包/类
/**
 * Convert an Avro schema to a Pig schema
 */
public static ResourceSchema convert(Schema schema) throws IOException {

    if (AvroStorageUtils.containsGenericUnion(schema))
        throw new IOException ("We don't accept schema containing generic unions.");

    Set<Schema> visitedRecords = new HashSet<Schema>();
    ResourceFieldSchema inSchema = inconvert(schema, FIELD, visitedRecords);

    ResourceSchema tupleSchema;
    if (inSchema.getType() == DataType.TUPLE) {
        tupleSchema = inSchema.getSchema();
    } else { // other typs
        ResourceFieldSchema tupleWrapper = AvroStorageUtils.wrapAsTuple(inSchema);

        ResourceSchema topSchema = new ResourceSchema();
        topSchema.setFields(new ResourceFieldSchema[] { tupleWrapper });

        tupleSchema = topSchema;

    }
    return tupleSchema;
}
开发者ID:sigmoidanalytics，项目名称:spork-streaming，代码行数:26，代码来源:AvroSchema2Pig.java

示例2: consumeComplexType

import org.apache.pig.ResourceSchema.ResourceFieldSchema; //导入方法依赖的package包/类
private Object consumeComplexType(PushbackInputStream in, ResourceFieldSchema complexFieldSchema) throws IOException {
    Object field;
    switch (complexFieldSchema.getType()) {
    case DataType.BAG:
        field = consumeBag(in, complexFieldSchema);
        break;
    case DataType.TUPLE:
        field = consumeTuple(in, complexFieldSchema);
        break;
    case DataType.MAP:
        field = consumeMap(in, complexFieldSchema);
        break;
    default:
        throw new IOException("Unknown complex data type");
    }
    return field;
}
开发者ID:sigmoidanalytics，项目名称:spork，代码行数:18，代码来源:Utf8StorageConverter.java

示例3: getPredicateFields

import org.apache.pig.ResourceSchema.ResourceFieldSchema; //导入方法依赖的package包/类
@Override
public List<String> getPredicateFields(String location, Job job) throws IOException {
    ResourceSchema schema = getSchema(location, job);
    List<String> predicateFields = new ArrayList<String>();
    for (ResourceFieldSchema field : schema.getFields()) {
        switch(field.getType()) {
        case DataType.BOOLEAN:
        case DataType.INTEGER:
        case DataType.LONG:
        case DataType.FLOAT:
        case DataType.DOUBLE:
        case DataType.DATETIME:
        case DataType.CHARARRAY:
        case DataType.BIGINTEGER:
        case DataType.BIGDECIMAL:
            predicateFields.add(field.getName());
            break;
        default:
            // Skip DataType.BYTEARRAY, DataType.TUPLE, DataType.MAP and DataType.BAG
            break;
        }
    }
    return predicateFields;
}
开发者ID:sigmoidanalytics，项目名称:spork，代码行数:25，代码来源:OrcStorage.java

示例4: convert

import org.apache.pig.ResourceSchema.ResourceFieldSchema; //导入方法依赖的package包/类
/**
 * Convert an Avro schema to a Pig schema
 */
public static ResourceSchema convert(Schema schema) throws IOException
{

    if (AvroStorageUtils.containsGenericUnion(schema))
        throw new IOException("We don't accept schema containing generic unions.");

    Set<Schema> visitedRecords = new HashSet<Schema>();
    ResourceFieldSchema inSchema = inconvert(schema, FIELD, visitedRecords);

    ResourceSchema tupleSchema;
    if (inSchema.getType() == DataType.TUPLE)
    {
        tupleSchema = inSchema.getSchema();
    }
    else
    { // other typs
        ResourceFieldSchema tupleWrapper = AvroStorageUtils.wrapAsTuple(inSchema);

        ResourceSchema topSchema = new ResourceSchema();
        topSchema.setFields(new ResourceFieldSchema[] { tupleWrapper });

        tupleSchema = topSchema;

    }
    return tupleSchema;
}
开发者ID:svemuri，项目名称:CalcEngine，代码行数:30，代码来源:AvroSchema2Pig.java

示例5: add2BagSchema

import org.apache.pig.ResourceSchema.ResourceFieldSchema; //导入方法依赖的package包/类
/**
 * Add a field schema to a bag schema
 */
static protected void add2BagSchema(ResourceFieldSchema fieldSchema,
                                    ResourceFieldSchema subFieldSchema) throws IOException
{

    ResourceFieldSchema wrapped =
            (subFieldSchema.getType() == DataType.TUPLE) ? subFieldSchema
                    : AvroStorageUtils.wrapAsTuple(subFieldSchema);

    ResourceSchema listSchema = new ResourceSchema();
    listSchema.setFields(new ResourceFieldSchema[] { wrapped });

    fieldSchema.setSchema(listSchema);

}
开发者ID:linkedin，项目名称:Cubert，代码行数:18，代码来源:AvroSchema2Pig.java

示例6: isTupleWrapper

import org.apache.pig.ResourceSchema.ResourceFieldSchema; //导入方法依赖的package包/类
/** check whether it is just a wrapped tuple */
public static boolean isTupleWrapper(ResourceFieldSchema pigSchema) {
    Boolean status = false;
    if(pigSchema.getType() == DataType.TUPLE)
        if(pigSchema.getName() != null)
            if(pigSchema.getName().equals(AvroStorageUtils.PIG_TUPLE_WRAPPER))
                status = true;
    return status;
}
开发者ID:linkedin，项目名称:Cubert，代码行数:10，代码来源:AvroStorageUtils.java

示例7: parseSimpleType

import org.apache.pig.ResourceSchema.ResourceFieldSchema; //导入方法依赖的package包/类
private Object parseSimpleType(byte[] b, ResourceFieldSchema simpleFieldSchema) throws IOException {
    Object field;
    switch (simpleFieldSchema.getType()) {
    case DataType.INTEGER:
        field = bytesToInteger(b);
        break;
    case DataType.LONG:
        field = bytesToLong(b);
        break;
    case DataType.FLOAT:
        field = bytesToFloat(b);
        break;
    case DataType.DOUBLE:
        field = bytesToDouble(b);
        break;
    case DataType.CHARARRAY:
        field = bytesToCharArray(b);
        break;
    case DataType.BYTEARRAY:
        field = new DataByteArray(b);
        break;
    case DataType.BOOLEAN:
        field = bytesToBoolean(b);
        break;
    case DataType.BIGINTEGER:
        field = bytesToBigInteger(b);
        break;
    case DataType.BIGDECIMAL:
        field = bytesToBigDecimal(b);
    case DataType.DATETIME:
        field = bytesToDateTime(b);
        break;
    default:
        throw new IOException("Unknown simple data type");
    }
    return field;
}
开发者ID:sigmoidanalytics，项目名称:spork-streaming，代码行数:38，代码来源:Utf8StorageConverter.java

示例8: getPigSchema

import org.apache.pig.ResourceSchema.ResourceFieldSchema; //导入方法依赖的package包/类
public static Schema getPigSchema(ResourceSchema rSchema) 
throws FrontendException {
    if(rSchema == null) {
        return null;
    }
    List<FieldSchema> fsList = new ArrayList<FieldSchema>();
    for(ResourceFieldSchema rfs : rSchema.getFields()) {
        FieldSchema fs = new FieldSchema(rfs.getName(), 
                rfs.getSchema() == null ? 
                        null : getPigSchema(rfs.getSchema()), rfs.getType());
        
        if(rfs.getType() == DataType.BAG) {
            if (fs.schema != null) { // allow partial schema
                if (fs.schema.size() == 1) {
                    FieldSchema innerFs = fs.schema.getField(0);
                    if (innerFs.type != DataType.TUPLE) {
                        ResourceFieldSchema.throwInvalidSchemaException();
                    }
                } else {
                    ResourceFieldSchema.throwInvalidSchemaException();
                }
            } 
        }
        fsList.add(fs);
    }
    return new Schema(fsList);
}
开发者ID:sigmoidanalytics，项目名称:spork-streaming，代码行数:28，代码来源:Schema.java

示例9: add2BagSchema

import org.apache.pig.ResourceSchema.ResourceFieldSchema; //导入方法依赖的package包/类
/**
  * Add a field schema to a bag schema
  */
static protected void add2BagSchema(ResourceFieldSchema fieldSchema,
                                ResourceFieldSchema subFieldSchema)
                                throws IOException {

    ResourceFieldSchema wrapped = (subFieldSchema.getType() == DataType.TUPLE)
                                                          ? subFieldSchema
                                                          : AvroStorageUtils.wrapAsTuple(subFieldSchema);

    ResourceSchema listSchema = new ResourceSchema();
    listSchema.setFields(new ResourceFieldSchema[] { wrapped });

    fieldSchema.setSchema(listSchema);

}
开发者ID:sigmoidanalytics，项目名称:spork-streaming，代码行数:18，代码来源:AvroSchema2Pig.java

示例10: isComplexType

import org.apache.pig.ResourceSchema.ResourceFieldSchema; //导入方法依赖的package包/类
static boolean isComplexType(ResourceFieldSchema fieldSchema) {
    return (fieldSchema != null && fieldSchema.getType() >= 100);
}
开发者ID:xushjie1987，项目名称:es-hadoop-v2.2.0，代码行数:4，代码来源:PigUtils.java

示例11: extractField

import org.apache.pig.ResourceSchema.ResourceFieldSchema; //导入方法依赖的package包/类
@Override
protected Object extractField(Object target) {
    List<String> fieldNames = getFieldNames();
    for (int index = 0; index < fieldNames.size(); index++) {
        String fieldName = fieldNames.get(index);
        if (target instanceof PigTuple) {
            PigTuple pt = (PigTuple) target;
            ResourceFieldSchema[] fields = pt.getSchema().getSchema().getFields();

            boolean foundField = false;
            for (int i = 0; i < fields.length && !foundField; i++) {
                ResourceFieldSchema field = fields[i];
                if (fieldName.equals(field.getName())) {
                    foundField = true;
                    byte type = field.getType();
                    try {
                        Object object = pt.getTuple().get(i);
                        if (DataType.isAtomic(type)) {
                            target = object.toString();
                        }
                        else if (type == DataType.TUPLE) {
                            PigTuple rpt = new PigTuple(field.getSchema());
                            if (object instanceof PigTuple) {
                                rpt.setTuple(((PigTuple) object).getTuple());
                            }
                            else {
                                rpt.setTuple((Tuple) object);
                            }
                            target = rpt;
                        }
                        else {
                            Assert.isTrue(false, String.format("Unsupported data type [%s] for field [%s]; use only 'primitives' or 'tuples'", DataType.findTypeName(type), fieldName));
                        }
                    } catch (ExecException ex) {
                        throw new EsHadoopIllegalStateException(String.format("Cannot retrieve field [%s]", fieldName), ex);
                    }
                }
            }
        }
        else {
            return NOT_FOUND;
        }
    }
    return target;
}
开发者ID:xushjie1987，项目名称:es-hadoop-v2.2.0，代码行数:46，代码来源:PigFieldExtractor.java

示例12: isCompatible

import org.apache.pig.ResourceSchema.ResourceFieldSchema; //导入方法依赖的package包/类
/**
 * Check whether Avro type is compatible with Pig type
 * 
 */
protected static boolean isCompatible(Schema avroSchema, ResourceFieldSchema pigSchema) {

    Schema.Type avroType = avroSchema.getType();
    byte pigType = pigSchema.getType();

    if (avroType.equals(Schema.Type.UNION)) {
        return true;
    } else if (pigType == DataType.TUPLE) {
        /* Tuple is compatible with any type; for users may want to
           get rid of the tuple wrapper */
        return true;
    }
    return  (avroType.equals(Schema.Type.ARRAY) && pigType == DataType.BAG)
                  || (avroType.equals(Schema.Type.MAP) && pigType == DataType.MAP)
                  || (avroType.equals(Schema.Type.STRING) 
                                                  && pigType == DataType.CHARARRAY 
                                                  || pigType == DataType.BIGCHARARRAY)
                  || (avroType.equals(Schema.Type.ENUM) 
                                                  && pigType == DataType.CHARARRAY)
                  || (avroType.equals(Schema.Type.BOOLEAN) 
                                                  && pigType == DataType.BOOLEAN 
                                                  || pigType == DataType.INTEGER)
                  || (avroType.equals(Schema.Type.BYTES) 
                                                  && pigType == DataType.BYTEARRAY)
                  || (avroType.equals(Schema.Type.DOUBLE) 
                                                  && pigType == DataType.DOUBLE
                                                  || pigType == DataType.FLOAT
                                                  || pigType == DataType.INTEGER 
                                                  || pigType == DataType.LONG)
                  || (avroType.equals(Schema.Type.FLOAT)
                                                  && pigType == DataType.FLOAT
                                                  || pigType == DataType.INTEGER 
                                                  || pigType == DataType.LONG)
                  || (avroType.equals(Schema.Type.FIXED) 
                                                  && pigType == DataType.BYTEARRAY)
                  || (avroType.equals(Schema.Type.INT) 
                                                  && pigType == DataType.INTEGER)
                  || (avroType.equals(Schema.Type.LONG)
                                                  && pigType == DataType.LONG 
                                                  || pigType == DataType.INTEGER);

}
开发者ID:linkedin，项目名称:Cubert，代码行数:47，代码来源:PigSchema2Avro.java

示例13: readField

import org.apache.pig.ResourceSchema.ResourceFieldSchema; //导入方法依赖的package包/类
private Object readField(String line, ResourceFieldSchema field, FixedWidthField column) 
                         throws IOException, IllegalArgumentException {

    int start = column.start;
    int end = Math.min(column.end, line.length());

    if (start > line.length())
        return null;

    if (end <= start)
        return null;

    String s  = line.substring(start, end);
    String sTrim = s.trim();

    switch (field.getType()) {
        case DataType.UNKNOWN:
        case DataType.BYTEARRAY:
        case DataType.CHARARRAY:
            if (s.trim().length() == 0)
                return null;
            return s.trim();

        case DataType.BOOLEAN:
            return Boolean.parseBoolean(sTrim);

        case DataType.INTEGER:
            return Integer.parseInt(sTrim);

        case DataType.LONG:
            return Long.parseLong(sTrim);

        case DataType.FLOAT:
            return Float.parseFloat(sTrim);
        
        case DataType.DOUBLE:
            return Double.parseDouble(sTrim);

        case DataType.DATETIME:
            return (new DateTime(sTrim)).toDateTime(DateTimeZone.UTC);

        case DataType.MAP:
        case DataType.TUPLE:
        case DataType.BAG:
            throw new IllegalArgumentException("Object types (map, tuple, bag) are not supported by FixedWidthLoader");
        
        default:
            throw new IllegalArgumentException(
                "Unknown type in input schema: " + field.getType());
    }
}
开发者ID:sigmoidanalytics，项目名称:spork-streaming，代码行数:52，代码来源:FixedWidthLoader.java


注：本文中的org.apache.pig.ResourceSchema.ResourceFieldSchema.getType方法示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。