当前位置: 首页>>代码示例>>Java>>正文

Java Column类代码示例

本文整理汇总了Java中org.embulk.spi.Column类的典型用法代码示例。如果您正苦于以下问题：Java Column类的具体用法？Java Column怎么用？Java Column使用的例子？那么, 这里精选的类代码示例或许可以为您提供帮助。


Column类属于org.embulk.spi包，在下文中一共展示了Column类的15个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: booleanColumn

import org.embulk.spi.Column; //导入依赖的package包/类
@Override
public void booleanColumn(Column column)
{
    Integer nrate = columnOptions.get(column).get("null_rate");
    if (Math.random() < (double) nrate / 10000) {
        pageBuilder.setNull(column);
    }
    else {
        if (Math.random() < 0.5) {
            pageBuilder.setBoolean(column, false);
        }
        else {
            pageBuilder.setBoolean(column, true);
        }
    }
}
开发者ID:yuokada，项目名称:embulk-input-randomj，代码行数:17，代码来源:RandomjColumnVisitor.java

示例2: stringColumn

import org.embulk.spi.Column; //导入依赖的package包/类
@Override
public void stringColumn(Column column)
{
    Integer nrate = columnOptions.get(column).get("null_rate");
    if (Math.random() < (double) nrate / 10000) {
        pageBuilder.setNull(column);
    }
    else {
        final Integer length = columnOptions.get(column).getOrDefault("length", 0);
        if (length == 0) {
            pageBuilder.setString(column, generator.generate(32));
        }
        else {
            pageBuilder.setString(column, generator.generate(length));
        }
    }
}
开发者ID:yuokada，项目名称:embulk-input-randomj，代码行数:18，代码来源:RandomjColumnVisitor.java

示例3: timestampColumn

import org.embulk.spi.Column; //导入依赖的package包/类
@Override
public void timestampColumn(Column column)
{
    Integer nrate = columnOptions.get(column).get("null_rate");
    if (Math.random() < (double) nrate / 10000) {
        pageBuilder.setNull(column);
    }
    else {
        final double randd = Math.random();
        LocalDateTime randomDate = LocalDateTime.now()
                .plusDays((long) (randd * 100))
                .plusSeconds((long) (randd * 1000000));
        Timestamp timestamp = Timestamp.ofEpochSecond(
                randomDate.atZone(zoneId).toEpochSecond()
        );
        pageBuilder.setTimestamp(column, timestamp);
    }
}
开发者ID:yuokada，项目名称:embulk-input-randomj，代码行数:19，代码来源:RandomjColumnVisitor.java

示例4: run

import org.embulk.spi.Column; //导入依赖的package包/类
@Override
public TaskReport run(TaskSource taskSource,
        Schema schema, int taskIndex,
        PageOutput output)
{
    PluginTask task = taskSource.loadTask(PluginTask.class);
    Integer rows = task.getRows();
    final HashMap<Column, Map<String, Integer>> columnOptions = getColumnOptions(task);
    try (PageBuilder pagebuilder =
            new PageBuilder(Exec.getBufferAllocator(), schema, output)) {
        IntStream.rangeClosed(
                taskIndex * rows + 1,
                taskIndex * rows + rows
        ).boxed().forEach(rowNumber -> {
            RandomjColumnVisitor visitor = new RandomjColumnVisitor(pagebuilder, task, rowNumber, columnOptions);
            schema.visitColumns(visitor);
            pagebuilder.addRecord();
        });
        pagebuilder.finish();
    }

    TaskReport taskReport = Exec.newTaskReport();
    taskReport.set("columns", schema.size());
    taskReport.set("rows", rows);
    return taskReport;
}
开发者ID:yuokada，项目名称:embulk-input-randomj，代码行数:27，代码来源:RandomjInputPlugin.java

示例5: getColumnOptions

import org.embulk.spi.Column; //导入依赖的package包/类
HashMap<Column, Map<String, Integer>> getColumnOptions(PluginTask task)
{
    SchemaConfig schemaConfig = task.getSchema();
    Schema schema = schemaConfig.toSchema();
    HashMap<Column, Map<String, Integer>> lengthMap = new HashMap<>();
    for (Column column : schema.getColumns()) {
        HashMap<String, Integer> miniMap = new HashMap<>();
        ColumnConfig c = schemaConfig.getColumn(column.getIndex());
        miniMap.put("length", c.getOption().get(Integer.class, "length", 0));
        miniMap.put("null_rate", c.getOption().get(Integer.class, "null_rate", 0));
        miniMap.put("max_value", c.getOption().get(Integer.class, "max_value", null));
        miniMap.put("min_value", c.getOption().get(Integer.class, "min_value", null));
        lengthMap.put(column, miniMap);
    }
    return lengthMap;
}
开发者ID:yuokada，项目名称:embulk-input-randomj，代码行数:17，代码来源:RandomjInputPlugin.java

示例6: add

import org.embulk.spi.Column; //导入依赖的package包/类
@Override
public void add(Page page) {
    reader.setPage(page);

    try {
        while (reader.nextRecord()) {
            List<String> sourceTexts = Lists.newArrayList();
            for (Column keyNameColumn : keyNameColumns) {
                String text = reader.isNull(keyNameColumn) ? "" : reader.getString(keyNameColumn);
                logger.debug(text);
                sourceTexts.add(text);
            }
            List<Translation> translations = translate(sourceTexts);
            for (int i = 0; i < keyNameColumns.size(); i++) {
                builder.setString(outputSchema.lookupColumn(keyNameColumns.get(i).getName() + task.getOutKeyNameSuffix()), translations.get(i).getTranslatedText());
            }
            setValue(builder);
            builder.addRecord();
            Thread.sleep(task.getSleep().get());
        }
    } catch (Exception  e) {
        throw new RuntimeException(e);
    } 
}
开发者ID:toyama0919，项目名称:embulk-filter-google_translate_api，代码行数:25，代码来源:GoogleTranslateApiPageOutput.java

示例7: setValue

import org.embulk.spi.Column; //导入依赖的package包/类
/**
 * @param builder
 */
private void setValue(PageBuilder builder) {
    for (Column inputColumn: inputColumns) {
        if (reader.isNull(inputColumn)) {
            builder.setNull(inputColumn);
            continue;
        }
        if (Types.STRING.equals(inputColumn.getType())) {
            builder.setString(inputColumn, reader.getString(inputColumn));
        } else if (Types.BOOLEAN.equals(inputColumn.getType())) {
            builder.setBoolean(inputColumn, reader.getBoolean(inputColumn));
        } else if (Types.DOUBLE.equals(inputColumn.getType())) {
            builder.setDouble(inputColumn, reader.getDouble(inputColumn));
        } else if (Types.LONG.equals(inputColumn.getType())) {
            builder.setLong(inputColumn, reader.getLong(inputColumn));
        } else if (Types.TIMESTAMP.equals(inputColumn.getType())) {
            builder.setTimestamp(inputColumn, reader.getTimestamp(inputColumn));
        } else if (Types.JSON.equals(inputColumn.getType())) {
            builder.setJson(inputColumn, reader.getJson(inputColumn));
        }
    }
}
开发者ID:toyama0919，项目名称:embulk-filter-google_translate_api，代码行数:25，代码来源:GoogleTranslateApiPageOutput.java

示例8: setFromLong

import org.embulk.spi.Column; //导入依赖的package包/类
public void setFromLong(Column outputColumn, long value)
{
    Type outputType = outputColumn.getType();
    TimestampUnit fromUnit = fromTimestampUnitMap.get(outputColumn.getName());
    if (outputType instanceof StringType) {
        TimestampFormatter timestampFormatter = timestampFormatterMap.get(outputColumn.getName());
        pageBuilder.setString(outputColumn, LongCast.asString(value, fromUnit, timestampFormatter));
    }
    else if (outputType instanceof TimestampType) {
        pageBuilder.setTimestamp(outputColumn, LongCast.asTimestamp(value, fromUnit));
    }
    else if (outputType instanceof LongType) {
        TimestampUnit toUnit = toTimestampUnitMap.get(outputColumn.getName());
        pageBuilder.setLong(outputColumn, LongCast.asLong(value, fromUnit, toUnit));
    }
    else if (outputType instanceof DoubleType) {
        TimestampUnit toUnit = toTimestampUnitMap.get(outputColumn.getName());
        pageBuilder.setDouble(outputColumn, LongCast.asDouble(value, fromUnit, toUnit));
    }
    else {
        assert false;
    }
}
开发者ID:sonots，项目名称:embulk-filter-timestamp_format，代码行数:24，代码来源:ColumnCaster.java

示例9: buildOuputSchema

import org.embulk.spi.Column; //导入依赖的package包/类
private Schema buildOuputSchema(final PluginTask task, final Schema inputSchema)
{
    List<ColumnConfig> columnConfigs = task.getColumns();
    ImmutableList.Builder<Column> builder = ImmutableList.builder();
    int i = 0;
    for (Column inputColumn : inputSchema.getColumns()) {
        String name = inputColumn.getName();
        Type   type = inputColumn.getType();
        ColumnConfig columnConfig = getColumnConfig(name, columnConfigs);
        if (columnConfig != null) {
            type = columnConfig.getType();
        }
        Column outputColumn = new Column(i++, name, type);
        builder.add(outputColumn);
    }
    return new Schema(builder.build());
}
开发者ID:sonots，项目名称:embulk-filter-timestamp_format，代码行数:18，代码来源:TimestampFormatFilterPlugin.java

示例10: buildOutputSchemaDecodePrefix_Columns

import org.embulk.spi.Column; //导入依赖的package包/类
@Test
public void buildOutputSchemaDecodePrefix_Columns()
{
    PluginTask task = taskFromYamlString(
            "type: base58",
            "columns:",
            "  - {name: public_id, encode: false, prefix: obj_}");
    Schema inputSchema = Schema.builder()
            .add("public_id", STRING)
            .build();

    Schema outputSchema = plugin.buildOutputSchema(task, inputSchema);
    assertEquals(1, outputSchema.size());

    Column column;
    {
        column = outputSchema.getColumn(0);
        assertEquals("public_id", column.getName());
    }
}
开发者ID:kfitzgerald，项目名称:embulk-filter-base58，代码行数:21，代码来源:TestBase58FilterPlugin.java

示例11: stringColumn

import org.embulk.spi.Column; //导入依赖的package包/类
@Override
public void stringColumn(final Column inputColumn)
{
    String name = inputColumn.getName();
    if (! shouldCast(name)){
        if (pageReader.isNull(inputColumn)) {
            pageBuilder.setNull(inputColumn);
        }
        else {
            pageBuilder.setString(inputColumn, pageReader.getString(inputColumn));
        }
    }
    else {
        final Column outputColumn = outputColumnMap.get(inputColumn.getName());
        PageBuildable op = new PageBuildable() {
            public void run() throws DataException {
                columnCaster.setFromString(outputColumn, pageReader.getString(inputColumn));
            }
        };
        withStopOnInvalidRecord(op, inputColumn, outputColumn);
    }
}
开发者ID:sonots，项目名称:embulk-filter-timestamp_format，代码行数:23，代码来源:ColumnVisitorImpl.java

示例12: newColumnGetter

import org.embulk.spi.Column; //导入依赖的package包/类
public BaseColumnGetter newColumnGetter(Column column)
{
    org.apache.avro.Schema fieldSchema = avroSchema.getField(column.getName()).schema();
    switch (fieldSchema.getType()) {
        case UNION:
            Schema.Type type = null;
            for (org.apache.avro.Schema sc : fieldSchema.getTypes()) {
                if (sc.getType() != Schema.Type.NULL) {
                    type = sc.getType();
                    break;
                }
            }
            return getColumnGetterFromTypeName(type);
        default :
            return getColumnGetterFromTypeName(fieldSchema.getType());
    }
}
开发者ID:joker1007，项目名称:embulk-parser-avro，代码行数:18，代码来源:ColumnGetterFactory.java

示例13: buildOutputSchema_Columns

import org.embulk.spi.Column; //导入依赖的package包/类
@Test
public void buildOutputSchema_Columns()
{
    PluginTask task = taskFromYamlString(
            "type: base58",
            "columns:",
            "  - {name: _id}");
    Schema inputSchema = Schema.builder()
            .add("_id", STRING)
            .build();

    Schema outputSchema = plugin.buildOutputSchema(task, inputSchema);
    assertEquals(1, outputSchema.size());

    Column column;
    {
        column = outputSchema.getColumn(0);
        assertEquals("_id", column.getName());
    }
}
开发者ID:kfitzgerald，项目名称:embulk-filter-base58，代码行数:21，代码来源:TestBase58FilterPlugin.java

示例14: longColumn

import org.embulk.spi.Column; //导入依赖的package包/类
@Override
public void longColumn(final Column inputColumn)
{
    String name = inputColumn.getName();
    if (! shouldCast(name)){
        if (pageReader.isNull(inputColumn)) {
            pageBuilder.setNull(inputColumn);
        }
        else {
            pageBuilder.setLong(inputColumn, pageReader.getLong(inputColumn));
        }
    }
    else {
        final Column outputColumn = outputColumnMap.get(name);
        PageBuildable op = new PageBuildable() {
            public void run() throws DataException {
                columnCaster.setFromLong(outputColumn, pageReader.getLong(inputColumn));
            }
        };
        withStopOnInvalidRecord(op, inputColumn, outputColumn);
    }
}
开发者ID:sonots，项目名称:embulk-filter-timestamp_format，代码行数:23，代码来源:ColumnVisitorImpl.java

示例15: setFromTimestamp

import org.embulk.spi.Column; //导入依赖的package包/类
public void setFromTimestamp(Column outputColumn, Timestamp value)
{
    Type outputType = outputColumn.getType();
    if (outputType instanceof StringType) {
        TimestampFormatter timestampFormatter = timestampFormatterMap.get(outputColumn.getName());
        pageBuilder.setString(outputColumn, TimestampCast.asString(value, timestampFormatter));
    }
    else if (outputType instanceof TimestampType) {
        pageBuilder.setTimestamp(outputColumn, value);
    }
    else if (outputType instanceof LongType) {
        TimestampUnit toUnit = toTimestampUnitMap.get(outputColumn.getName());
        pageBuilder.setLong(outputColumn, TimestampCast.asLong(value, toUnit));
    }
    else if (outputType instanceof DoubleType) {
        TimestampUnit toUnit = toTimestampUnitMap.get(outputColumn.getName());
        pageBuilder.setDouble(outputColumn, TimestampCast.asDouble(value, toUnit));
    }
    else {
        assert false;
    }
}
开发者ID:sonots，项目名称:embulk-filter-timestamp_format，代码行数:23，代码来源:ColumnCaster.java


注：本文中的org.embulk.spi.Column类示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。