Java Exec.getBufferAllocator方法代码示例

本文整理汇总了Java中org.embulk.spi.Exec.getBufferAllocator方法的典型用法代码示例。如果您正苦于以下问题：Java Exec.getBufferAllocator方法的具体用法？Java Exec.getBufferAllocator怎么用？Java Exec.getBufferAllocator使用的例子？那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类org.embulk.spi.Exec的用法示例。

在下文中一共展示了Exec.getBufferAllocator方法的15个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: run

import org.embulk.spi.Exec; //导入方法依赖的package包/类
@Override
public TaskReport run(TaskSource taskSource,
        Schema schema, int taskIndex,
        PageOutput output)
{
    PluginTask task = taskSource.loadTask(PluginTask.class);
    Integer rows = task.getRows();
    final HashMap<Column, Map<String, Integer>> columnOptions = getColumnOptions(task);
    try (PageBuilder pagebuilder =
            new PageBuilder(Exec.getBufferAllocator(), schema, output)) {
        IntStream.rangeClosed(
                taskIndex * rows + 1,
                taskIndex * rows + rows
        ).boxed().forEach(rowNumber -> {
            RandomjColumnVisitor visitor = new RandomjColumnVisitor(pagebuilder, task, rowNumber, columnOptions);
            schema.visitColumns(visitor);
            pagebuilder.addRecord();
        });
        pagebuilder.finish();
    }

    TaskReport taskReport = Exec.newTaskReport();
    taskReport.set("columns", schema.size());
    taskReport.set("rows", rows);
    return taskReport;
}

开发者ID:yuokada，项目名称:embulk-input-randomj，代码行数:27，代码来源:RandomjInputPlugin.java

示例2: Icu4jPageOutput

import org.embulk.spi.Exec; //导入方法依赖的package包/类
public Icu4jPageOutput(TaskSource taskSource, Schema inputSchema, Schema outputSchema, PageOutput output) {
    this.task = taskSource.loadTask(PluginTask.class);
    this.keyNameColumns = Lists.newArrayList();
    this.transliteratorsList = Lists.newArrayList();
    this.inputSchema = inputSchema;
    this.outputSchema = outputSchema;

    for (String keyName : task.getKeyNames()) {
        keyNameColumns.add(inputSchema.lookupColumn(keyName));
    }
    for (Map<String, String> setting : task.getSettings()) {
        List<Transliterator> tokenizers = Lists.newArrayList();
        for (String convertType : setting.get("transliterators").split(",")) {
            Transliterator transliterator = Transliterator.getInstance(convertType);
            tokenizers.add(transliterator);
        }
        transliteratorsList.add(tokenizers);
    }
    reader = new PageReader(inputSchema);
    builder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
}

开发者ID:toyama0919，项目名称:embulk-filter-icu4j，代码行数:22，代码来源:Icu4jPageOutput.java

示例3: WootheePageOutput

import org.embulk.spi.Exec; //导入方法依赖的package包/类
public WootheePageOutput(TaskSource taskSource, Schema inputSchema, Schema outputSchema, PageOutput output) {
    this.task = taskSource.loadTask(PluginTask.class);
    this.outputColumns = outputSchema.getColumns();
    this.inputColumns = inputSchema.getColumns();
    Map<String, Column> inputColumnMap = Maps.newHashMap();
    this.wootheeColumnMap = Maps.newHashMap();
    for (Column column : outputColumns) {
        if (!inputColumns.contains(column)) {
            wootheeColumnMap.put(column.getName(), column);
        } else {
            inputColumnMap.put(column.getName(), column);
        }
    }
    this.reader = new PageReader(inputSchema);
    this.builder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
    this.keyNameColumn = inputColumnMap.get(task.getKeyName());
}

开发者ID:toyama0919，项目名称:embulk-filter-woothee，代码行数:18，代码来源:WootheePageOutput.java

示例4: JoinFilePageOutput

import org.embulk.spi.Exec; //导入方法依赖的package包/类
JoinFilePageOutput(
        Schema inputSchema,
        Schema outputSchema,
        Column joinBaseColumn,
        HashMap<String, HashMap<String, String>> table,
        List<Column> joinColumns,
        HashMap<String, TimestampParser> timestampParserMap,
        PageOutput pageOutput
        )
{
    this.pageReader = new PageReader(inputSchema);
    this.pageBuilder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, pageOutput);
    this.joinBaseColumn = joinBaseColumn;
    this.table = table;
    this.joinColumns = joinColumns;
    this.timestampParserMap = timestampParserMap;
}

开发者ID:civitaspo，项目名称:embulk-filter-join_file，代码行数:18，代码来源:JoinFilePageOutput.java

示例5: run

import org.embulk.spi.Exec; //导入方法依赖的package包/类
@Override
public void run(TaskSource taskSource, Schema schema,
        FileInput input, PageOutput output)
{
    PluginTask task = taskSource.loadTask(PluginTask.class);
    LineDecoder lineDecoder = new LineDecoder(input,task);
    PageBuilder pageBuilder = new PageBuilder(Exec.getBufferAllocator(), schema, output);
    String line = null;
    final String columnName = task.getColumnName();

    while( input.nextFile() ){
        while(true){
          line = lineDecoder.poll();

          if( line == null ){
              break;
          }

          pageBuilder.setString(0, line);
          pageBuilder.addRecord();
        }
    }
    pageBuilder.finish();
}

开发者ID:sonots，项目名称:embulk-parser-none，代码行数:25，代码来源:NoneParserPlugin.java

示例6: GoogleTranslateApiPageOutput

import org.embulk.spi.Exec; //导入方法依赖的package包/类
public GoogleTranslateApiPageOutput(TaskSource taskSource, Schema inputSchema, Schema outputSchema, PageOutput output) {
    this.task = taskSource.loadTask(PluginTask.class);
    this.outputSchema = outputSchema;
    this.inputColumns = inputSchema.getColumns();
    this.reader = new PageReader(inputSchema);
    this.builder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
    this.keyNameColumns = Lists.newArrayList();
    for (String keyName : task.getKeyNames()) {
        this.keyNameColumns.add(outputSchema.lookupColumn(keyName));
    }
    this.translate = createTranslateService();

    this.srcLang = task.getSourceLang().isPresent() ? TranslateOption.sourceLanguage(task.getSourceLang().get()) : null;
    this.model = task.getModel().isPresent() ? TranslateOption.model(task.getModel().get()) : null;
}

开发者ID:toyama0919，项目名称:embulk-filter-google_translate_api，代码行数:16，代码来源:GoogleTranslateApiPageOutput.java

示例7: open

import org.embulk.spi.Exec; //导入方法依赖的package包/类
@Override
public PageOutput open(TaskSource taskSource, final Schema inputSchema,
        final Schema outputSchema, final PageOutput output)
{
    final PluginTask task = taskSource.loadTask(PluginTask.class);

    return new PageOutput()
    {
        private PageReader pageReader = new PageReader(inputSchema);
        private PageBuilder pageBuilder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
        private BuildJsonVisitorImpl visitor = new BuildJsonVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);

        @Override
        public void finish()
        {
            pageBuilder.finish();
        }

        @Override
        public void close()
        {
            pageBuilder.close();
        }

        @Override
        public void add(Page page)
        {
            pageReader.setPage(page);

            while (pageReader.nextRecord()) {
                outputSchema.visitColumns(visitor);
                pageBuilder.addRecord();
            }
        }
    };
}

开发者ID:hiroyuki-sato，项目名称:embulk-filter-build_json，代码行数:37，代码来源:BuildJsonFilterPlugin.java

示例8: open

import org.embulk.spi.Exec; //导入方法依赖的package包/类
@Override
public PageOutput open(final TaskSource taskSource, final Schema inputSchema,
                       final Schema outputSchema, final PageOutput output)
{
    final PluginTask task = taskSource.loadTask(PluginTask.class);

    return new PageOutput() {
        private PageReader pageReader = new PageReader(inputSchema);
        private PageBuilder pageBuilder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
        private ColumnVisitorImpl visitor = new ColumnVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);

        @Override
        public void finish()
        {
            pageBuilder.finish();
        }

        @Override
        public void close()
        {
            pageBuilder.close();
        }

        @Override
        public void add(Page page)
        {
            pageReader.setPage(page);

            while (pageReader.nextRecord()) {
                inputSchema.visitColumns(visitor);
                pageBuilder.addRecord();
            }
        }
    };
}

开发者ID:sonots，项目名称:embulk-filter-typecast，代码行数:36，代码来源:TypecastFilterPlugin.java

示例9: open

import org.embulk.spi.Exec; //导入方法依赖的package包/类
@Override
public PageOutput open(final TaskSource taskSource, final Schema inputSchema,
                       final Schema outputSchema, final PageOutput output) {

    final PluginTask task = taskSource.loadTask(PluginTask.class);
    final Map<String, List<Base58Column>> base58ColumnMap = convertBase58ColumnListToMap(task.getColumns());
    final Map<String, Column> outputColumnMap = convertColumnListToMap(outputSchema.getColumns(), logger);

    return new PageOutput() {
        private PageReader reader = new PageReader(inputSchema);
        private PageBuilder builder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);

        @Override
        public void add(Page page) {
            reader.setPage(page);
            while (reader.nextRecord()) {
                setValue(base58ColumnMap, outputColumnMap, reader, outputSchema, builder);
                builder.addRecord();
            }
        }

        @Override
        public void finish() {
            builder.finish();
        }

        @Override
        public void close() {
            builder.close();
        }
    };
}

开发者ID:kfitzgerald，项目名称:embulk-filter-base58，代码行数:33，代码来源:Base58FilterPlugin.java

示例10: open

import org.embulk.spi.Exec; //导入方法依赖的package包/类
@Override
public PageOutput open(final TaskSource taskSource, final Schema inputSchema,
        final Schema outputSchema, final PageOutput output)
{
    final PluginTask task = taskSource.loadTask(PluginTask.class);

    return new PageOutput() {
        private PageReader pageReader = new PageReader(inputSchema);
        private PageBuilder pageBuilder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
        private ColumnVisitorImpl visitor = new ColumnVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);

        @Override
        public void finish()
        {
            pageBuilder.finish();
        }

        @Override
        public void close()
        {
            pageBuilder.close();
        }

        @Override
        public void add(Page page)
        {
            pageReader.setPage(page);

            while (pageReader.nextRecord()) {
                inputSchema.visitColumns(visitor);
                pageBuilder.addRecord();
            }
        }
    };
}

开发者ID:sonots，项目名称:embulk-filter-timestamp_format，代码行数:36，代码来源:TimestampFormatFilterPlugin.java

示例11: open

import org.embulk.spi.Exec; //导入方法依赖的package包/类
@Override
public PageOutput open(final TaskSource taskSource, final Schema inputSchema,
        final Schema outputSchema, final PageOutput output)
{
    final PluginTask task = taskSource.loadTask(PluginTask.class);

    return new PageOutput() {
        private PageReader pageReader = new PageReader(inputSchema);
        private PageBuilder pageBuilder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
        private ColumnVisitorImpl visitor = new ColumnVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);

        @Override
        public void finish()
        {
            pageBuilder.finish();
        }

        @Override
        public void close()
        {
            pageBuilder.close();
        }

        @Override
        public void add(Page page)
        {
            pageReader.setPage(page);

            while (pageReader.nextRecord()) {
                outputSchema.visitColumns(visitor);
                pageBuilder.addRecord();
            }
        }
    };
}

开发者ID:sonots，项目名称:embulk-filter-column，代码行数:36，代码来源:ColumnFilterPlugin.java

示例12: FilteredPageOutput

import org.embulk.spi.Exec; //导入方法依赖的package包/类
FilteredPageOutput(PluginTask task, Schema inputSchema, Schema outputSchema, PageOutput pageOutput)
{
    this.stopOnInvalidRecord = task.getStopOnInvalidRecord();
    this.keepExpandingJsonColumn = task.getKeepExpandingJsonColumn();
    this.jsonColumn = initializeJsonColumn(task, inputSchema);
    this.unchangedColumns = initializeUnchangedColumns(inputSchema,
                                                       outputSchema,
                                                       jsonColumn);
    this.expandedColumns = initializeExpandedColumns(task,
                                                     outputSchema);

    this.pageReader = new PageReader(inputSchema);
    this.pageBuilder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, pageOutput);
    this.parseContext = initializeParseContext();
}

开发者ID:civitaspo，项目名称:embulk-filter-expand_json，代码行数:16，代码来源:FilteredPageOutput.java

示例13: DistinctFilterPageOutput

import org.embulk.spi.Exec; //导入方法依赖的package包/类
DistinctFilterPageOutput(PluginTask task, Schema inputSchema,
                         Schema outputSchema, PageOutput pageOutput)
{
    this.pageReader = new PageReader(inputSchema);
    this.pageBuilder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, pageOutput);
    this.visitor = new ColumnVisitorImpl(pageReader, pageBuilder);
    this.outputSchema = outputSchema;
    this.distinctColumns = task.getDistinctColumns();
}

开发者ID:civitaspo，项目名称:embulk-filter-distinct，代码行数:10，代码来源:DistinctFilterPageOutput.java

示例14: newPageBuilder

import org.embulk.spi.Exec; //导入方法依赖的package包/类
private PageBuilder newPageBuilder(Schema schema, PageOutput output)
{
    return new PageBuilder(Exec.getBufferAllocator(), schema, output);
}

开发者ID:CyberAgent，项目名称:embulk-input-parquet_hadoop，代码行数:5，代码来源:ParquetHadoopInputPlugin.java

示例15: run

import org.embulk.spi.Exec; //导入方法依赖的package包/类
@Override
public void run(TaskSource taskSource, Schema schema, FileInput input, PageOutput output)
{
    PluginTask task = taskSource.loadTask(PluginTask.class);

    setColumnNameValues(schema);

    final SchemaConfig schemaConfig = getSchemaConfig(task);
    final TimestampParser[] timestampParsers = Timestamps.newTimestampColumnParsers(task, schemaConfig);
    final LineDecoder decoder = newLineDecoder(input, task);
    final JsonParser jsonParser = newJsonParser();
    final boolean stopOnInvalidRecord = task.getStopOnInvalidRecord();

    try (final PageBuilder pageBuilder = new PageBuilder(Exec.getBufferAllocator(), schema, output)) {
        ColumnVisitorImpl visitor = new ColumnVisitorImpl(task, schema, pageBuilder, timestampParsers);

        while (decoder.nextFile()) { // TODO this implementation should be improved with new JsonParser API on Embulk v0.8.3
            lineNumber = 0;

            while ((line = decoder.poll()) != null) {
                lineNumber++;

                try {
                    Value value = jsonParser.parse(line);

                    if (!value.isMapValue()) {
                        throw new JsonRecordValidateException("Json string is not representing map value.");
                    }

                    final Map<Value, Value> record = value.asMapValue().map();
                    for (Column column : schema.getColumns()) {
                        Value v = record.get(getColumnNameValue(column));
                        visitor.setValue(v);
                        column.visit(visitor);
                    }

                    pageBuilder.addRecord();
                }
                catch (JsonRecordValidateException | JsonParseException e) {
                    if (stopOnInvalidRecord) {
                        throw new DataException(String.format("Invalid record at line %d: %s", lineNumber, line), e);
                    }
                    log.warn(String.format("Skipped line %d (%s): %s", lineNumber, e.getMessage(), line));
                }
            }
        }

        pageBuilder.finish();
    }
}

开发者ID:shun0102，项目名称:embulk-parser-jsonl，代码行数:51，代码来源:JsonlParserPlugin.java

注：本文中的org.embulk.spi.Exec.getBufferAllocator方法示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。