当前位置: 首页>>代码示例>>Java>>正文

Java PageBuilder类代码示例

本文整理汇总了Java中org.embulk.spi.PageBuilder类的典型用法代码示例。如果您正苦于以下问题：Java PageBuilder类的具体用法？Java PageBuilder怎么用？Java PageBuilder使用的例子？那么, 这里精选的类代码示例或许可以为您提供帮助。


PageBuilder类属于org.embulk.spi包，在下文中一共展示了PageBuilder类的15个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: run

import org.embulk.spi.PageBuilder; //导入依赖的package包/类
@Override
public TaskReport run(TaskSource taskSource,
        Schema schema, int taskIndex,
        PageOutput output)
{
    PluginTask task = taskSource.loadTask(PluginTask.class);
    Integer rows = task.getRows();
    final HashMap<Column, Map<String, Integer>> columnOptions = getColumnOptions(task);
    try (PageBuilder pagebuilder =
            new PageBuilder(Exec.getBufferAllocator(), schema, output)) {
        IntStream.rangeClosed(
                taskIndex * rows + 1,
                taskIndex * rows + rows
        ).boxed().forEach(rowNumber -> {
            RandomjColumnVisitor visitor = new RandomjColumnVisitor(pagebuilder, task, rowNumber, columnOptions);
            schema.visitColumns(visitor);
            pagebuilder.addRecord();
        });
        pagebuilder.finish();
    }

    TaskReport taskReport = Exec.newTaskReport();
    taskReport.set("columns", schema.size());
    taskReport.set("rows", rows);
    return taskReport;
}
开发者ID:yuokada，项目名称:embulk-input-randomj，代码行数:27，代码来源:RandomjInputPlugin.java

示例2: setValue

import org.embulk.spi.PageBuilder; //导入依赖的package包/类
/**
 * @param builder
 */
private void setValue(PageBuilder builder) {
    for (Column inputColumn: inputColumns) {
        if (reader.isNull(inputColumn)) {
            builder.setNull(inputColumn);
            continue;
        }
        if (Types.STRING.equals(inputColumn.getType())) {
            builder.setString(inputColumn, reader.getString(inputColumn));
        } else if (Types.BOOLEAN.equals(inputColumn.getType())) {
            builder.setBoolean(inputColumn, reader.getBoolean(inputColumn));
        } else if (Types.DOUBLE.equals(inputColumn.getType())) {
            builder.setDouble(inputColumn, reader.getDouble(inputColumn));
        } else if (Types.LONG.equals(inputColumn.getType())) {
            builder.setLong(inputColumn, reader.getLong(inputColumn));
        } else if (Types.TIMESTAMP.equals(inputColumn.getType())) {
            builder.setTimestamp(inputColumn, reader.getTimestamp(inputColumn));
        } else if (Types.JSON.equals(inputColumn.getType())) {
            builder.setJson(inputColumn, reader.getJson(inputColumn));
        }
    }
}
开发者ID:toyama0919，项目名称:embulk-filter-google_translate_api，代码行数:25，代码来源:GoogleTranslateApiPageOutput.java

示例3: filter

import org.embulk.spi.PageBuilder; //导入依赖的package包/类
private List<Object[]> filter(PluginTask task, Schema inputSchema, Object ... objects)
{
    TestPageBuilderReader.MockPageOutput output = new TestPageBuilderReader.MockPageOutput();
    Schema outputSchema = BuildJsonFilterPlugin.buildOutputSchema(task, inputSchema);
    PageBuilder pageBuilder = new PageBuilder(runtime.getBufferAllocator(), outputSchema, output);
    PageReader pageReader = new PageReader(inputSchema);
    BuildJsonVisitorImpl visitor = new BuildJsonVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);

    List<Page> pages = PageTestUtils.buildPage(runtime.getBufferAllocator(), inputSchema, objects);
    for (Page page : pages) {
        pageReader.setPage(page);

        while (pageReader.nextRecord()) {
            outputSchema.visitColumns(visitor);
            pageBuilder.addRecord();
        }
    }
    pageBuilder.finish();
    pageBuilder.close();
    return Pages.toObjects(outputSchema, output.pages);
}
开发者ID:hiroyuki-sato，项目名称:embulk-filter-build_json，代码行数:22，代码来源:TestBuildJsonVisitorImpl.java

示例4: ColumnCaster

import org.embulk.spi.PageBuilder; //导入依赖的package包/类
ColumnCaster(PluginTask task, Schema inputSchema, Schema outputSchema, PageReader pageReader, PageBuilder pageBuilder)
{
    this.task         = task;
    this.inputSchema  = inputSchema;
    this.outputSchema = outputSchema;
    this.pageReader   = pageReader;
    this.pageBuilder  = pageBuilder;

    buildTimestampParserMap();
    buildTimestampFormatterMap();
    buildFromTimestampUnitMap();
    buildToTimestampUnitMap();

    JsonCaster jsonCaster = new JsonCaster(task, timestampParserMap, timestampFormatterMap, fromTimestampUnitMap, toTimestampUnitMap);
    this.jsonVisitor = new JsonVisitor(task, jsonCaster);
}
开发者ID:sonots，项目名称:embulk-filter-timestamp_format，代码行数:17，代码来源:ColumnCaster.java

示例5: testRun

import org.embulk.spi.PageBuilder; //导入依赖的package包/类
@Test
public void testRun() throws IOException
{
    RecordPagingIterable<ObjectNode> mockRecordPagingIterable = Mockito.mock(RecordPagingIterable.class);
    JavaType javaType = OBJECT_MAPPER.getTypeFactory().constructParametrizedType(List.class, List.class, ObjectNode.class);
    List<ObjectNode> objectNodeList = OBJECT_MAPPER.readValue(this.getClass().getResourceAsStream("/fixtures/campaign_response_full.json"), javaType);
    Mockito.when(mockRecordPagingIterable.iterator()).thenReturn(objectNodeList.iterator());
    Mockito.when(mockMarketoRestClient.getCampaign()).thenReturn(mockRecordPagingIterable);
    CampaignInputPlugin.PluginTask task = configSource.loadConfig(CampaignInputPlugin.PluginTask.class);
    ServiceResponseMapper<? extends ValueLocator> mapper = campaignInputPlugin.buildServiceResponseMapper(task);
    RecordImporter recordImporter = mapper.createRecordImporter();
    PageBuilder mockPageBuilder = Mockito.mock(PageBuilder.class);
    campaignInputPlugin.ingestServiceData(task, recordImporter, 1, mockPageBuilder);
    Mockito.verify(mockMarketoRestClient, Mockito.times(1)).getCampaign();
    Schema embulkSchema = mapper.getEmbulkSchema();
    ArgumentCaptor<Long> longArgumentCaptor = ArgumentCaptor.forClass(Long.class);
    Mockito.verify(mockPageBuilder, Mockito.times(10)).setLong(Mockito.eq(embulkSchema.lookupColumn("id")), longArgumentCaptor.capture());
    List<Long> allValues = longArgumentCaptor.getAllValues();
    assertArrayEquals(new Long[]{1003L, 1004L, 1005L, 1006L, 1007L, 1008L, 1029L, 1048L, 1051L, 1065L}, allValues.toArray());
}
开发者ID:treasure-data，项目名称:embulk-input-marketo，代码行数:21，代码来源:CampaignInputPluginTest.java

示例6: Icu4jPageOutput

import org.embulk.spi.PageBuilder; //导入依赖的package包/类
public Icu4jPageOutput(TaskSource taskSource, Schema inputSchema, Schema outputSchema, PageOutput output) {
    this.task = taskSource.loadTask(PluginTask.class);
    this.keyNameColumns = Lists.newArrayList();
    this.transliteratorsList = Lists.newArrayList();
    this.inputSchema = inputSchema;
    this.outputSchema = outputSchema;

    for (String keyName : task.getKeyNames()) {
        keyNameColumns.add(inputSchema.lookupColumn(keyName));
    }
    for (Map<String, String> setting : task.getSettings()) {
        List<Transliterator> tokenizers = Lists.newArrayList();
        for (String convertType : setting.get("transliterators").split(",")) {
            Transliterator transliterator = Transliterator.getInstance(convertType);
            tokenizers.add(transliterator);
        }
        transliteratorsList.add(tokenizers);
    }
    reader = new PageReader(inputSchema);
    builder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
}
开发者ID:toyama0919，项目名称:embulk-filter-icu4j，代码行数:22，代码来源:Icu4jPageOutput.java

示例7: WootheePageOutput

import org.embulk.spi.PageBuilder; //导入依赖的package包/类
public WootheePageOutput(TaskSource taskSource, Schema inputSchema, Schema outputSchema, PageOutput output) {
    this.task = taskSource.loadTask(PluginTask.class);
    this.outputColumns = outputSchema.getColumns();
    this.inputColumns = inputSchema.getColumns();
    Map<String, Column> inputColumnMap = Maps.newHashMap();
    this.wootheeColumnMap = Maps.newHashMap();
    for (Column column : outputColumns) {
        if (!inputColumns.contains(column)) {
            wootheeColumnMap.put(column.getName(), column);
        } else {
            inputColumnMap.put(column.getName(), column);
        }
    }
    this.reader = new PageReader(inputSchema);
    this.builder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
    this.keyNameColumn = inputColumnMap.get(task.getKeyName());
}
开发者ID:toyama0919，项目名称:embulk-filter-woothee，代码行数:18，代码来源:WootheePageOutput.java

示例8: filter

import org.embulk.spi.PageBuilder; //导入依赖的package包/类
private List<Object[]> filter(PluginTask task, Schema inputSchema, Object ... objects)
{
    MockPageOutput output = new MockPageOutput();
    Schema outputSchema = ColumnFilterPlugin.buildOutputSchema(task, inputSchema);
    PageBuilder pageBuilder = new PageBuilder(runtime.getBufferAllocator(), outputSchema, output);
    PageReader pageReader = new PageReader(inputSchema);
    ColumnVisitorImpl visitor = new ColumnVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);

    List<Page> pages = PageTestUtils.buildPage(runtime.getBufferAllocator(), inputSchema, objects);
    for (Page page : pages) {
        pageReader.setPage(page);

        while (pageReader.nextRecord()) {
            outputSchema.visitColumns(visitor);
            pageBuilder.addRecord();
        }
    }
    pageBuilder.finish();
    pageBuilder.close();
    return Pages.toObjects(outputSchema, output.pages);
}
开发者ID:sonots，项目名称:embulk-filter-column，代码行数:22，代码来源:TestColumnVisitorImpl.java

示例9: testOpen

import org.embulk.spi.PageBuilder; //导入依赖的package包/类
@Test
public void testOpen(final @Mocked PageReader reader, final @Mocked PageBuilder builder, final @Mocked Page page) throws Exception {
    new NonStrictExpectations() {{
        taskSource.loadTask(PluginTask.class); result = task;
        task.getDelimiter(); result = "";
        reader.nextRecord(); result = true; result = false;
    }};

    SpeedometerFilterPlugin plugin = new SpeedometerFilterPlugin();
    PageOutput output = plugin.open(taskSource, schema, schema, inPageOutput);
    output.add(page);

    new Verifications() {{
        taskSource.loadTask(PluginTask.class); times = 1;
        builder.addRecord(); times = 1;
        builder.finish(); times = 0;
        reader.nextRecord(); times = 2;
        reader.setPage(page); times = 1;
        schema.visitColumns(withInstanceOf(ColumnVisitor.class)); times = 1;
    }};
}
开发者ID:hata，项目名称:embulk-filter-speedometer，代码行数:22，代码来源:TestSpeedometerFilterPlugin.java

示例10: testFinish

import org.embulk.spi.PageBuilder; //导入依赖的package包/类
@Test
public void testFinish(final @Mocked PageReader reader, final @Mocked PageBuilder builder, final @Mocked Page page) throws Exception {
    new NonStrictExpectations() {{
        taskSource.loadTask(PluginTask.class); result = task;
        task.getDelimiter(); result = "";
    }};

    SpeedometerFilterPlugin plugin = new SpeedometerFilterPlugin();
    PageOutput output = plugin.open(taskSource, schema, schema, inPageOutput);
    output.finish();

    new Verifications() {{
        taskSource.loadTask(PluginTask.class); times = 1;
        builder.finish(); times = 1;
    }};
}
开发者ID:hata，项目名称:embulk-filter-speedometer，代码行数:17，代码来源:TestSpeedometerFilterPlugin.java

示例11: testClose

import org.embulk.spi.PageBuilder; //导入依赖的package包/类
@Test
public void testClose(final @Mocked PageReader reader, final @Mocked PageBuilder builder, final @Mocked Page page) throws Exception {
    new NonStrictExpectations() {{
        taskSource.loadTask(PluginTask.class); result = task;
        task.getDelimiter(); result = "";
    }};

    SpeedometerFilterPlugin plugin = new SpeedometerFilterPlugin();
    PageOutput output = plugin.open(taskSource, schema, schema, inPageOutput);
    output.close();

    new Verifications() {{
        taskSource.loadTask(PluginTask.class); times = 1;
        builder.close(); times = 1;
        reader.close(); times = 1;
    }};
}
开发者ID:hata，项目名称:embulk-filter-speedometer，代码行数:18，代码来源:TestSpeedometerFilterPlugin.java

示例12: JoinFilePageOutput

import org.embulk.spi.PageBuilder; //导入依赖的package包/类
JoinFilePageOutput(
        Schema inputSchema,
        Schema outputSchema,
        Column joinBaseColumn,
        HashMap<String, HashMap<String, String>> table,
        List<Column> joinColumns,
        HashMap<String, TimestampParser> timestampParserMap,
        PageOutput pageOutput
        )
{
    this.pageReader = new PageReader(inputSchema);
    this.pageBuilder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, pageOutput);
    this.joinBaseColumn = joinBaseColumn;
    this.table = table;
    this.joinColumns = joinColumns;
    this.timestampParserMap = timestampParserMap;
}
开发者ID:civitaspo，项目名称:embulk-filter-join_file，代码行数:18，代码来源:JoinFilePageOutput.java

示例13: setInputValues

import org.embulk.spi.PageBuilder; //导入依赖的package包/类
private void setInputValues(PageBuilder pageBuilder) {
    for (Column inputColumn: pageReader.getSchema().getColumns()) {
        if (pageReader.isNull(inputColumn)) {
            pageBuilder.setNull(inputColumn);
            continue;
        }

        if (Types.STRING.equals(inputColumn.getType())) {
            pageBuilder.setString(inputColumn, pageReader.getString(inputColumn));
        }
        else if (Types.BOOLEAN.equals(inputColumn.getType())) {
            pageBuilder.setBoolean(inputColumn, pageReader.getBoolean(inputColumn));
        }
        else if (Types.DOUBLE.equals(inputColumn.getType())) {
            pageBuilder.setDouble(inputColumn, pageReader.getDouble(inputColumn));
        }
        else if (Types.LONG.equals(inputColumn.getType())) {
            pageBuilder.setLong(inputColumn, pageReader.getLong(inputColumn));
        }
        else if (Types.TIMESTAMP.equals(inputColumn.getType())) {
            pageBuilder.setTimestamp(inputColumn, pageReader.getTimestamp(inputColumn));
        }
    }
}
开发者ID:civitaspo，项目名称:embulk-filter-join_file，代码行数:25，代码来源:JoinFilePageOutput.java

示例14: run

import org.embulk.spi.PageBuilder; //导入依赖的package包/类
@Override
public void run(TaskSource taskSource, Schema schema,
        FileInput input, PageOutput output)
{
    PluginTask task = taskSource.loadTask(PluginTask.class);
    LineDecoder lineDecoder = new LineDecoder(input,task);
    PageBuilder pageBuilder = new PageBuilder(Exec.getBufferAllocator(), schema, output);
    String line = null;
    final String columnName = task.getColumnName();

    while( input.nextFile() ){
        while(true){
          line = lineDecoder.poll();

          if( line == null ){
              break;
          }

          pageBuilder.setString(0, line);
          pageBuilder.addRecord();
        }
    }
    pageBuilder.finish();
}
开发者ID:sonots，项目名称:embulk-parser-none，代码行数:25，代码来源:NoneParserPlugin.java

示例15: RandomjColumnVisitor

import org.embulk.spi.PageBuilder; //导入依赖的package包/类
public RandomjColumnVisitor(PageBuilder pageBuilder, PluginTask task, Integer row, Map<Column, Map<String, Integer>> columnOptions)
{
    this.task = task;
    this.pageBuilder = pageBuilder;
    this.row = row;
    this.columnOptions = columnOptions;
    this.rnd = new Random();
}
开发者ID:yuokada，项目名称:embulk-input-randomj，代码行数:9，代码来源:RandomjColumnVisitor.java


注：本文中的org.embulk.spi.PageBuilder类示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。