当前位置: 首页>>代码示例>>Java>>正文

Java Schema.visitColumns方法代码示例

本文整理汇总了Java中org.embulk.spi.Schema.visitColumns方法的典型用法代码示例。如果您正苦于以下问题：Java Schema.visitColumns方法的具体用法？Java Schema.visitColumns怎么用？Java Schema.visitColumns使用的例子？那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类org.embulk.spi.Schema的用法示例。


在下文中一共展示了Schema.visitColumns方法的10个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。

示例1: filter

import org.embulk.spi.Schema; //导入方法依赖的package包/类
private List<Object[]> filter(PluginTask task, Schema inputSchema, Object ... objects)
{
    TestPageBuilderReader.MockPageOutput output = new TestPageBuilderReader.MockPageOutput();
    Schema outputSchema = BuildJsonFilterPlugin.buildOutputSchema(task, inputSchema);
    PageBuilder pageBuilder = new PageBuilder(runtime.getBufferAllocator(), outputSchema, output);
    PageReader pageReader = new PageReader(inputSchema);
    BuildJsonVisitorImpl visitor = new BuildJsonVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);

    List<Page> pages = PageTestUtils.buildPage(runtime.getBufferAllocator(), inputSchema, objects);
    for (Page page : pages) {
        pageReader.setPage(page);

        while (pageReader.nextRecord()) {
            outputSchema.visitColumns(visitor);
            pageBuilder.addRecord();
        }
    }
    pageBuilder.finish();
    pageBuilder.close();
    return Pages.toObjects(outputSchema, output.pages);
}
开发者ID:hiroyuki-sato，项目名称:embulk-filter-build_json，代码行数:22，代码来源:TestBuildJsonVisitorImpl.java

示例2: filter

import org.embulk.spi.Schema; //导入方法依赖的package包/类
private List<Object[]> filter(PluginTask task, Schema inputSchema, Object ... objects)
{
    MockPageOutput output = new MockPageOutput();
    Schema outputSchema = ColumnFilterPlugin.buildOutputSchema(task, inputSchema);
    PageBuilder pageBuilder = new PageBuilder(runtime.getBufferAllocator(), outputSchema, output);
    PageReader pageReader = new PageReader(inputSchema);
    ColumnVisitorImpl visitor = new ColumnVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);

    List<Page> pages = PageTestUtils.buildPage(runtime.getBufferAllocator(), inputSchema, objects);
    for (Page page : pages) {
        pageReader.setPage(page);

        while (pageReader.nextRecord()) {
            outputSchema.visitColumns(visitor);
            pageBuilder.addRecord();
        }
    }
    pageBuilder.finish();
    pageBuilder.close();
    return Pages.toObjects(outputSchema, output.pages);
}
开发者ID:sonots，项目名称:embulk-filter-column，代码行数:22，代码来源:TestColumnVisitorImpl.java

示例3: open

import org.embulk.spi.Schema; //导入方法依赖的package包/类
@Override
public PageOutput open(TaskSource taskSource, final Schema inputSchema,
        final Schema outputSchema, final PageOutput output)
{
    final PluginTask task = taskSource.loadTask(PluginTask.class);

    return new PageOutput()
    {
        private PageReader pageReader = new PageReader(inputSchema);
        private PageBuilder pageBuilder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
        private BuildJsonVisitorImpl visitor = new BuildJsonVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);

        @Override
        public void finish()
        {
            pageBuilder.finish();
        }

        @Override
        public void close()
        {
            pageBuilder.close();
        }

        @Override
        public void add(Page page)
        {
            pageReader.setPage(page);

            while (pageReader.nextRecord()) {
                outputSchema.visitColumns(visitor);
                pageBuilder.addRecord();
            }
        }
    };
}
开发者ID:hiroyuki-sato，项目名称:embulk-filter-build_json，代码行数:37，代码来源:BuildJsonFilterPlugin.java

示例4: open

import org.embulk.spi.Schema; //导入方法依赖的package包/类
@Override
public PageOutput open(final TaskSource taskSource, final Schema inputSchema,
                       final Schema outputSchema, final PageOutput output)
{
    final PluginTask task = taskSource.loadTask(PluginTask.class);

    return new PageOutput() {
        private PageReader pageReader = new PageReader(inputSchema);
        private PageBuilder pageBuilder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
        private ColumnVisitorImpl visitor = new ColumnVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);

        @Override
        public void finish()
        {
            pageBuilder.finish();
        }

        @Override
        public void close()
        {
            pageBuilder.close();
        }

        @Override
        public void add(Page page)
        {
            pageReader.setPage(page);

            while (pageReader.nextRecord()) {
                inputSchema.visitColumns(visitor);
                pageBuilder.addRecord();
            }
        }
    };
}
开发者ID:sonots，项目名称:embulk-filter-typecast，代码行数:36，代码来源:TypecastFilterPlugin.java

示例5: open

import org.embulk.spi.Schema; //导入方法依赖的package包/类
@Override
public PageOutput open(final TaskSource taskSource, final Schema inputSchema,
        final Schema outputSchema, final PageOutput output)
{
    final PluginTask task = taskSource.loadTask(PluginTask.class);

    return new PageOutput() {
        private PageReader pageReader = new PageReader(inputSchema);
        private PageBuilder pageBuilder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
        private ColumnVisitorImpl visitor = new ColumnVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);

        @Override
        public void finish()
        {
            pageBuilder.finish();
        }

        @Override
        public void close()
        {
            pageBuilder.close();
        }

        @Override
        public void add(Page page)
        {
            pageReader.setPage(page);

            while (pageReader.nextRecord()) {
                inputSchema.visitColumns(visitor);
                pageBuilder.addRecord();
            }
        }
    };
}
开发者ID:sonots，项目名称:embulk-filter-timestamp_format，代码行数:36，代码来源:TimestampFormatFilterPlugin.java

示例6: open

import org.embulk.spi.Schema; //导入方法依赖的package包/类
@Override
public PageOutput open(final TaskSource taskSource, final Schema inputSchema,
        final Schema outputSchema, final PageOutput output)
{
    final PluginTask task = taskSource.loadTask(PluginTask.class);

    return new PageOutput() {
        private PageReader pageReader = new PageReader(inputSchema);
        private PageBuilder pageBuilder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);
        private ColumnVisitorImpl visitor = new ColumnVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);

        @Override
        public void finish()
        {
            pageBuilder.finish();
        }

        @Override
        public void close()
        {
            pageBuilder.close();
        }

        @Override
        public void add(Page page)
        {
            pageReader.setPage(page);

            while (pageReader.nextRecord()) {
                outputSchema.visitColumns(visitor);
                pageBuilder.addRecord();
            }
        }
    };
}
开发者ID:sonots，项目名称:embulk-filter-column，代码行数:36，代码来源:ColumnFilterPlugin.java

示例7: open

import org.embulk.spi.Schema; //导入方法依赖的package包/类
@Override
public PageOutput open(TaskSource taskSource, final Schema schema, FileOutput output) {
	final PluginTask task = taskSource.loadTask(PluginTask.class);

	final Sheet sheet = newWorkbook(task);

	final FileOutputOutputStream stream = new FileOutputOutputStream(output, task.getBufferAllocator(),
			CloseMode.CLOSE);
	stream.nextFile();

	return new PageOutput() {
		private final PageReader pageReader = new PageReader(schema);

		@Override
		public void add(Page page) {
			pageReader.setPage(page);
			PoiExcelColumnVisitor visitor = new PoiExcelColumnVisitor(task, schema, sheet, pageReader);
			while (pageReader.nextRecord()) {
				schema.visitColumns(visitor);
				visitor.endRecord();
			}
		}

		@Override
		public void finish() {
			Workbook book = sheet.getWorkbook();
			try (FileOutputOutputStream os = stream) {
				book.write(os);
				os.finish();
			} catch (IOException e) {
				throw new RuntimeException(e);
			}
		}

		@Override
		public void close() {
			stream.close();
		}
	};
}
开发者ID:hishidama，项目名称:embulk-formatter-poi_excel，代码行数:41，代码来源:PoiExcelFormatterPlugin.java

示例8: convertSchema

import org.embulk.spi.Schema; //导入方法依赖的package包/类
private MessageType convertSchema(Schema schema)
{
    SchemaConvertColumnVisitor visitor = new SchemaConvertColumnVisitor();
    schema.visitColumns(visitor);
    String messageName = "embulk";
    return new MessageType(messageName, visitor.getConvertedFields());
}
开发者ID:choplin，项目名称:embulk-output-parquet，代码行数:8，代码来源:EmbulkWriteSupport.java

示例9: open

import org.embulk.spi.Schema; //导入方法依赖的package包/类
@Override
public PageOutput open(final TaskSource taskSource, final Schema inputSchema,
        final Schema outputSchema, final PageOutput output)
{
    final PluginTask task = taskSource.loadTask(PluginTask.class);
    final boolean orCondition = task.getCondition().toLowerCase().equals("or");
    final PageReader pageReader = new PageReader(inputSchema);
    final PageBuilder pageBuilder = new PageBuilder(Exec.getBufferAllocator(), outputSchema, output);

    final AbstractGuardColumnVisitor guradVisitor;
    if (task.getWhere().isPresent()) {
        ParserExp parserExp = new Parser(inputSchema).parse(task.getWhere().get());
        guradVisitor = new GuardColumnVisitorWhereImpl(task, inputSchema, outputSchema, pageReader, parserExp);
    }
    else if (orCondition) {
        guradVisitor = new GuardColumnVisitorOrImpl(task, inputSchema, outputSchema, pageReader);
    }
    else {
        guradVisitor = new GuardColumnVisitorAndImpl(task, inputSchema, outputSchema, pageReader);
    }

    final BuilderColumnVisitorImpl builderVisitor;
    builderVisitor = new BuilderColumnVisitorImpl(task, inputSchema, outputSchema, pageReader, pageBuilder);

    return new PageOutput() {
        @Override
        public void finish()
        {
            pageBuilder.finish();
        }

        @Override
        public void close()
        {
            pageBuilder.close();
        }

        @Override
        public void add(Page page)
        {
            pageReader.setPage(page);

            while (pageReader.nextRecord()) {
                if (guradVisitor.visitColumns(inputSchema)) {
                    // output.add(page); did not work, double release() error occurred. We need to copy from reader to builder...
                    outputSchema.visitColumns(builderVisitor);
                    pageBuilder.addRecord();
                }
            }
        }
    };
}
开发者ID:sonots，项目名称:embulk-filter-row，代码行数:53，代码来源:RowFilterPlugin.java

示例10: run

import org.embulk.spi.Schema; //导入方法依赖的package包/类
protected void run(PluginTask task, Schema schema, Workbook workbook, List<String> sheetNames, PageOutput output) {
	final int flushCount = task.getFlushCount();

	try (PageBuilder pageBuilder = new PageBuilder(Exec.getBufferAllocator(), schema, output)) {
		for (String sheetName : sheetNames) {
			Sheet sheet = workbook.getSheet(sheetName);
			if (sheet == null) {
				if (task.getIgnoreSheetNotFound()) {
					log.info("ignore: not found sheet={}", sheetName);
					continue;
				} else {
					throw new RuntimeException(MessageFormat.format("not found sheet={0}", sheetName));
				}
			}

			log.info("sheet={}", sheetName);
			PoiExcelVisitorFactory factory = newPoiExcelVisitorFactory(task, schema, sheet, pageBuilder);
			PoiExcelColumnVisitor visitor = factory.getPoiExcelColumnVisitor();
			final int skipHeaderLines = factory.getVisitorValue().getSheetBean().getSkipHeaderLines();

			int count = 0;
			for (Row row : sheet) {
				int rowIndex = row.getRowNum();
				if (rowIndex < skipHeaderLines) {
					log.debug("row({}) skipped", rowIndex);
					continue;
				}
				if (log.isDebugEnabled()) {
					log.debug("row({}) start", rowIndex);
				}

				visitor.setRow(row);
				schema.visitColumns(visitor);
				pageBuilder.addRecord();

				if (++count >= flushCount) {
					log.trace("flush");
					pageBuilder.flush();
					count = 0;
				}

				if (log.isDebugEnabled()) {
					log.debug("row({}) end", rowIndex);
				}
			}
			pageBuilder.flush();
		}
		pageBuilder.finish();
	}
}
开发者ID:hishidama，项目名称:embulk-parser-poi_excel，代码行数:51，代码来源:PoiExcelParserPlugin.java


注：本文中的org.embulk.spi.Schema.visitColumns方法示例由纯净天空整理自Github/MSDocs等开源代码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。