本文整理汇总了Java中parquet.hadoop.ParquetWriter.DEFAULT_PAGE_SIZE属性的典型用法代码示例。如果您正苦于以下问题：Java ParquetWriter.DEFAULT_PAGE_SIZE属性的具体用法？Java ParquetWriter.DEFAULT_PAGE_SIZE怎么用？Java ParquetWriter.DEFAULT_PAGE_SIZE使用的例子？那么恭喜您, 这里精选的属性代码示例或许可以为您提供帮助。您也可以进一步了解该属性所在类parquet.hadoop.ParquetWriter的用法示例。
在下文中一共展示了ParquetWriter.DEFAULT_PAGE_SIZE属性的3个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Java代码示例。
示例1: run
/**
* Write the file.
*
* @param args the command-line arguments
* @return the process exit code
* @throws Exception if something goes wrong
*/
public int run(final String[] args) throws Exception {
Cli cli = Cli.builder().setArgs(args).addOptions(CliCommonOpts.IOFileOpts.values()).build();
int result = cli.runCmd();
if (result != 0) {
return result;
}
File inputFile = new File(cli.getArgValueAsString(CliCommonOpts.IOFileOpts.INPUT));
Path outputPath = new Path(cli.getArgValueAsString(CliCommonOpts.IOFileOpts.OUTPUT));
AvroParquetWriter<Stock> writer =
new AvroParquetWriter<Stock>(outputPath, Stock.SCHEMA$,
CompressionCodecName.SNAPPY,
ParquetWriter.DEFAULT_BLOCK_SIZE,
ParquetWriter.DEFAULT_PAGE_SIZE,
true);
for (Stock stock : AvroStockUtils.fromCsvFile(inputFile)) {
writer.write(stock);
}
writer.close();
return 0;
}
示例2: open
@Override
public void open() {
Preconditions.checkState(state.equals(ReaderWriterState.NEW),
"Unable to open a writer from state:%s", state);
logger.debug(
"Opening data file with pathTmp:{} (final path will be path:{})",
pathTmp, path);
try {
CompressionCodecName codecName = CompressionCodecName.UNCOMPRESSED;
if (enableCompression) {
if (SnappyCodec.isNativeCodeLoaded()) {
codecName = CompressionCodecName.SNAPPY;
} else {
logger.warn("Compression enabled, but Snappy native code not loaded. " +
"Parquet file will not be compressed.");
}
}
avroParquetWriter = new AvroParquetWriter<E>(fileSystem.makeQualified(pathTmp),
schema, codecName, DEFAULT_BLOCK_SIZE,
ParquetWriter.DEFAULT_PAGE_SIZE);
} catch (IOException e) {
throw new DatasetWriterException("Unable to create writer to path:" + pathTmp, e);
}
state = ReaderWriterState.OPEN;
}
示例3: getDefaultPageSize
public String getDefaultPageSize()
{
int defaultPageSize = ParquetWriter.DEFAULT_PAGE_SIZE;
int mbPageSize = defaultPageSize / 1024;
return Integer.toString( mbPageSize );
}