當前位置: 首頁>>代碼示例>>Java>>正文

Java TaskConfig.setStubWrapper方法代碼示例

本文整理匯總了Java中org.apache.flink.runtime.operators.util.TaskConfig.setStubWrapper方法的典型用法代碼示例。如果您正苦於以下問題：Java TaskConfig.setStubWrapper方法的具體用法？Java TaskConfig.setStubWrapper怎麽用？Java TaskConfig.setStubWrapper使用的例子？那麽, 這裏精選的方法代碼示例或許可以為您提供幫助。您也可以進一步了解該方法所在類org.apache.flink.runtime.operators.util.TaskConfig的用法示例。


在下文中一共展示了TaskConfig.setStubWrapper方法的15個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Java代碼示例。

示例1: registerFileInputTask

import org.apache.flink.runtime.operators.util.TaskConfig; //導入方法依賴的package包/類
public void registerFileInputTask(AbstractInvokable inTask,
		Class<? extends DelimitedInputFormat<Record>> stubClass, String inPath, String delimiter)
{
	DelimitedInputFormat<Record> format;
	try {
		format = stubClass.newInstance();
	}
	catch (Throwable t) {
		throw new RuntimeException("Could not instantiate test input format.", t);
	}

	format.setFilePath(inPath);
	format.setDelimiter(delimiter);

	TaskConfig dsConfig = new TaskConfig(this.mockEnv.getTaskConfiguration());
	dsConfig.setStubWrapper(new UserCodeObjectWrapper<>(format));

	this.inputSplitProvider.addInputSplits(inPath, 5);
}
開發者ID:axbaretto，項目名稱:flink，代碼行數:20，代碼來源:TaskTestBase.java

示例2: createOutput

import org.apache.flink.runtime.operators.util.TaskConfig; //導入方法依賴的package包/類
private static OutputFormatVertex createOutput(JobGraph jobGraph, String resultPath, int numSubTasks,
		TypeSerializerFactory<?> serializer) {
	OutputFormatVertex output = JobGraphUtils.createFileOutput(jobGraph, "Final Output", numSubTasks);
	TaskConfig outputConfig = new TaskConfig(output.getConfiguration());
	{

		outputConfig.addInputToGroup(0);
		outputConfig.setInputSerializer(serializer, 0);

		outputConfig.setStubWrapper(new UserCodeClassWrapper<CsvOutputFormat>(CsvOutputFormat.class));
		outputConfig.setStubParameter(FileOutputFormat.FILE_PARAMETER_KEY, resultPath);

		Configuration outputUserConfig = outputConfig.getStubParameters();
		outputUserConfig.setString(CsvOutputFormat.RECORD_DELIMITER_PARAMETER, "\n");
		outputUserConfig.setString(CsvOutputFormat.FIELD_DELIMITER_PARAMETER, " ");
		outputUserConfig.setClass(CsvOutputFormat.FIELD_TYPE_PARAMETER_PREFIX + 0, LongValue.class);
		outputUserConfig.setInteger(CsvOutputFormat.RECORD_POSITION_PARAMETER_PREFIX + 0, 0);
		outputUserConfig.setClass(CsvOutputFormat.FIELD_TYPE_PARAMETER_PREFIX + 1, LongValue.class);
		outputUserConfig.setInteger(CsvOutputFormat.RECORD_POSITION_PARAMETER_PREFIX + 1, 1);
		outputUserConfig.setInteger(CsvOutputFormat.NUM_FIELDS_PARAMETER, 2);
	}

	return output;
}
開發者ID:citlab，項目名稱:vs.msc.ws14，代碼行數:25，代碼來源:ConnectedComponentsNepheleITCase.java

示例3: createMapper

import org.apache.flink.runtime.operators.util.TaskConfig; //導入方法依賴的package包/類
private static AbstractJobVertex createMapper(JobGraph jobGraph, int numSubTasks, TypeSerializerFactory<?> serializer) {
	AbstractJobVertex pointsInput = JobGraphUtils.createTask(RegularPactTask.class, "Map[DotProducts]", jobGraph, numSubTasks);

	{
		TaskConfig taskConfig = new TaskConfig(pointsInput.getConfiguration());

		taskConfig.setStubWrapper(new UserCodeClassWrapper<DotProducts>(DotProducts.class));
		taskConfig.addOutputShipStrategy(ShipStrategyType.FORWARD);
		taskConfig.setOutputSerializer(serializer);
		taskConfig.setDriver(CollectorMapDriver.class);
		taskConfig.setDriverStrategy(DriverStrategy.COLLECTOR_MAP);

		taskConfig.addInputToGroup(0);
		taskConfig.setInputLocalStrategy(0, LocalStrategy.NONE);
		taskConfig.setInputSerializer(serializer, 0);

		taskConfig.setBroadcastInputName("models", 0);
		taskConfig.addBroadcastInputToGroup(0);
		taskConfig.setBroadcastInputSerializer(serializer, 0);
	}

	return pointsInput;
}
開發者ID:citlab，項目名稱:vs.msc.ws14，代碼行數:24，代碼來源:BroadcastVarsNepheleITCase.java

示例4: createOutput

import org.apache.flink.runtime.operators.util.TaskConfig; //導入方法依賴的package包/類
private static OutputFormatVertex createOutput(JobGraph jobGraph, String resultPath, int numSubTasks, TypeSerializerFactory<?> serializer) {
	OutputFormatVertex output = JobGraphUtils.createFileOutput(jobGraph, "Output", numSubTasks);

	{
		TaskConfig taskConfig = new TaskConfig(output.getConfiguration());
		taskConfig.addInputToGroup(0);
		taskConfig.setInputSerializer(serializer, 0);

		@SuppressWarnings("unchecked")
		CsvOutputFormat outFormat = new CsvOutputFormat("\n", " ", LongValue.class, LongValue.class, LongValue.class);
		outFormat.setOutputFilePath(new Path(resultPath));
		
		taskConfig.setStubWrapper(new UserCodeObjectWrapper<CsvOutputFormat>(outFormat));
	}

	return output;
}
開發者ID:citlab，項目名稱:vs.msc.ws14，代碼行數:18，代碼來源:BroadcastVarsNepheleITCase.java

示例5: createPointsInput

import org.apache.flink.runtime.operators.util.TaskConfig; //導入方法依賴的package包/類
private static InputFormatVertex createPointsInput(JobGraph jobGraph, String pointsPath, int numSubTasks, TypeSerializerFactory<?> serializer) {
	@SuppressWarnings("unchecked")
	CsvInputFormat pointsInFormat = new CsvInputFormat('|', IntValue.class, DoubleValue.class, DoubleValue.class, DoubleValue.class);
	InputFormatVertex pointsInput = JobGraphUtils.createInput(pointsInFormat, pointsPath, "[Points]", jobGraph, numSubTasks);
	{
		TaskConfig taskConfig = new TaskConfig(pointsInput.getConfiguration());
		taskConfig.addOutputShipStrategy(ShipStrategyType.FORWARD);
		taskConfig.setOutputSerializer(serializer);
		
		TaskConfig chainedMapper = new TaskConfig(new Configuration());
		chainedMapper.setDriverStrategy(DriverStrategy.COLLECTOR_MAP);
		chainedMapper.setStubWrapper(new UserCodeObjectWrapper<PointBuilder>(new PointBuilder()));
		chainedMapper.addOutputShipStrategy(ShipStrategyType.FORWARD);
		chainedMapper.setOutputSerializer(serializer);
		
		taskConfig.addChainedTask(ChainedCollectorMapDriver.class, chainedMapper, "Build points");
	}

	return pointsInput;
}
開發者ID:citlab，項目名稱:vs.msc.ws14，代碼行數:21，代碼來源:KMeansIterativeNepheleITCase.java

示例6: createCentersInput

import org.apache.flink.runtime.operators.util.TaskConfig; //導入方法依賴的package包/類
private static InputFormatVertex createCentersInput(JobGraph jobGraph, String centersPath, int numSubTasks, TypeSerializerFactory<?> serializer) {
	@SuppressWarnings("unchecked")
	CsvInputFormat modelsInFormat = new CsvInputFormat('|', IntValue.class, DoubleValue.class, DoubleValue.class, DoubleValue.class);
	InputFormatVertex modelsInput = JobGraphUtils.createInput(modelsInFormat, centersPath, "[Models]", jobGraph, numSubTasks);

	{
		TaskConfig taskConfig = new TaskConfig(modelsInput.getConfiguration());
		taskConfig.addOutputShipStrategy(ShipStrategyType.FORWARD);
		taskConfig.setOutputSerializer(serializer);

		TaskConfig chainedMapper = new TaskConfig(new Configuration());
		chainedMapper.setDriverStrategy(DriverStrategy.COLLECTOR_MAP);
		chainedMapper.setStubWrapper(new UserCodeObjectWrapper<PointBuilder>(new PointBuilder()));
		chainedMapper.addOutputShipStrategy(ShipStrategyType.FORWARD);
		chainedMapper.setOutputSerializer(serializer);
		
		taskConfig.addChainedTask(ChainedCollectorMapDriver.class, chainedMapper, "Build centers");
	}

	return modelsInput;
}
開發者ID:citlab，項目名稱:vs.msc.ws14，代碼行數:22，代碼來源:KMeansIterativeNepheleITCase.java

示例7: createOutput

import org.apache.flink.runtime.operators.util.TaskConfig; //導入方法依賴的package包/類
private static OutputFormatVertex createOutput(JobGraph jobGraph, String resultPath, int numSubTasks, TypeSerializerFactory<?> serializer) {
	
	OutputFormatVertex output = JobGraphUtils.createFileOutput(jobGraph, "Output", numSubTasks);

	{
		TaskConfig taskConfig = new TaskConfig(output.getConfiguration());
		taskConfig.addInputToGroup(0);
		taskConfig.setInputSerializer(serializer, 0);

		PointOutFormat outFormat = new PointOutFormat();
		outFormat.setOutputFilePath(new Path(resultPath));
		
		taskConfig.setStubWrapper(new UserCodeObjectWrapper<PointOutFormat>(outFormat));
	}

	return output;
}
開發者ID:citlab，項目名稱:vs.msc.ws14，代碼行數:18，代碼來源:KMeansIterativeNepheleITCase.java

示例8: registerFileInputTask

import org.apache.flink.runtime.operators.util.TaskConfig; //導入方法依賴的package包/類
public void registerFileInputTask(AbstractInvokable inTask,
		Class<? extends DelimitedInputFormat> stubClass, String inPath, String delimiter)
{
	DelimitedInputFormat format;
	try {
		format = stubClass.newInstance();
	}
	catch (Throwable t) {
		throw new RuntimeException("Could not instantiate test input format.", t);
	}
	
	format.setFilePath(inPath);
	format.setDelimiter(delimiter);
	
	TaskConfig dsConfig = new TaskConfig(this.mockEnv.getTaskConfiguration());
	dsConfig.setStubWrapper(new UserCodeObjectWrapper<DelimitedInputFormat>(format));
	
	this.inputSplitProvider.addInputSplits(inPath, 5);

	inTask.setEnvironment(this.mockEnv);

	inTask.registerInputOutput();
}
開發者ID:citlab，項目名稱:vs.msc.ws14，代碼行數:24，代碼來源:TaskTestBase.java

示例9: registerTask

import org.apache.flink.runtime.operators.util.TaskConfig; //導入方法依賴的package包/類
public void registerTask(
		@SuppressWarnings("rawtypes") Class<? extends Driver> driver,
		Class<? extends RichFunction> stubClass) {

	final TaskConfig config = new TaskConfig(this.mockEnv.getTaskConfiguration());
	config.setDriver(driver);
	config.setStubWrapper(new UserCodeClassWrapper<>(stubClass));
}
開發者ID:axbaretto，項目名稱:flink，代碼行數:9，代碼來源:TaskTestBase.java

示例10: registerFileOutputTask

import org.apache.flink.runtime.operators.util.TaskConfig; //導入方法依賴的package包/類
public void registerFileOutputTask(FileOutputFormat<Record> outputFormat, String outPath) {
	TaskConfig dsConfig = new TaskConfig(this.mockEnv.getTaskConfiguration());

	outputFormat.setOutputFilePath(new Path(outPath));
	outputFormat.setWriteMode(WriteMode.OVERWRITE);

	dsConfig.setStubWrapper(new UserCodeObjectWrapper<>(outputFormat));
}
開發者ID:axbaretto，項目名稱:flink，代碼行數:9，代碼來源:TaskTestBase.java

示例11: createDualInputVertex

import org.apache.flink.runtime.operators.util.TaskConfig; //導入方法依賴的package包/類
private JobVertex createDualInputVertex(DualInputPlanNode node) throws CompilerException {
	final String taskName = node.getNodeName();
	final DriverStrategy ds = node.getDriverStrategy();
	final JobVertex vertex = new JobVertex(taskName);
	final TaskConfig config = new TaskConfig(vertex.getConfiguration());
	vertex.setResources(node.getMinResources(), node.getPreferredResources());
	vertex.setInvokableClass( (this.currentIteration != null && node.isOnDynamicPath()) ? IterationIntermediateTask.class : BatchTask.class);
	
	// set user code
	config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
	config.setStubParameters(node.getProgramOperator().getParameters());
	
	// set the driver strategy
	config.setDriver(ds.getDriverClass());
	config.setDriverStrategy(ds);
	if (node.getComparator1() != null) {
		config.setDriverComparator(node.getComparator1(), 0);
	}
	if (node.getComparator2() != null) {
		config.setDriverComparator(node.getComparator2(), 1);
	}
	if (node.getPairComparator() != null) {
		config.setDriverPairComparator(node.getPairComparator());
	}
	
	// assign memory, file-handles, etc.
	assignDriverResources(node, config);
	return vertex;
}
開發者ID:axbaretto，項目名稱:flink，代碼行數:30，代碼來源:JobGraphGenerator.java

示例12: createDataSourceVertex

import org.apache.flink.runtime.operators.util.TaskConfig; //導入方法依賴的package包/類
private InputFormatVertex createDataSourceVertex(SourcePlanNode node) throws CompilerException {
	final InputFormatVertex vertex = new InputFormatVertex(node.getNodeName());
	final TaskConfig config = new TaskConfig(vertex.getConfiguration());

	vertex.setResources(node.getMinResources(), node.getPreferredResources());
	vertex.setInvokableClass(DataSourceTask.class);
	vertex.setFormatDescription(getDescriptionForUserCode(node.getProgramOperator().getUserCodeWrapper()));

	// set user code
	config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
	config.setStubParameters(node.getProgramOperator().getParameters());

	config.setOutputSerializer(node.getSerializer());
	return vertex;
}
開發者ID:axbaretto，項目名稱:flink，代碼行數:16，代碼來源:JobGraphGenerator.java

示例13: createDataSinkVertex

import org.apache.flink.runtime.operators.util.TaskConfig; //導入方法依賴的package包/類
private JobVertex createDataSinkVertex(SinkPlanNode node) throws CompilerException {
	final OutputFormatVertex vertex = new OutputFormatVertex(node.getNodeName());
	final TaskConfig config = new TaskConfig(vertex.getConfiguration());

	vertex.setResources(node.getMinResources(), node.getPreferredResources());
	vertex.setInvokableClass(DataSinkTask.class);
	vertex.setFormatDescription(getDescriptionForUserCode(node.getProgramOperator().getUserCodeWrapper()));
	
	// set user code
	config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
	config.setStubParameters(node.getProgramOperator().getParameters());

	return vertex;
}
開發者ID:axbaretto，項目名稱:flink，代碼行數:15，代碼來源:JobGraphGenerator.java

示例14: createDualInputVertex

import org.apache.flink.runtime.operators.util.TaskConfig; //導入方法依賴的package包/類
private AbstractJobVertex createDualInputVertex(DualInputPlanNode node) throws CompilerException {
	final String taskName = node.getNodeName();
	final DriverStrategy ds = node.getDriverStrategy();
	final AbstractJobVertex vertex = new AbstractJobVertex(taskName);
	final TaskConfig config = new TaskConfig(vertex.getConfiguration());
	vertex.setInvokableClass( (this.currentIteration != null && node.isOnDynamicPath()) ? IterationIntermediatePactTask.class : RegularPactTask.class);
	
	// set user code
	config.setStubWrapper(node.getPactContract().getUserCodeWrapper());
	config.setStubParameters(node.getPactContract().getParameters());
	
	// set the driver strategy
	config.setDriver(ds.getDriverClass());
	config.setDriverStrategy(ds);
	if (node.getComparator1() != null) {
		config.setDriverComparator(node.getComparator1(), 0);
	}
	if (node.getComparator2() != null) {
		config.setDriverComparator(node.getComparator2(), 1);
	}
	if (node.getPairComparator() != null) {
		config.setDriverPairComparator(node.getPairComparator());
	}
	
	// assign memory, file-handles, etc.
	assignDriverResources(node, config);
	return vertex;
}
開發者ID:citlab，項目名稱:vs.msc.ws14，代碼行數:29，代碼來源:NepheleJobGraphGenerator.java

示例15: createDataSourceVertex

import org.apache.flink.runtime.operators.util.TaskConfig; //導入方法依賴的package包/類
private InputFormatVertex createDataSourceVertex(SourcePlanNode node) throws CompilerException {
	final InputFormatVertex vertex = new InputFormatVertex(node.getNodeName());
	final TaskConfig config = new TaskConfig(vertex.getConfiguration());

	vertex.setInvokableClass(DataSourceTask.class);

	// set user code
	config.setStubWrapper(node.getPactContract().getUserCodeWrapper());
	config.setStubParameters(node.getPactContract().getParameters());

	config.setOutputSerializer(node.getSerializer());
	return vertex;
}
開發者ID:citlab，項目名稱:vs.msc.ws14，代碼行數:14，代碼來源:NepheleJobGraphGenerator.java


注：本文中的org.apache.flink.runtime.operators.util.TaskConfig.setStubWrapper方法示例由純淨天空整理自Github/MSDocs等開源代碼及文檔管理平台，相關代碼片段篩選自各路編程大神貢獻的開源項目，源碼版權歸原作者所有，傳播和使用請參考對應項目的License；未經允許，請勿轉載。