本文整理匯總了Java中org.apache.spark.SparkConf.get方法的典型用法代碼示例。如果您正苦於以下問題：Java SparkConf.get方法的具體用法？Java SparkConf.get怎麽用？Java SparkConf.get使用的例子？那麽, 這裏精選的方法代碼示例或許可以為您提供幫助。您也可以進一步了解該方法所在類org.apache.spark.SparkConf的用法示例。
在下文中一共展示了SparkConf.get方法的2個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Java代碼示例。
示例1: updateLocalConfiguration
import org.apache.spark.SparkConf; //導入方法依賴的package包/類
/**
* When using a persistent context the running Context's configuration will override a passed
* in configuration. Spark allows us to override these inherited properties via
* SparkContext.setLocalProperty
*/
private void updateLocalConfiguration(final JavaSparkContext sparkContext, final SparkConf sparkConfiguration) {
/*
* While we could enumerate over the entire SparkConfiguration and copy into the Thread
* Local properties of the Spark Context this could cause adverse effects with future
* versions of Spark. Since the api for setting multiple local properties at once is
* restricted as private, we will only set those properties we know can effect SparkGraphComputer
* Execution rather than applying the entire configuration.
*/
final String[] validPropertyNames = {
"spark.job.description",
"spark.jobGroup.id",
"spark.job.interruptOnCancel",
"spark.scheduler.pool"
};
for (String propertyName : validPropertyNames) {
if (sparkConfiguration.contains(propertyName)) {
String propertyValue = sparkConfiguration.get(propertyName);
this.logger.info("Setting Thread Local SparkContext Property - "
+ propertyName + " : " + propertyValue);
sparkContext.setLocalProperty(propertyName, sparkConfiguration.get(propertyName));
}
}
}
示例2: ElasticsearchListener
import org.apache.spark.SparkConf; //導入方法依賴的package包/類
/**
* Listener constructor.
*
* Attempts to establish the default mappings in the Elasticsearch index.
*
* @param conf
*/
public ElasticsearchListener(SparkConf conf) {
esHost = conf.get("spark.elasticsearch.host", "localhost");
esPort = conf.get("spark.elasticsearch.port", "9200");
esIndex = conf.get("spark.elasticsearch.index", "spark");
appId = conf.getAppId();
appName = conf.get("spark.app.name", "");
esConnector = new ElasticsearchConnector(esHost, esPort, esIndex);
indexInitialized = esConnector.addDefaultMappings();
if (!indexInitialized) {
LOGGER.warn("Failed to initialize Elasticsearch index '" + esIndex + "' on " + esHost + ":" + esPort);
}
}