當前位置: 首頁>>代碼示例>>Java>>正文


Java YarnConfiguration.getStrings方法代碼示例

本文整理匯總了Java中org.apache.hadoop.yarn.conf.YarnConfiguration.getStrings方法的典型用法代碼示例。如果您正苦於以下問題:Java YarnConfiguration.getStrings方法的具體用法?Java YarnConfiguration.getStrings怎麽用?Java YarnConfiguration.getStrings使用的例子?那麽, 這裏精選的方法代碼示例或許可以為您提供幫助。您也可以進一步了解該方法所在org.apache.hadoop.yarn.conf.YarnConfiguration的用法示例。


在下文中一共展示了YarnConfiguration.getStrings方法的1個代碼示例,這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚,您的評價將有助於係統推薦出更棒的Java代碼示例。

示例1: createPreparer

import org.apache.hadoop.yarn.conf.YarnConfiguration; //導入方法依賴的package包/類
protected TwillPreparer createPreparer(YarnConfiguration yarnConfiguration, List<Property> propertyList) {
  BundledJarRunner.Arguments discoveryArgs = new BundledJarRunner.Arguments.Builder()
    .setJarFileName(YARN_BUNDLED_JAR_NAME)
    .setLibFolder("/lib")
    .setMainClassName("com.dremio.dac.daemon.DremioDaemon")
    .setMainArgs(new String[] {})
    .createArguments();

  DacDaemonYarnApplication dacDaemonApp = new DacDaemonYarnApplication(dremioConfig, yarnConfiguration,
    new DacDaemonYarnApplication.Environment());
  File jarFile = new File(dacDaemonApp.getYarnBundledJarName());

  Preconditions.checkState(jarFile.exists());
  Preconditions.checkState(jarFile.canRead());

  TwillRunnerService twillRunner = startTwillRunner(yarnConfiguration);

  Map<String, String> envVars = Maps.newHashMap();
  envVars.put("MALLOC_ARENA_MAX", "4");
  envVars.put("MALLOC_MMAP_THRESHOLD_", "131072");
  envVars.put("MALLOC_TRIM_THRESHOLD_", "131072");
  envVars.put("MALLOC_TOP_PAD_", "131072");
  envVars.put("MALLOC_MMAP_MAX_", "65536");

  // maprfs specific env vars to enable read ahead throttling
  envVars.put("MAPR_IMPALA_RA_THROTTLE", "true");
  envVars.put("MAPR_MAX_RA_STREAMS", "800");

  try {
    String userName = UserGroupInformation.getCurrentUser().getUserName();
    envVars.put("HADOOP_USER_NAME", userName);
  } catch (IOException e) {
    logger.error("Exception while trying to fill out HADOOP_USER_NAME with current user", e);
  }

  String[] yarnClasspath = yarnConfiguration.getStrings(YarnConfiguration.YARN_APPLICATION_CLASSPATH,
    YarnConfiguration.DEFAULT_YARN_APPLICATION_CLASSPATH);
  final TwillPreparer preparer = twillRunner.prepare(dacDaemonApp)
    .addLogHandler(new YarnTwillLogHandler())
    .withApplicationClassPaths(yarnClasspath)
    .withBundlerClassAcceptor(new HadoopClassExcluder())
    .setLogLevels(ImmutableMap.of(Logger.ROOT_LOGGER_NAME, yarnContainerLogLevel()))
    .withEnv(envVars)
    .withMaxRetries(YARN_RUNNABLE_NAME, MAX_APP_RESTART_RETRIES)
    .withArguments(YARN_RUNNABLE_NAME, discoveryArgs.toArray());

  for (String classpathJar : dacDaemonApp.getJarNames()) {
    preparer.withClassPaths(classpathJar);
  }

  preparer.addJVMOptions(prepareCommandOptions(yarnConfiguration, propertyList));

  String queue = yarnConfiguration.get(DacDaemonYarnApplication.YARN_QUEUE_NAME);
  if (queue != null) {
    preparer.setSchedulerQueue(queue);
  }

  if (dremioConfig.getBoolean(DremioConfig.DEBUG_YARN_ENABLED)) {
    preparer.enableDebugging(true, YARN_RUNNABLE_NAME);
  }

  return preparer;
}
 
開發者ID:dremio,項目名稱:dremio-oss,代碼行數:64,代碼來源:YarnController.java


注:本文中的org.apache.hadoop.yarn.conf.YarnConfiguration.getStrings方法示例由純淨天空整理自Github/MSDocs等開源代碼及文檔管理平台,相關代碼片段篩選自各路編程大神貢獻的開源項目,源碼版權歸原作者所有,傳播和使用請參考對應項目的License;未經允許,請勿轉載。