當前位置: 首頁>>代碼示例>>Java>>正文


Java Resource.getMemory方法代碼示例

本文整理匯總了Java中org.apache.hadoop.yarn.api.records.Resource.getMemory方法的典型用法代碼示例。如果您正苦於以下問題:Java Resource.getMemory方法的具體用法?Java Resource.getMemory怎麽用?Java Resource.getMemory使用的例子?那麽, 這裏精選的方法代碼示例或許可以為您提供幫助。您也可以進一步了解該方法所在org.apache.hadoop.yarn.api.records.Resource的用法示例。


在下文中一共展示了Resource.getMemory方法的15個代碼示例,這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚,您的評價將有助於係統推薦出更棒的Java代碼示例。

示例1: refreshMaximumAllocation

import org.apache.hadoop.yarn.api.records.Resource; //導入方法依賴的package包/類
protected void refreshMaximumAllocation(Resource newMaxAlloc) {
  maxAllocWriteLock.lock();
  try {
    configuredMaximumAllocation = Resources.clone(newMaxAlloc);
    int maxMemory = newMaxAlloc.getMemory();
    if (maxNodeMemory != -1) {
      maxMemory = Math.min(maxMemory, maxNodeMemory);
    }
    int maxVcores = newMaxAlloc.getVirtualCores();
    if (maxNodeVCores != -1) {
      maxVcores = Math.min(maxVcores, maxNodeVCores);
    }
    int maxGcores = newMaxAlloc.getGpuCores();
    if (maxNodeGCores != -1) {
      maxGcores = Math.min(maxGcores, maxNodeGCores);
    }
    maximumAllocation = Resources.createResource(maxMemory, maxVcores, maxGcores);
  } finally {
    maxAllocWriteLock.unlock();
  }
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:22,代碼來源:AbstractYarnScheduler.java

示例2: getRunningAggregateAppResourceUsage

import org.apache.hadoop.yarn.api.records.Resource; //導入方法依賴的package包/類
synchronized AggregateAppResourceUsage getRunningAggregateAppResourceUsage() {
  long currentTimeMillis = System.currentTimeMillis();
  // Don't walk the whole container list if the resources were computed
  // recently.
  if ((currentTimeMillis - lastMemoryAggregateAllocationUpdateTime)
      > MEM_AGGREGATE_ALLOCATION_CACHE_MSECS) {
    long memorySeconds = 0;
    long vcoreSeconds = 0;
    long gcoreSeconds = 0;
    for (RMContainer rmContainer : this.liveContainers.values()) {
      long usedMillis = currentTimeMillis - rmContainer.getCreationTime();
      Resource resource = rmContainer.getContainer().getResource();
      memorySeconds += resource.getMemory() * usedMillis /  
          DateUtils.MILLIS_PER_SECOND;
      vcoreSeconds += resource.getVirtualCores() * usedMillis  
          / DateUtils.MILLIS_PER_SECOND;
      gcoreSeconds += resource.getGpuCores() * usedMillis / DateUtils.MILLIS_PER_SECOND;
    }

    lastMemoryAggregateAllocationUpdateTime = currentTimeMillis;
    lastMemorySeconds = memorySeconds;
    lastVcoreSeconds = vcoreSeconds;
    lastGcoreSeconds = gcoreSeconds;
  }
  return new AggregateAppResourceUsage(lastMemorySeconds, lastVcoreSeconds, lastGcoreSeconds);
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:27,代碼來源:SchedulerApplicationAttempt.java

示例3: updateAttemptMetrics

import org.apache.hadoop.yarn.api.records.Resource; //導入方法依賴的package包/類
private static void updateAttemptMetrics(RMContainerImpl container) {
  // If this is a preempted container, update preemption metrics
  Resource resource = container.getContainer().getResource();
  RMAppAttempt rmAttempt = container.rmContext.getRMApps()
      .get(container.getApplicationAttemptId().getApplicationId())
      .getCurrentAppAttempt();
  if (ContainerExitStatus.PREEMPTED == container.finishedStatus
    .getExitStatus()) {
    rmAttempt.getRMAppAttemptMetrics().updatePreemptionInfo(resource,
      container);
  }

  if (rmAttempt != null) {
    long usedMillis = container.finishTime - container.creationTime;
    long memorySeconds = resource.getMemory()
                          * usedMillis / DateUtils.MILLIS_PER_SECOND;
    long vcoreSeconds = resource.getVirtualCores()
                         * usedMillis / DateUtils.MILLIS_PER_SECOND;
    long gcoreSeconds = resource.getGpuCores()
                         * usedMillis / DateUtils.MILLIS_PER_SECOND;
    rmAttempt.getRMAppAttemptMetrics()
              .updateAggregateAppResourceUsage(memorySeconds,vcoreSeconds, gcoreSeconds);
  }
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:25,代碼來源:RMContainerImpl.java

示例4: compare

import org.apache.hadoop.yarn.api.records.Resource; //導入方法依賴的package包/類
@Override
public int compare(Resource arg0, Resource arg1) {
  int mem0 = arg0.getMemory();
  int mem1 = arg1.getMemory();
  int cpu0 = arg0.getVirtualCores();
  int cpu1 = arg1.getVirtualCores();
  int gpu0 = arg0.getGpuCores();
  int gpu1 = arg1.getGpuCores();
  if(mem0 == mem1) {
    if(cpu0 == cpu1) {
      if(gpu0 == gpu1) {
        return 0;
      }
      if(gpu0 < gpu1) {
        return 1;
      }
      return -1;
    }
    if(cpu0 < cpu1) {
      return 1;
    }
    return -1;
  }
  if(mem0 < mem1) { 
    return 1;
  }
  return -1;
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:29,代碼來源:AMRMClientImpl.java

示例5: compare

import org.apache.hadoop.yarn.api.records.Resource; //導入方法依賴的package包/類
@Override
public int compare(Resource clusterResource, Resource lhs, Resource rhs) {
  
  if (lhs.equals(rhs)) {
    return 0;
  }

  float[] lValues = new float[] {
    (clusterResource.getMemory() != 0) ? (float) lhs.getMemory() / clusterResource.getMemory() : lhs.getMemory(),
    (clusterResource.getVirtualCores() != 0) ? (float) lhs.getVirtualCores() / clusterResource.getVirtualCores() : lhs.getVirtualCores(),
    (clusterResource.getGpuCores() != 0) ? (float) lhs.getGpuCores() / clusterResource.getGpuCores() : 0.0f };
  Arrays.sort(lValues);

  float[] rValues = new float[] {
    (clusterResource.getMemory() != 0) ? (float) rhs.getMemory() / clusterResource.getMemory() : rhs.getMemory(),
    (clusterResource.getVirtualCores() != 0) ? (float) rhs.getVirtualCores() / clusterResource.getVirtualCores() : rhs.getVirtualCores(),
    (clusterResource.getGpuCores() != 0) ? (float) rhs.getGpuCores() / clusterResource.getGpuCores() : 0.0f };
  Arrays.sort(rValues);

  int diff = 0;
  for(int i = 0; i < 3; i++) {
    float l = lValues[i];
    float r = rValues[i];
    if (l < r) {
      diff = -1;
    } else if (l > r) {
      diff = 1;
    }
  }
  
  return diff;
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:33,代碼來源:DominantResourceCalculator.java

示例6: isInvalidDivisor

import org.apache.hadoop.yarn.api.records.Resource; //導入方法依賴的package包/類
@Override
public boolean isInvalidDivisor(Resource r) {
  if (r.getMemory() == 0.0f || r.getVirtualCores() == 0.0f || r.getGpuCores() == 0.0f) {
    return true;
  }
  return false;
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:8,代碼來源:DominantResourceCalculator.java

示例7: calculateContainerResourceMetrics

import org.apache.hadoop.yarn.api.records.Resource; //導入方法依賴的package包/類
private AggregateAppResourceUsage calculateContainerResourceMetrics(
    RMContainer rmContainer) {
  Resource resource = rmContainer.getContainer().getResource();
  long usedMillis =
      rmContainer.getFinishTime() - rmContainer.getCreationTime();
  long memorySeconds = resource.getMemory()
                        * usedMillis / DateUtils.MILLIS_PER_SECOND;
  long vcoreSeconds = resource.getVirtualCores()
                        * usedMillis / DateUtils.MILLIS_PER_SECOND;
  long gcoreSeconds = resource.getGpuCores()
                        * usedMillis / DateUtils.MILLIS_PER_SECOND;
  return new AggregateAppResourceUsage(memorySeconds, vcoreSeconds, gcoreSeconds);
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:14,代碼來源:TestContainerResourceUsage.java

示例8: computeQueueAbsoluteUsedCapacity

import org.apache.hadoop.yarn.api.records.Resource; //導入方法依賴的package包/類
private float computeQueueAbsoluteUsedCapacity(CSQueue queue, 
    int expectedMemory, Resource clusterResource) {
  return (
      ((float)expectedMemory / (float)clusterResource.getMemory())
    );
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:7,代碼來源:TestParentQueue.java

示例9: checkIfAMResourceUsageOverLimit

import org.apache.hadoop.yarn.api.records.Resource; //導入方法依賴的package包/類
@Override
public boolean checkIfAMResourceUsageOverLimit(Resource usage, Resource maxAMResource) {
  return usage.getMemory() > maxAMResource.getMemory();
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:5,代碼來源:FairSharePolicy.java

示例10: compare

import org.apache.hadoop.yarn.api.records.Resource; //導入方法依賴的package包/類
@Override
public int compare(Resource unused, Resource lhs, Resource rhs) {
  // Only consider memory
  return lhs.getMemory() - rhs.getMemory();
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:6,代碼來源:DefaultResourceCalculator.java

示例11: computeAvailableContainers

import org.apache.hadoop.yarn.api.records.Resource; //導入方法依賴的package包/類
@Override
public int computeAvailableContainers(Resource available, Resource required) {
  // Only consider memory
  return available.getMemory() / required.getMemory();
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:6,代碼來源:DefaultResourceCalculator.java

示例12: isInvalidDivisor

import org.apache.hadoop.yarn.api.records.Resource; //導入方法依賴的package包/類
public boolean isInvalidDivisor(Resource r) {
  if (r.getMemory() == 0.0f) {
    return true;
  }
  return false;
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:7,代碼來源:DefaultResourceCalculator.java

示例13: computeQueueUsedCapacity

import org.apache.hadoop.yarn.api.records.Resource; //導入方法依賴的package包/類
private float computeQueueUsedCapacity(CSQueue queue,
    int expectedMemory, Resource clusterResource) {
  return (expectedMemory / 
      (clusterResource.getMemory() * queue.getAbsoluteCapacity()));
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:6,代碼來源:TestChildQueueOrder.java

示例14: getMaximumAllocationPerQueue

import org.apache.hadoop.yarn.api.records.Resource; //導入方法依賴的package包/類
/**
 * Get the per queue setting for the maximum limit to allocate to
 * each container request.
 *
 * @param queue
 *          name of the queue
 * @return setting specified per queue else falls back to the cluster setting
 */
public Resource getMaximumAllocationPerQueue(String queue) {
  String queuePrefix = getQueuePrefix(queue);
  int maxAllocationMbPerQueue = getInt(queuePrefix + MAXIMUM_ALLOCATION_MB,
      (int)UNDEFINED);
  int maxAllocationVcoresPerQueue = getInt(
      queuePrefix + MAXIMUM_ALLOCATION_VCORES, (int)UNDEFINED);
  int maxAllocationGcoresPerQueue = getInt(
      queuePrefix + MAXIMUM_ALLOCATION_GCORES, (int)UNDEFINED);
  if (LOG.isDebugEnabled()) {
    LOG.debug("max alloc mb per queue for " + queue + " is "
        + maxAllocationMbPerQueue);
    LOG.debug("max alloc vcores per queue for " + queue + " is "
        + maxAllocationVcoresPerQueue);
    LOG.debug("max alloc gcores per queue for " + queue + " is "
        + maxAllocationGcoresPerQueue);
  }
  Resource clusterMax = getMaximumAllocation();
  if (maxAllocationMbPerQueue == (int)UNDEFINED) {
    LOG.info("max alloc mb per queue for " + queue + " is undefined");
    maxAllocationMbPerQueue = clusterMax.getMemory();
  }
  if (maxAllocationVcoresPerQueue == (int)UNDEFINED) {
     LOG.info("max alloc vcore per queue for " + queue + " is undefined");
    maxAllocationVcoresPerQueue = clusterMax.getVirtualCores();
  }
  if (maxAllocationGcoresPerQueue == (int)UNDEFINED) {
    LOG.info("max alloc gcore per queue for " + queue + " is undefined");
    maxAllocationGcoresPerQueue = clusterMax.getGpuCores();
  }
  Resource result = Resources.createResource(maxAllocationMbPerQueue,
      maxAllocationVcoresPerQueue, maxAllocationGcoresPerQueue);
  if (maxAllocationMbPerQueue > clusterMax.getMemory()
      || maxAllocationVcoresPerQueue > clusterMax.getVirtualCores()
      || maxAllocationGcoresPerQueue > clusterMax.getGpuCores()) {
    throw new IllegalArgumentException(
        "Queue maximum allocation cannot be larger than the cluster setting"
        + " for queue " + queue
        + " max allocation per queue: " + result
        + " cluster setting: " + clusterMax);
  }
  return result;
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:51,代碼來源:CapacitySchedulerConfiguration.java

示例15: updateQueueWithNodeUpdate

import org.apache.hadoop.yarn.api.records.Resource; //導入方法依賴的package包/類
private void updateQueueWithNodeUpdate(
        NodeUpdateSchedulerEventWrapper eventWrapper) {
  RMNodeWrapper node = (RMNodeWrapper) eventWrapper.getRMNode();
  List<UpdatedContainerInfo> containerList = node.getContainerUpdates();
  for (UpdatedContainerInfo info : containerList) {
    for (ContainerStatus status : info.getCompletedContainers()) {
      ContainerId containerId = status.getContainerId();
      SchedulerAppReport app = scheduler.getSchedulerAppInfo(
              containerId.getApplicationAttemptId());

      if (app == null) {
        // this happens for the AM container
        // The app have already removed when the NM sends the release
        // information.
        continue;
      }

      String queue =
          appQueueMap.get(containerId.getApplicationAttemptId()
            .getApplicationId());
      int releasedMemory = 0, releasedVCores = 0;
      if (status.getExitStatus() == ContainerExitStatus.SUCCESS) {
        for (RMContainer rmc : app.getLiveContainers()) {
          if (rmc.getContainerId() == containerId) {
            releasedMemory += rmc.getContainer().getResource().getMemory();
            releasedVCores += rmc.getContainer()
                    .getResource().getVirtualCores();
            break;
          }
        }
      } else if (status.getExitStatus() == ContainerExitStatus.ABORTED) {
        if (preemptionContainerMap.containsKey(containerId)) {
          Resource preResource = preemptionContainerMap.get(containerId);
          releasedMemory += preResource.getMemory();
          releasedVCores += preResource.getVirtualCores();
          preemptionContainerMap.remove(containerId);
        }
      }
      // update queue counters
      updateQueueMetrics(queue, releasedMemory, releasedVCores);
    }
  }
}
 
開發者ID:naver,項目名稱:hadoop,代碼行數:44,代碼來源:ResourceSchedulerWrapper.java


注:本文中的org.apache.hadoop.yarn.api.records.Resource.getMemory方法示例由純淨天空整理自Github/MSDocs等開源代碼及文檔管理平台,相關代碼片段篩選自各路編程大神貢獻的開源項目,源碼版權歸原作者所有,傳播和使用請參考對應項目的License;未經允許,請勿轉載。