org.apache.hadoop.yarn.util.resource

## Class DefaultResourceCalculator

- java.lang.Object
- - org.apache.hadoop.yarn.util.resource.ResourceCalculator
  - - org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator

-     @InterfaceAudience.Private
       @InterfaceStability.Unstable
      public class DefaultResourceCalculator
      extends ResourceCalculator

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `DefaultResourceCalculator()`

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `int` | `compare(org.apache.hadoop.yarn.api.records.Resource unused, org.apache.hadoop.yarn.api.records.Resource lhs, org.apache.hadoop.yarn.api.records.Resource rhs, boolean singleType)` On a cluster with capacity `clusterResource`, compare `lhs` and `rhs`.
    `long` | `computeAvailableContainers(org.apache.hadoop.yarn.api.records.Resource available, org.apache.hadoop.yarn.api.records.Resource required)` Compute the number of containers which can be allocated given `available` and `required` resources.
    `float` | `divide(org.apache.hadoop.yarn.api.records.Resource unused, org.apache.hadoop.yarn.api.records.Resource numerator, org.apache.hadoop.yarn.api.records.Resource denominator)` Divide resource `numerator` by resource `denominator` using specified policy (domination, average, fairness etc.); hence overall `clusterResource` is provided for context.
    `org.apache.hadoop.yarn.api.records.Resource` | `divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator, float denominator)` Divide-and-ceil `numerator` by `denominator`.
    `org.apache.hadoop.yarn.api.records.Resource` | `divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator, int denominator)` Divide-and-ceil `numerator` by `denominator`.
    `boolean` | `fitsIn(org.apache.hadoop.yarn.api.records.Resource smaller, org.apache.hadoop.yarn.api.records.Resource bigger)` Check if a smaller resource can be contained by bigger resource.
    `boolean` | `isAllInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)` Determine if all resources are zero.
    `boolean` | `isAnyMajorResourceAboveZero(org.apache.hadoop.yarn.api.records.Resource resource)` Check if resource has any major resource types (which are all NodeManagers included) has a > 0 value.
    `boolean` | `isAnyMajorResourceZeroOrNegative(org.apache.hadoop.yarn.api.records.Resource resource)` Check if resource has any major resource types (which are all NodeManagers included) a zero value or negative value.
    `boolean` | `isInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)` Determine if a resource is not suitable for use as a divisor (will result in divide by 0, etc)
    `org.apache.hadoop.yarn.api.records.Resource` | `multiplyAndNormalizeDown(org.apache.hadoop.yarn.api.records.Resource r, double by, org.apache.hadoop.yarn.api.records.Resource stepFactor)` Multiply resource `r` by factor `by` and normalize down using step-factor `stepFactor`.
    `org.apache.hadoop.yarn.api.records.Resource` | `multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r, double[] by, org.apache.hadoop.yarn.api.records.Resource stepFactor)` Multiply resource `r` by factor `by` and normalize up using step-factor `stepFactor`.
    `org.apache.hadoop.yarn.api.records.Resource` | `multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r, double by, org.apache.hadoop.yarn.api.records.Resource stepFactor)` Multiply resource `r` by factor `by` and normalize up using step-factor `stepFactor`.
    `org.apache.hadoop.yarn.api.records.Resource` | `normalize(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource minimumResource, org.apache.hadoop.yarn.api.records.Resource maximumResource, org.apache.hadoop.yarn.api.records.Resource stepFactor)` Normalize resource `r` given the base `minimumResource` and verify against max allowed `maximumResource` using a step factor for the normalization.
    `org.apache.hadoop.yarn.api.records.Resource` | `normalizeDown(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource stepFactor)` Get resource `r`and normalize down using step-factor `stepFactor`.
    `float` | `ratio(org.apache.hadoop.yarn.api.records.Resource a, org.apache.hadoop.yarn.api.records.Resource b)` Ratio of resource `a` to resource `b`.
    `org.apache.hadoop.yarn.api.records.Resource` | `roundDown(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource stepFactor)` Round-down resource `r` given factor `stepFactor`.
    `org.apache.hadoop.yarn.api.records.Resource` | `roundUp(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource stepFactor)` Round-up resource `r` given factor `stepFactor`.

    - ### Methods inherited from class org.apache.hadoop.yarn.util.resource.ResourceCalculator

      `compare, divideAndCeil, divideAndCeil, divideAndCeil, divideAndCeil, roundDown, roundDown, roundUp, roundUp`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### DefaultResourceCalculator

          public DefaultResourceCalculator()

  - ### Method Detail

    - #### compare

          public int compare(org.apache.hadoop.yarn.api.records.Resource unused,
                             org.apache.hadoop.yarn.api.records.Resource lhs,
                             org.apache.hadoop.yarn.api.records.Resource rhs,
                             boolean singleType)

      Description copied from class: `ResourceCalculator`
      On a cluster with capacity `clusterResource`, compare `lhs` and `rhs`. Consider all resources unless `singleType` is set to true. When `singleType` is set to true, consider only one resource as per the `ResourceCalculator` implementation; the `DefaultResourceCalculator` considers memory and `DominantResourceCalculator` considers the dominant resource.
      Specified by:
        `compare` in class `ResourceCalculator`
      Parameters:
        `unused` - cluster capacity
        `lhs` - First `Resource` to compare
        `rhs` - Second `Resource` to compare
        `singleType` - Whether to consider a single resource type or all resource types
      Returns:
        -1 if `lhs` is smaller, 0 if equal and 1 if it is larger

    - #### computeAvailableContainers

          public long computeAvailableContainers(org.apache.hadoop.yarn.api.records.Resource available,
                                                 org.apache.hadoop.yarn.api.records.Resource required)

      Description copied from class: `ResourceCalculator`
      Compute the number of containers which can be allocated given `available` and `required` resources.
      Specified by:
        `computeAvailableContainers` in class `ResourceCalculator`
      Parameters:
        `available` - available resources
        `required` - required resources
      Returns:
        number of containers which can be allocated

    - #### divide

          public float divide(org.apache.hadoop.yarn.api.records.Resource unused,
                              org.apache.hadoop.yarn.api.records.Resource numerator,
                              org.apache.hadoop.yarn.api.records.Resource denominator)

      Description copied from class: `ResourceCalculator`
      Divide resource `numerator` by resource `denominator` using specified policy (domination, average, fairness etc.); hence overall `clusterResource` is provided for context.
      Specified by:
        `divide` in class `ResourceCalculator`
      Parameters:
        `unused` - cluster resources
        `numerator` - numerator
        `denominator` - denominator
      Returns:
        `numerator`/`denominator` using specific policy

    - #### isInvalidDivisor

          public boolean isInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)

      Description copied from class: `ResourceCalculator`
      Determine if a resource is not suitable for use as a divisor (will result in divide by 0, etc)
      Specified by:
        `isInvalidDivisor` in class `ResourceCalculator`
      Parameters:
        `r` - resource
      Returns:
        true if divisor is invalid (should not be used), false else

    - #### ratio

          public float ratio(org.apache.hadoop.yarn.api.records.Resource a,
                             org.apache.hadoop.yarn.api.records.Resource b)

      Description copied from class: `ResourceCalculator`
      Ratio of resource `a` to resource `b`.
      Specified by:
        `ratio` in class `ResourceCalculator`
      Parameters:
        `a` - resource
        `b` - resource
      Returns:
        ratio of resource `a` to resource `b`

    - #### divideAndCeil

          public org.apache.hadoop.yarn.api.records.Resource divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
                                                                           int denominator)

      Description copied from class: `ResourceCalculator`
      Divide-and-ceil `numerator` by `denominator`.
      Specified by:
        `divideAndCeil` in class `ResourceCalculator`
      Parameters:
        `numerator` - numerator resource
        `denominator` - denominator
      Returns:
        resultant resource

    - #### divideAndCeil

          public org.apache.hadoop.yarn.api.records.Resource divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
                                                                           float denominator)

      Description copied from class: `ResourceCalculator`
      Divide-and-ceil `numerator` by `denominator`.
      Specified by:
        `divideAndCeil` in class `ResourceCalculator`
      Parameters:
        `numerator` - numerator resource
        `denominator` - denominator
      Returns:
        resultant resource

    - #### normalize

          public org.apache.hadoop.yarn.api.records.Resource normalize(org.apache.hadoop.yarn.api.records.Resource r,
                                                                       org.apache.hadoop.yarn.api.records.Resource minimumResource,
                                                                       org.apache.hadoop.yarn.api.records.Resource maximumResource,
                                                                       org.apache.hadoop.yarn.api.records.Resource stepFactor)

      Description copied from class: `ResourceCalculator`
      Normalize resource `r` given the base `minimumResource` and verify against max allowed `maximumResource` using a step factor for the normalization.
      Specified by:
        `normalize` in class `ResourceCalculator`
      Parameters:
        `r` - resource
        `minimumResource` - minimum value
        `maximumResource` - the upper bound of the resource to be allocated
        `stepFactor` - the increment for resources to be allocated
      Returns:
        normalized resource

    - #### roundUp

          public org.apache.hadoop.yarn.api.records.Resource roundUp(org.apache.hadoop.yarn.api.records.Resource r,
                                                                     org.apache.hadoop.yarn.api.records.Resource stepFactor)

      Description copied from class: `ResourceCalculator`
      Round-up resource `r` given factor `stepFactor`.
      Specified by:
        `roundUp` in class `ResourceCalculator`
      Parameters:
        `r` - resource
        `stepFactor` - step-factor
      Returns:
        rounded resource

    - #### roundDown

          public org.apache.hadoop.yarn.api.records.Resource roundDown(org.apache.hadoop.yarn.api.records.Resource r,
                                                                       org.apache.hadoop.yarn.api.records.Resource stepFactor)

      Description copied from class: `ResourceCalculator`
      Round-down resource `r` given factor `stepFactor`.
      Specified by:
        `roundDown` in class `ResourceCalculator`
      Parameters:
        `r` - resource
        `stepFactor` - step-factor
      Returns:
        rounded resource

    - #### multiplyAndNormalizeUp

          public org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r,
                                                                                    double by,
                                                                                    org.apache.hadoop.yarn.api.records.Resource stepFactor)

      Description copied from class: `ResourceCalculator`
      Multiply resource `r` by factor `by` and normalize up using step-factor `stepFactor`.
      Specified by:
        `multiplyAndNormalizeUp` in class `ResourceCalculator`
      Parameters:
        `r` - resource to be multiplied
        `by` - multiplier
        `stepFactor` - factor by which to normalize up
      Returns:
        resulting normalized resource

    - #### multiplyAndNormalizeUp

          public org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r,
                                                                                    double[] by,
                                                                                    org.apache.hadoop.yarn.api.records.Resource stepFactor)

      Description copied from class: `ResourceCalculator`
      Multiply resource `r` by factor `by` and normalize up using step-factor `stepFactor`.
      Specified by:
        `multiplyAndNormalizeUp` in class `ResourceCalculator`
      Parameters:
        `r` - resource to be multiplied
        `by` - multiplier array for all resource types
        `stepFactor` - factor by which to normalize up
      Returns:
        resulting normalized resource

    - #### multiplyAndNormalizeDown

          public org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeDown(org.apache.hadoop.yarn.api.records.Resource r,
                                                                                      double by,
                                                                                      org.apache.hadoop.yarn.api.records.Resource stepFactor)

      Description copied from class: `ResourceCalculator`
      Multiply resource `r` by factor `by` and normalize down using step-factor `stepFactor`.
      Specified by:
        `multiplyAndNormalizeDown` in class `ResourceCalculator`
      Parameters:
        `r` - resource to be multiplied
        `by` - multiplier
        `stepFactor` - factor by which to normalize down
      Returns:
        resulting normalized resource

    - #### fitsIn

          public boolean fitsIn(org.apache.hadoop.yarn.api.records.Resource smaller,
                                org.apache.hadoop.yarn.api.records.Resource bigger)

      Description copied from class: `ResourceCalculator`
      Check if a smaller resource can be contained by bigger resource.
      Specified by:
        `fitsIn` in class `ResourceCalculator`

    - #### normalizeDown

          public org.apache.hadoop.yarn.api.records.Resource normalizeDown(org.apache.hadoop.yarn.api.records.Resource r,
                                                                           org.apache.hadoop.yarn.api.records.Resource stepFactor)

      Description copied from class: `ResourceCalculator`
      Get resource `r`and normalize down using step-factor `stepFactor`.
      Specified by:
        `normalizeDown` in class `ResourceCalculator`
      Parameters:
        `r` - resource to be multiplied
        `stepFactor` - factor by which to normalize down
      Returns:
        resulting normalized resource

    - #### isAnyMajorResourceZeroOrNegative

          public boolean isAnyMajorResourceZeroOrNegative(org.apache.hadoop.yarn.api.records.Resource resource)

      Description copied from class: `ResourceCalculator`
      Check if resource has any major resource types (which are all NodeManagers included) a zero value or negative value.
      Specified by:
        `isAnyMajorResourceZeroOrNegative` in class `ResourceCalculator`
      Parameters:
        `resource` - resource
      Returns:
        returns true if any resource is zero.

    - #### isAnyMajorResourceAboveZero

          public boolean isAnyMajorResourceAboveZero(org.apache.hadoop.yarn.api.records.Resource resource)

      Description copied from class: `ResourceCalculator`
      Check if resource has any major resource types (which are all NodeManagers included) has a > 0 value.
      Specified by:
        `isAnyMajorResourceAboveZero` in class `ResourceCalculator`
      Parameters:
        `resource` - resource
      Returns:
        returns true if any resource is > 0

    - #### isAllInvalidDivisor

          public boolean isAllInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)

      Description copied from class: `ResourceCalculator`
      Determine if all resources are zero.
      Specified by:
        `isAllInvalidDivisor` in class `ResourceCalculator`
      Parameters:
        `r` - resource
      Returns:
        true if all divisors are invalid (should not be used), false else

Copyright © 2008–2022 Apache Software Foundation. All rights reserved.