org.apache.hadoop.yarn.client.api

## Class AMRMClient<T extends AMRMClient.ContainerRequest>

- java.lang.Object
- - org.apache.hadoop.service.AbstractService
  - - org.apache.hadoop.yarn.client.api.AMRMClient<T>

- All Implemented Interfaces:
    Closeable, AutoCloseable, org.apache.hadoop.service.Service
  Direct Known Subclasses:
    AMRMClientImpl

      @InterfaceAudience.Public
       @InterfaceStability.Stable
      public abstract class AMRMClient<T extends AMRMClient.ContainerRequest>
      extends org.apache.hadoop.service.AbstractService

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `AMRMClient.ContainerRequest` Object to represent a single container request for resources.

    - ### Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

      `org.apache.hadoop.service.Service.STATE`

  - ### Constructor Summary

    Constructors
    Modifier | Constructor and Description
    --- | ---
    `protected` | `AMRMClient(String name)`

  - ### Method Summary

    All Methods Static Methods Instance Methods Abstract Methods Concrete Methods Deprecated Methods
    Modifier and Type | Method and Description
    --- | ---
    `abstract void` | `addContainerRequest(T req)` Request containers for resources before calling `allocate`
    `void` | `addSchedulingRequests(Collection<org.apache.hadoop.yarn.api.records.SchedulingRequest> schedulingRequests)` Add a Collection of SchedulingRequests.
    `abstract org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse` | `allocate(float progressIndicator)` Request additional containers and receive new container allocations.
    `static <T extends AMRMClient.ContainerRequest> AMRMClient<T>` | `createAMRMClient()` Create a new instance of AMRMClient.
    `abstract org.apache.hadoop.yarn.api.records.Resource` | `getAvailableResources()` Get the currently available resources in the cluster.
    `abstract int` | `getClusterNodeCount()` Get the current number of nodes in the cluster.
    `abstract Collection<T>` | `getMatchingRequests(long allocationRequestId)` Get outstanding `ContainerRequest`s matching the given allocationRequestId.
    `List<? extends Collection<T>>` | `getMatchingRequests(org.apache.hadoop.yarn.api.records.Priority priority, String resourceName, org.apache.hadoop.yarn.api.records.ExecutionType executionType, org.apache.hadoop.yarn.api.records.Resource capability)` Get outstanding `ContainerRequest`s matching the given parameters.
    `List<? extends Collection<T>>` | `getMatchingRequests(org.apache.hadoop.yarn.api.records.Priority priority, String resourceName, org.apache.hadoop.yarn.api.records.ExecutionType executionType, org.apache.hadoop.yarn.api.records.Resource capability, String profile)`
    `abstract List<? extends Collection<T>>` | `getMatchingRequests(org.apache.hadoop.yarn.api.records.Priority priority, String resourceName, org.apache.hadoop.yarn.api.records.Resource capability)` Get outstanding `ContainerRequest`s matching the given parameters.
    `NMTokenCache` | `getNMTokenCache()` Get the NM token cache of the `AMRMClient`.
    `org.apache.hadoop.yarn.client.api.TimelineV2Client` | `getRegisteredTimelineV2Client()` Get registered timeline v2 client.
    `abstract org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse` | `registerApplicationMaster(String appHostName, int appHostPort, String appTrackingUrl)` Register the application master.
    `org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse` | `registerApplicationMaster(String appHostName, int appHostPort, String appTrackingUrl, Map<Set<String>,org.apache.hadoop.yarn.api.resource.PlacementConstraint> placementConstraints)` Register the application master.
    `void` | `registerTimelineV2Client(org.apache.hadoop.yarn.client.api.TimelineV2Client client)` Register TimelineV2Client to AMRMClient.
    `abstract void` | `releaseAssignedContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)` Release containers assigned by the Resource Manager.
    `abstract void` | `removeContainerRequest(T req)` Remove previous container request.
    `void` | `requestContainerResourceChange(org.apache.hadoop.yarn.api.records.Container container, org.apache.hadoop.yarn.api.records.Resource capability)` Deprecated. use `requestContainerUpdate(Container, UpdateContainerRequest)`
    `abstract void` | `requestContainerUpdate(org.apache.hadoop.yarn.api.records.Container container, org.apache.hadoop.yarn.api.records.UpdateContainerRequest updateContainerRequest)` Request a container update before calling `allocate`.
    `void` | `setNMTokenCache(NMTokenCache nmTokenCache)` Set the NM token cache for the `AMRMClient`.
    `abstract void` | `unregisterApplicationMaster(org.apache.hadoop.yarn.api.records.FinalApplicationStatus appStatus, String appMessage, String appTrackingUrl)` Unregister the application master.
    `abstract void` | `updateBlacklist(List<String> blacklistAdditions, List<String> blacklistRemovals)` Update application's blacklist with addition or removal resources.
    `void` | `updateTrackingUrl(String trackingUrl)` Update application's tracking url on next heartbeat.
    `void` | `waitFor(Supplier<Boolean> check)` Wait for `check` to return true for each 1000 ms.
    `void` | `waitFor(Supplier<Boolean> check, int checkEveryMillis)` Wait for `check` to return true for each `checkEveryMillis` ms.
    `void` | `waitFor(Supplier<Boolean> check, int checkEveryMillis, int logInterval)` Wait for `check` to return true for each `checkEveryMillis` ms.

    - ### Methods inherited from class org.apache.hadoop.service.AbstractService

      `close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceInit, serviceStart, serviceStop, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Constructor Detail

    - #### AMRMClient

          @InterfaceAudience.Private
          protected AMRMClient(String name)

  - ### Method Detail

    - #### createAMRMClient

          @InterfaceAudience.Public
          public static <T extends AMRMClient.ContainerRequest> AMRMClient<T> createAMRMClient()

      Create a new instance of AMRMClient. For usage:

           AMRMClient.<T>createAMRMClientContainerRequest()

      Returns:
        the newly create AMRMClient instance.

    - #### addSchedulingRequests

          @InterfaceAudience.Public
           @InterfaceStability.Unstable
          public void addSchedulingRequests(Collection<org.apache.hadoop.yarn.api.records.SchedulingRequest> schedulingRequests)

      Add a Collection of SchedulingRequests. The AMRMClient will ensure that all requests in the same batch are sent in the same allocate call.
      Parameters:
        `schedulingRequests` - Collection of Scheduling Requests.

    - #### registerApplicationMaster

          public abstract org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse registerApplicationMaster(String appHostName,
                                                                                                                                 int appHostPort,
                                                                                                                                 String appTrackingUrl)
                                                                                                                          throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                                                                 IOException

      Register the application master. This must be called before any other interaction
      Parameters:
        `appHostName` - Name of the host on which master is running
        `appHostPort` - Port master is listening on
        `appTrackingUrl` - URL at which the master info can be seen
      Returns:
        `RegisterApplicationMasterResponse`
      Throws:
        `org.apache.hadoop.yarn.exceptions.YarnException`
        `IOException`

    - #### registerApplicationMaster

          @InterfaceAudience.Public
           @InterfaceStability.Unstable
          public org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse registerApplicationMaster(String appHostName,
                                                                                                                                                                                int appHostPort,
                                                                                                                                                                                String appTrackingUrl,
                                                                                                                                                                                Map<Set<String>,org.apache.hadoop.yarn.api.resource.PlacementConstraint> placementConstraints)
                                                                                                                                                                         throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                                                                                                                IOException

      Register the application master. This must be called before any other interaction
      Parameters:
        `appHostName` - Name of the host on which master is running
        `appHostPort` - Port master is listening on
        `appTrackingUrl` - URL at which the master info can be seen
        `placementConstraints` - Placement Constraints mappings.
      Returns:
        `RegisterApplicationMasterResponse`
      Throws:
        `org.apache.hadoop.yarn.exceptions.YarnException`
        `IOException`

    - #### allocate

          public abstract org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse allocate(float progressIndicator)
                                                                                        throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                               IOException

      Request additional containers and receive new container allocations. Requests made via `addContainerRequest` are sent to the `ResourceManager`. New containers assigned to the master are retrieved. Status of completed containers and node health updates are also retrieved. This also doubles up as a heartbeat to the ResourceManager and must be made periodically. The call may not always return any new allocations of containers. App should not make concurrent allocate requests. May cause request loss.

      Note : If the user has not removed container requests that have already been satisfied, then the re-register may end up sending the entire container requests to the RM (including matched requests). Which would mean the RM could end up giving it a lot of new allocated containers.

      Parameters:
        `progressIndicator` - Indicates progress made by the master
      Returns:
        the response of the allocate request
      Throws:
        `org.apache.hadoop.yarn.exceptions.YarnException`
        `IOException`

    - #### unregisterApplicationMaster

          public abstract void unregisterApplicationMaster(org.apache.hadoop.yarn.api.records.FinalApplicationStatus appStatus,
                                                           String appMessage,
                                                           String appTrackingUrl)
                                                    throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                           IOException

      Unregister the application master. This must be called in the end.
      Parameters:
        `appStatus` - Success/Failure status of the master
        `appMessage` - Diagnostics message on failure
        `appTrackingUrl` - New URL to get master info
      Throws:
        `org.apache.hadoop.yarn.exceptions.YarnException`
        `IOException`

    - #### addContainerRequest

          public abstract void addContainerRequest(T req)

      Request containers for resources before calling `allocate`
      Parameters:
        `req` - Resource request

    - #### removeContainerRequest

          public abstract void removeContainerRequest(T req)

      Remove previous container request. The previous container request may have already been sent to the ResourceManager. So even after the remove request the app must be prepared to receive an allocation for the previous request even after the remove request
      Parameters:
        `req` - Resource request

    - #### requestContainerResourceChange

          @Deprecated
          public void requestContainerResourceChange(org.apache.hadoop.yarn.api.records.Container container,
                                                                 org.apache.hadoop.yarn.api.records.Resource capability)

      Deprecated. use `requestContainerUpdate(Container, UpdateContainerRequest)`
      Request container resource change before calling `allocate`. Any previous pending resource change request of the same container will be removed. Application that calls this method is expected to maintain the `Container`s that are returned from previous successful allocations or resource changes. By passing in the existing container and a target resource capability to this method, the application requests the ResourceManager to change the existing resource allocation to the target resource allocation.
      Parameters:
        `container` - The container returned from the last successful resource allocation or resource change
        `capability` - The target resource capability of the container

    - #### requestContainerUpdate

          public abstract void requestContainerUpdate(org.apache.hadoop.yarn.api.records.Container container,
                                                      org.apache.hadoop.yarn.api.records.UpdateContainerRequest updateContainerRequest)

      Request a container update before calling `allocate`. Any previous pending update request of the same container will be removed.
      Parameters:
        `container` - The container returned from the last successful resource allocation or update
        `updateContainerRequest` - The `UpdateContainerRequest`.

    - #### releaseAssignedContainer

          public abstract void releaseAssignedContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)

      Release containers assigned by the Resource Manager. If the app cannot use the container or wants to give up the container then it can release them. The app needs to make new requests for the released resource capability if it still needs it. eg. it released non-local resources
      Parameters:
        `containerId` -

    - #### getAvailableResources

          public abstract org.apache.hadoop.yarn.api.records.Resource getAvailableResources()

      Get the currently available resources in the cluster. A valid value is available after a call to allocate has been made
      Returns:
        Currently available resources

    - #### getClusterNodeCount

          public abstract int getClusterNodeCount()

      Get the current number of nodes in the cluster. A valid values is available after a call to allocate has been made
      Returns:
        Current number of nodes in the cluster

    - #### getMatchingRequests

          @InterfaceStability.Evolving
          public abstract List<? extends Collection<T>> getMatchingRequests(org.apache.hadoop.yarn.api.records.Priority priority,
                                                                                                         String resourceName,
                                                                                                         org.apache.hadoop.yarn.api.records.Resource capability)

      Get outstanding `ContainerRequest`s matching the given parameters. These ContainerRequests should have been added via `addContainerRequest` earlier in the lifecycle. For performance, the AMRMClient may return its internal collection directly without creating a copy. Users should not perform mutable operations on the return value. Each collection in the list contains requests with identical `Resource` size that fit in the given capability. In a collection, requests will be returned in the same order as they were added. NOTE: This API only matches Container requests that were created by the client WITHOUT the allocationRequestId being set.
      Returns:
        Collection of request matching the parameters

    - #### getMatchingRequests

          @InterfaceStability.Evolving
          public List<? extends Collection<T>> getMatchingRequests(org.apache.hadoop.yarn.api.records.Priority priority,
                                                                                                String resourceName,
                                                                                                org.apache.hadoop.yarn.api.records.ExecutionType executionType,
                                                                                                org.apache.hadoop.yarn.api.records.Resource capability)

      Get outstanding `ContainerRequest`s matching the given parameters. These ContainerRequests should have been added via `addContainerRequest` earlier in the lifecycle. For performance, the AMRMClient may return its internal collection directly without creating a copy. Users should not perform mutable operations on the return value. Each collection in the list contains requests with identical `Resource` size that fit in the given capability. In a collection, requests will be returned in the same order as they were added. specify an `ExecutionType`. NOTE: This API only matches Container requests that were created by the client WITHOUT the allocationRequestId being set.
      Parameters:
        `priority` - Priority
        `resourceName` - Location
        `executionType` - ExecutionType
        `capability` - Capability
      Returns:
        Collection of request matching the parameters

    - #### getMatchingRequests

          @InterfaceStability.Evolving
          public List<? extends Collection<T>> getMatchingRequests(org.apache.hadoop.yarn.api.records.Priority priority,
                                                                                                String resourceName,
                                                                                                org.apache.hadoop.yarn.api.records.ExecutionType executionType,
                                                                                                org.apache.hadoop.yarn.api.records.Resource capability,
                                                                                                String profile)

    - #### getMatchingRequests

          @InterfaceStability.Evolving
          public abstract Collection<T> getMatchingRequests(long allocationRequestId)

      Get outstanding `ContainerRequest`s matching the given allocationRequestId. These ContainerRequests should have been added via `addContainerRequest` earlier in the lifecycle. For performance, the AMRMClient may return its internal collection directly without creating a copy. Users should not perform mutable operations on the return value. NOTE: This API only matches Container requests that were created by the client WITH the allocationRequestId being set to a non-default value.
      Parameters:
        `allocationRequestId` - Allocation Request Id
      Returns:
        Collection of request matching the parameters

    - #### updateBlacklist

          public abstract void updateBlacklist(List<String> blacklistAdditions,
                                               List<String> blacklistRemovals)

      Update application's blacklist with addition or removal resources.
      Parameters:
        `blacklistAdditions` - list of resources which should be added to the application blacklist
        `blacklistRemovals` - list of resources which should be removed from the application blacklist

    - #### setNMTokenCache

          public void setNMTokenCache(NMTokenCache nmTokenCache)

      Set the NM token cache for the `AMRMClient`. This cache must be shared with the `NMClient` used to manage containers for the `AMRMClient`

      If a NM token cache is not set, the `NMTokenCache.getSingleton()` singleton instance will be used.

      Parameters:
        `nmTokenCache` - the NM token cache to use.

    - #### getNMTokenCache

          public NMTokenCache getNMTokenCache()

      Get the NM token cache of the `AMRMClient`. This cache must be shared with the `NMClient` used to manage containers for the `AMRMClient`.

      If a NM token cache is not set, the `NMTokenCache.getSingleton()` singleton instance will be used.

      Returns:
        the NM token cache.

    - #### registerTimelineV2Client

          public void registerTimelineV2Client(org.apache.hadoop.yarn.client.api.TimelineV2Client client)

      Register TimelineV2Client to AMRMClient. Writer's address for the timeline V2 client will be updated dynamically if registered.
      Parameters:
        `client` - the timeline v2 client to register

    - #### getRegisteredTimelineV2Client

          public org.apache.hadoop.yarn.client.api.TimelineV2Client getRegisteredTimelineV2Client()

      Get registered timeline v2 client.
      Returns:
        the registered timeline v2 client

    - #### updateTrackingUrl

          @InterfaceAudience.Public
           @InterfaceStability.Unstable
          public void updateTrackingUrl(String trackingUrl)

      Update application's tracking url on next heartbeat.
      Parameters:
        `trackingUrl` - new tracking url for this application

    - #### waitFor

          public void waitFor(Supplier<Boolean> check)
                       throws InterruptedException

      Wait for `check` to return true for each 1000 ms. See also `waitFor(java.util.function.Supplier, int)` and `waitFor(java.util.function.Supplier, int, int)`
      Parameters:
        `check` - the condition for which it should wait
      Throws:
        `InterruptedException`

    - #### waitFor

          public void waitFor(Supplier<Boolean> check,
                              int checkEveryMillis)
                       throws InterruptedException

      Wait for `check` to return true for each `checkEveryMillis` ms. See also `waitFor(java.util.function.Supplier, int, int)`
      Parameters:
        `check` - user defined checker
        `checkEveryMillis` - interval to call `check`
      Throws:
        `InterruptedException`

    - #### waitFor

          public void waitFor(Supplier<Boolean> check,
                              int checkEveryMillis,
                              int logInterval)
                       throws InterruptedException

      Wait for `check` to return true for each `checkEveryMillis` ms. In the main loop, this method will log the message "waiting in main loop" for each `logInterval` times iteration to confirm the thread is alive.
      Parameters:
        `check` - user defined checker
        `checkEveryMillis` - interval to call `check`
        `logInterval` - interval to log for each
      Throws:
        `InterruptedException`

Copyright © 2008–2022 Apache Software Foundation. All rights reserved.