org.apache.hadoop.mapred

## Class YARNRunner

- java.lang.Object
- - org.apache.hadoop.mapred.YARNRunner

- All Implemented Interfaces:
    org.apache.hadoop.ipc.VersionedProtocol, org.apache.hadoop.mapreduce.protocol.ClientProtocol

      public class YARNRunner
      extends Object
      implements org.apache.hadoop.mapreduce.protocol.ClientProtocol

  This class enables the current JobClient (0.22 hadoop) to run on YARN.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static org.apache.hadoop.yarn.api.records.Priority` | `AM_CONTAINER_PRIORITY`

    - ### Fields inherited from interface org.apache.hadoop.mapreduce.protocol.ClientProtocol

      `versionID`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `YARNRunner(org.apache.hadoop.conf.Configuration conf)` Yarn runner incapsulates the client interface of yarn
    `YARNRunner(org.apache.hadoop.conf.Configuration conf, ResourceMgrDelegate resMgrDelegate)` Similar to `YARNRunner(Configuration)` but allowing injecting `ResourceMgrDelegate`.
    `YARNRunner(org.apache.hadoop.conf.Configuration conf, ResourceMgrDelegate resMgrDelegate, ClientCache clientCache)` Similar to `YARNRunner(Configuration, ResourceMgrDelegate)` but allowing injecting `ClientCache`.

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `cancelDelegationToken(org.apache.hadoop.security.token.Token<org.apache.hadoop.mapreduce.security.token.delegation.DelegationTokenIdentifier> arg0)`
    `void` | `close()`
    `org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext` | `createApplicationSubmissionContext(org.apache.hadoop.conf.Configuration jobConf, String jobSubmitDir, org.apache.hadoop.security.Credentials ts)` Constructs all the necessary information to start the MR AM.
    `org.apache.hadoop.mapreduce.TaskTrackerInfo[]` | `getActiveTrackers()`
    `org.apache.hadoop.mapreduce.JobStatus[]` | `getAllJobs()`
    `org.apache.hadoop.mapreduce.TaskTrackerInfo[]` | `getBlacklistedTrackers()`
    `org.apache.hadoop.mapreduce.QueueInfo[]` | `getChildQueues(String parent)`
    `org.apache.hadoop.mapreduce.ClusterMetrics` | `getClusterMetrics()`
    `org.apache.hadoop.security.token.Token<org.apache.hadoop.mapreduce.security.token.delegation.DelegationTokenIdentifier>` | `getDelegationToken(org.apache.hadoop.io.Text renewer)`
    `String` | `getFilesystemName()`
    `org.apache.hadoop.mapreduce.Counters` | `getJobCounters(org.apache.hadoop.mapreduce.JobID arg0)`
    `String` | `getJobHistoryDir()`
    `org.apache.hadoop.mapreduce.JobStatus` | `getJobStatus(org.apache.hadoop.mapreduce.JobID jobID)`
    `org.apache.hadoop.mapreduce.Cluster.JobTrackerStatus` | `getJobTrackerStatus()`
    `org.apache.hadoop.mapreduce.v2.LogParams` | `getLogFileParams(org.apache.hadoop.mapreduce.JobID jobID, org.apache.hadoop.mapreduce.TaskAttemptID taskAttemptID)`
    `org.apache.hadoop.mapreduce.JobID` | `getNewJobID()`
    `org.apache.hadoop.ipc.ProtocolSignature` | `getProtocolSignature(String protocol, long clientVersion, int clientMethodsHash)`
    `long` | `getProtocolVersion(String arg0, long arg1)`
    `org.apache.hadoop.mapreduce.QueueInfo` | `getQueue(String queueName)`
    `org.apache.hadoop.mapreduce.QueueAclsInfo[]` | `getQueueAclsForCurrentUser()`
    `org.apache.hadoop.security.authorize.AccessControlList` | `getQueueAdmins(String arg0)`
    `org.apache.hadoop.mapreduce.QueueInfo[]` | `getQueues()`
    `org.apache.hadoop.mapreduce.QueueInfo[]` | `getRootQueues()`
    `String` | `getStagingAreaDir()`
    `String` | `getSystemDir()`
    `org.apache.hadoop.mapreduce.TaskCompletionEvent[]` | `getTaskCompletionEvents(org.apache.hadoop.mapreduce.JobID arg0, int arg1, int arg2)`
    `String[]` | `getTaskDiagnostics(org.apache.hadoop.mapreduce.TaskAttemptID arg0)`
    `org.apache.hadoop.mapreduce.TaskReport[]` | `getTaskReports(org.apache.hadoop.mapreduce.JobID jobID, org.apache.hadoop.mapreduce.TaskType taskType)`
    `long` | `getTaskTrackerExpiryInterval()`
    `void` | `killJob(org.apache.hadoop.mapreduce.JobID arg0)`
    `boolean` | `killTask(org.apache.hadoop.mapreduce.TaskAttemptID arg0, boolean arg1)`
    `long` | `renewDelegationToken(org.apache.hadoop.security.token.Token<org.apache.hadoop.mapreduce.security.token.delegation.DelegationTokenIdentifier> arg0)`
    `void` | `setJobPriority(org.apache.hadoop.mapreduce.JobID arg0, String arg1)`
    `void` | `setResourceMgrDelegate(ResourceMgrDelegate resMgrDelegate)`
    `org.apache.hadoop.mapreduce.JobStatus` | `submitJob(org.apache.hadoop.mapreduce.JobID jobId, String jobSubmitDir, org.apache.hadoop.security.Credentials ts)`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### AM_CONTAINER_PRIORITY

          public static final org.apache.hadoop.yarn.api.records.Priority AM_CONTAINER_PRIORITY

  - ### Constructor Detail

    - #### YARNRunner

          public YARNRunner(org.apache.hadoop.conf.Configuration conf)

      Yarn runner incapsulates the client interface of yarn
      Parameters:
        `conf` - the configuration object for the client

    - #### YARNRunner

          public YARNRunner(org.apache.hadoop.conf.Configuration conf,
                            ResourceMgrDelegate resMgrDelegate)

      Similar to `YARNRunner(Configuration)` but allowing injecting `ResourceMgrDelegate`. Enables mocking and testing.
      Parameters:
        `conf` - the configuration object for the client
        `resMgrDelegate` - the resourcemanager client handle.

    - #### YARNRunner

          public YARNRunner(org.apache.hadoop.conf.Configuration conf,
                            ResourceMgrDelegate resMgrDelegate,
                            ClientCache clientCache)

      Similar to `YARNRunner(Configuration, ResourceMgrDelegate)` but allowing injecting `ClientCache`. Enable mocking and testing.
      Parameters:
        `conf` - the configuration object
        `resMgrDelegate` - the resource manager delegate
        `clientCache` - the client cache object.

  - ### Method Detail

    - #### setResourceMgrDelegate

          @InterfaceAudience.Private
          public void setResourceMgrDelegate(ResourceMgrDelegate resMgrDelegate)

    - #### cancelDelegationToken

          public void cancelDelegationToken(org.apache.hadoop.security.token.Token<org.apache.hadoop.mapreduce.security.token.delegation.DelegationTokenIdentifier> arg0)
                                     throws IOException,
                                            InterruptedException

      Specified by:
        `cancelDelegationToken` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getActiveTrackers

          public org.apache.hadoop.mapreduce.TaskTrackerInfo[] getActiveTrackers()
                                                                          throws IOException,
                                                                                 InterruptedException

      Specified by:
        `getActiveTrackers` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getAllJobs

          public org.apache.hadoop.mapreduce.JobStatus[] getAllJobs()
                                                             throws IOException,
                                                                    InterruptedException

      Specified by:
        `getAllJobs` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getBlacklistedTrackers

          public org.apache.hadoop.mapreduce.TaskTrackerInfo[] getBlacklistedTrackers()
                                                                               throws IOException,
                                                                                      InterruptedException

      Specified by:
        `getBlacklistedTrackers` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getClusterMetrics

          public org.apache.hadoop.mapreduce.ClusterMetrics getClusterMetrics()
                                                                       throws IOException,
                                                                              InterruptedException

      Specified by:
        `getClusterMetrics` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getDelegationToken

          public org.apache.hadoop.security.token.Token<org.apache.hadoop.mapreduce.security.token.delegation.DelegationTokenIdentifier> getDelegationToken(org.apache.hadoop.io.Text renewer)
                                                                                                                                                     throws IOException,
                                                                                                                                                            InterruptedException

      Specified by:
        `getDelegationToken` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getFilesystemName

          public String getFilesystemName()
                                   throws IOException,
                                          InterruptedException

      Specified by:
        `getFilesystemName` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getNewJobID

          public org.apache.hadoop.mapreduce.JobID getNewJobID()
                                                        throws IOException,
                                                               InterruptedException

      Specified by:
        `getNewJobID` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getQueue

          public org.apache.hadoop.mapreduce.QueueInfo getQueue(String queueName)
                                                         throws IOException,
                                                                InterruptedException

      Specified by:
        `getQueue` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getQueueAclsForCurrentUser

          public org.apache.hadoop.mapreduce.QueueAclsInfo[] getQueueAclsForCurrentUser()
                                                                                 throws IOException,
                                                                                        InterruptedException

      Specified by:
        `getQueueAclsForCurrentUser` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getQueues

          public org.apache.hadoop.mapreduce.QueueInfo[] getQueues()
                                                            throws IOException,
                                                                   InterruptedException

      Specified by:
        `getQueues` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getRootQueues

          public org.apache.hadoop.mapreduce.QueueInfo[] getRootQueues()
                                                                throws IOException,
                                                                       InterruptedException

      Specified by:
        `getRootQueues` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getChildQueues

          public org.apache.hadoop.mapreduce.QueueInfo[] getChildQueues(String parent)
                                                                 throws IOException,
                                                                        InterruptedException

      Specified by:
        `getChildQueues` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getStagingAreaDir

          public String getStagingAreaDir()
                                   throws IOException,
                                          InterruptedException

      Specified by:
        `getStagingAreaDir` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getSystemDir

          public String getSystemDir()
                              throws IOException,
                                     InterruptedException

      Specified by:
        `getSystemDir` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getTaskTrackerExpiryInterval

          public long getTaskTrackerExpiryInterval()
                                            throws IOException,
                                                   InterruptedException

      Specified by:
        `getTaskTrackerExpiryInterval` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### submitJob

          public org.apache.hadoop.mapreduce.JobStatus submitJob(org.apache.hadoop.mapreduce.JobID jobId,
                                                                 String jobSubmitDir,
                                                                 org.apache.hadoop.security.Credentials ts)
                                                          throws IOException,
                                                                 InterruptedException

      Specified by:
        `submitJob` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### createApplicationSubmissionContext

          public org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext createApplicationSubmissionContext(org.apache.hadoop.conf.Configuration jobConf,
                                                                                                                    String jobSubmitDir,
                                                                                                                    org.apache.hadoop.security.Credentials ts)
                                                                                                             throws IOException

      Constructs all the necessary information to start the MR AM.
      Parameters:
        `jobConf` - the configuration for the MR job
        `jobSubmitDir` - the directory path for the job
        `ts` - the security credentials for the job
      Returns:
        ApplicationSubmissionContext
      Throws:
        `IOException` - on IO error (e.g. path resolution)

    - #### setJobPriority

          public void setJobPriority(org.apache.hadoop.mapreduce.JobID arg0,
                                     String arg1)
                              throws IOException,
                                     InterruptedException

      Specified by:
        `setJobPriority` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getProtocolVersion

          public long getProtocolVersion(String arg0,
                                         long arg1)
                                  throws IOException

      Specified by:
        `getProtocolVersion` in interface `org.apache.hadoop.ipc.VersionedProtocol`
      Throws:
        `IOException`

    - #### renewDelegationToken

          public long renewDelegationToken(org.apache.hadoop.security.token.Token<org.apache.hadoop.mapreduce.security.token.delegation.DelegationTokenIdentifier> arg0)
                                    throws IOException,
                                           InterruptedException

      Specified by:
        `renewDelegationToken` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getJobCounters

          public org.apache.hadoop.mapreduce.Counters getJobCounters(org.apache.hadoop.mapreduce.JobID arg0)
                                                              throws IOException,
                                                                     InterruptedException

      Specified by:
        `getJobCounters` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getJobHistoryDir

          public String getJobHistoryDir()
                                  throws IOException,
                                         InterruptedException

      Specified by:
        `getJobHistoryDir` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getJobStatus

          public org.apache.hadoop.mapreduce.JobStatus getJobStatus(org.apache.hadoop.mapreduce.JobID jobID)
                                                             throws IOException,
                                                                    InterruptedException

      Specified by:
        `getJobStatus` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getTaskCompletionEvents

          public org.apache.hadoop.mapreduce.TaskCompletionEvent[] getTaskCompletionEvents(org.apache.hadoop.mapreduce.JobID arg0,
                                                                                           int arg1,
                                                                                           int arg2)
                                                                                    throws IOException,
                                                                                           InterruptedException

      Specified by:
        `getTaskCompletionEvents` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getTaskDiagnostics

          public String[] getTaskDiagnostics(org.apache.hadoop.mapreduce.TaskAttemptID arg0)
                                      throws IOException,
                                             InterruptedException

      Specified by:
        `getTaskDiagnostics` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getTaskReports

          public org.apache.hadoop.mapreduce.TaskReport[] getTaskReports(org.apache.hadoop.mapreduce.JobID jobID,
                                                                         org.apache.hadoop.mapreduce.TaskType taskType)
                                                                  throws IOException,
                                                                         InterruptedException

      Specified by:
        `getTaskReports` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### killJob

          public void killJob(org.apache.hadoop.mapreduce.JobID arg0)
                       throws IOException,
                              InterruptedException

      Specified by:
        `killJob` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### killTask

          public boolean killTask(org.apache.hadoop.mapreduce.TaskAttemptID arg0,
                                  boolean arg1)
                           throws IOException,
                                  InterruptedException

      Specified by:
        `killTask` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getQueueAdmins

          public org.apache.hadoop.security.authorize.AccessControlList getQueueAdmins(String arg0)
                                                                                throws IOException

      Specified by:
        `getQueueAdmins` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`

    - #### getJobTrackerStatus

          public org.apache.hadoop.mapreduce.Cluster.JobTrackerStatus getJobTrackerStatus()
                                                                                   throws IOException,
                                                                                          InterruptedException

      Specified by:
        `getJobTrackerStatus` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`
        `InterruptedException`

    - #### getProtocolSignature

          public org.apache.hadoop.ipc.ProtocolSignature getProtocolSignature(String protocol,
                                                                              long clientVersion,
                                                                              int clientMethodsHash)
                                                                       throws IOException

      Specified by:
        `getProtocolSignature` in interface `org.apache.hadoop.ipc.VersionedProtocol`
      Throws:
        `IOException`

    - #### getLogFileParams

          public org.apache.hadoop.mapreduce.v2.LogParams getLogFileParams(org.apache.hadoop.mapreduce.JobID jobID,
                                                                           org.apache.hadoop.mapreduce.TaskAttemptID taskAttemptID)
                                                                    throws IOException

      Specified by:
        `getLogFileParams` in interface `org.apache.hadoop.mapreduce.protocol.ClientProtocol`
      Throws:
        `IOException`

    - #### close

          public void close()
                     throws IOException

      Throws:
        `IOException`

Copyright © 2008–2022 Apache Software Foundation. All rights reserved.