org.apache.hadoop.mapreduce.v2.util

## Class MRApps

- java.lang.Object
- - org.apache.hadoop.yarn.util.Apps
  - - org.apache.hadoop.mapreduce.v2.util.MRApps

-     @InterfaceAudience.Private
       @InterfaceStability.Unstable
      public class MRApps
      extends org.apache.hadoop.yarn.util.Apps

  Helper class for MR applications

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `MRApps.TaskAttemptStateUI`
    `static class` | `MRApps.TaskStateUI`

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static org.slf4j.Logger` | `LOG`

    - ### Fields inherited from class org.apache.hadoop.yarn.util.Apps

      `APP, ID`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `MRApps()`

  - ### Method Summary

    All Methods Static Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `static void` | `addLog4jSystemProperties(org.apache.hadoop.mapred.Task task, List<String> vargs, org.apache.hadoop.conf.Configuration conf)` Add the JVM system properties necessary to configure `ContainerLogAppender` or `ContainerRollingLogAppender`.
    `static void` | `addToEnvironment(Map<String,String> environment, String variable, String value, org.apache.hadoop.conf.Configuration conf)`
    `static ClassLoader` | `createJobClassLoader(org.apache.hadoop.conf.Configuration conf)` Creates a `ApplicationClassLoader` if `MRJobConfig.MAPREDUCE_JOB_CLASSLOADER` is set to true, and the APP_CLASSPATH environment variable is set.
    `static String` | `crossPlatformifyMREnv(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.ApplicationConstants.Environment env)`
    `static String` | `getChildLogLevel(org.apache.hadoop.conf.Configuration conf, boolean isMap)`
    `static org.apache.hadoop.fs.Path` | `getEndJobCommitFailureFile(org.apache.hadoop.conf.Configuration conf, String user, JobId jobId)`
    `static org.apache.hadoop.fs.Path` | `getEndJobCommitSuccessFile(org.apache.hadoop.conf.Configuration conf, String user, JobId jobId)`
    `static String` | `getJobFile(org.apache.hadoop.conf.Configuration conf, String user, org.apache.hadoop.mapreduce.JobID jobId)`
    `static org.apache.hadoop.fs.Path` | `getStagingAreaDir(org.apache.hadoop.conf.Configuration conf, String user)`
    `static org.apache.hadoop.fs.Path` | `getStartJobCommitFile(org.apache.hadoop.conf.Configuration conf, String user, JobId jobId)`
    `static String` | `getSystemPropertiesToLog(org.apache.hadoop.conf.Configuration conf)` Return lines for system property keys and values per configuration.
    `static void` | `setClassLoader(ClassLoader classLoader, org.apache.hadoop.conf.Configuration conf)` Sets the provided classloader on the given configuration and as the thread context classloader if the classloader is not null.
    `static void` | `setClasspath(Map<String,String> environment, org.apache.hadoop.conf.Configuration conf)`
    `static void` | `setEnvFromInputProperty(Map<String,String> env, String propName, String defaultPropValue, org.apache.hadoop.conf.Configuration conf)`
    `static void` | `setEnvFromInputString(Map<String,String> env, String envString, org.apache.hadoop.conf.Configuration conf)`
    `static void` | `setJobClassLoader(org.apache.hadoop.conf.Configuration conf)` Creates and sets a `ApplicationClassLoader` on the given configuration and as the thread context classloader, if `MRJobConfig.MAPREDUCE_JOB_CLASSLOADER` is set to true, and the APP_CLASSPATH environment variable is set.
    `static void` | `setupDistributedCache(org.apache.hadoop.conf.Configuration conf, Map<String,org.apache.hadoop.yarn.api.records.LocalResource> localResources)`
    `static void` | `setupDistributedCacheLocal(org.apache.hadoop.conf.Configuration conf)` Set up the DistributedCache related configs to make `DistributedCache.getLocalCacheFiles(Configuration)` and `DistributedCache.getLocalCacheArchives(Configuration)` working.
    `static MRApps.TaskAttemptStateUI` | `taskAttemptState(String attemptStateStr)`
    `static MRApps.TaskStateUI` | `taskState(String taskStateStr)`
    `static String` | `taskSymbol(TaskType type)`
    `static TaskType` | `taskType(String symbol)`
    `static JobId` | `toJobID(String jid)`
    `static String` | `toString(JobId jid)`
    `static String` | `toString(TaskAttemptId taid)`
    `static String` | `toString(TaskId tid)`
    `static TaskAttemptId` | `toTaskAttemptID(String taid)`
    `static TaskId` | `toTaskID(String tid)`

    - ### Methods inherited from class org.apache.hadoop.yarn.util.Apps

      `addToEnvironment, addToEnvironment, crossPlatformify, getEnvVarsFromInputProperty, isApplicationFinalState, setEnvFromInputProperty, setEnvFromInputString, setEnvFromInputString, shouldCountTowardsNodeBlacklisting, shouldHaveNext, throwParseException, toAppID, toAppID`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### LOG

          public static final org.slf4j.Logger LOG

  - ### Constructor Detail

    - #### MRApps

          public MRApps()

  - ### Method Detail

    - #### toString

          public static String toString(JobId jid)

    - #### toJobID

          public static JobId toJobID(String jid)

    - #### toString

          public static String toString(TaskId tid)

    - #### toTaskID

          public static TaskId toTaskID(String tid)

    - #### toString

          public static String toString(TaskAttemptId taid)

    - #### toTaskAttemptID

          public static TaskAttemptId toTaskAttemptID(String taid)

    - #### taskSymbol

          public static String taskSymbol(TaskType type)

    - #### taskType

          public static TaskType taskType(String symbol)

    - #### taskAttemptState

          public static MRApps.TaskAttemptStateUI taskAttemptState(String attemptStateStr)

    - #### taskState

          public static MRApps.TaskStateUI taskState(String taskStateStr)

    - #### setClasspath

          public static void setClasspath(Map<String,String> environment,
                                          org.apache.hadoop.conf.Configuration conf)
                                   throws IOException

      Throws:
        `IOException`

    - #### setJobClassLoader

          public static void setJobClassLoader(org.apache.hadoop.conf.Configuration conf)
                                        throws IOException

      Creates and sets a `ApplicationClassLoader` on the given configuration and as the thread context classloader, if `MRJobConfig.MAPREDUCE_JOB_CLASSLOADER` is set to true, and the APP_CLASSPATH environment variable is set.
      Parameters:
        `conf` -
      Throws:
        `IOException`

    - #### createJobClassLoader

          public static ClassLoader createJobClassLoader(org.apache.hadoop.conf.Configuration conf)
                                                  throws IOException

      Creates a `ApplicationClassLoader` if `MRJobConfig.MAPREDUCE_JOB_CLASSLOADER` is set to true, and the APP_CLASSPATH environment variable is set.
      Parameters:
        `conf` -
      Returns:
        the created job classloader, or null if the job classloader is not enabled or the APP_CLASSPATH environment variable is not set
      Throws:
        `IOException`

    - #### setClassLoader

          public static void setClassLoader(ClassLoader classLoader,
                                            org.apache.hadoop.conf.Configuration conf)

      Sets the provided classloader on the given configuration and as the thread context classloader if the classloader is not null.
      Parameters:
        `classLoader` -
        `conf` -

    - #### getStagingAreaDir

          public static org.apache.hadoop.fs.Path getStagingAreaDir(org.apache.hadoop.conf.Configuration conf,
                                                                    String user)

    - #### getJobFile

          public static String getJobFile(org.apache.hadoop.conf.Configuration conf,
                                          String user,
                                          org.apache.hadoop.mapreduce.JobID jobId)

    - #### getEndJobCommitSuccessFile

          public static org.apache.hadoop.fs.Path getEndJobCommitSuccessFile(org.apache.hadoop.conf.Configuration conf,
                                                                             String user,
                                                                             JobId jobId)

    - #### getEndJobCommitFailureFile

          public static org.apache.hadoop.fs.Path getEndJobCommitFailureFile(org.apache.hadoop.conf.Configuration conf,
                                                                             String user,
                                                                             JobId jobId)

    - #### getStartJobCommitFile

          public static org.apache.hadoop.fs.Path getStartJobCommitFile(org.apache.hadoop.conf.Configuration conf,
                                                                        String user,
                                                                        JobId jobId)

    - #### setupDistributedCache

          public static void setupDistributedCache(org.apache.hadoop.conf.Configuration conf,
                                                   Map<String,org.apache.hadoop.yarn.api.records.LocalResource> localResources)
                                            throws IOException

      Throws:
        `IOException`

    - #### setupDistributedCacheLocal

          public static void setupDistributedCacheLocal(org.apache.hadoop.conf.Configuration conf)
                                                 throws IOException

      Set up the DistributedCache related configs to make `DistributedCache.getLocalCacheFiles(Configuration)` and `DistributedCache.getLocalCacheArchives(Configuration)` working.
      Parameters:
        `conf` -
      Throws:
        `IOException`

    - #### getChildLogLevel

          public static String getChildLogLevel(org.apache.hadoop.conf.Configuration conf,
                                                boolean isMap)

    - #### addLog4jSystemProperties

          public static void addLog4jSystemProperties(org.apache.hadoop.mapred.Task task,
                                                      List<String> vargs,
                                                      org.apache.hadoop.conf.Configuration conf)

      Add the JVM system properties necessary to configure `ContainerLogAppender` or `ContainerRollingLogAppender`.
      Parameters:
        `task` - for map/reduce, or null for app master
        `vargs` - the argument list to append to
        `conf` - configuration of MR job

    - #### getSystemPropertiesToLog

          public static String getSystemPropertiesToLog(org.apache.hadoop.conf.Configuration conf)

      Return lines for system property keys and values per configuration.
      Returns:
        the formatted string for the system property lines or null if no properties are specified.

    - #### setEnvFromInputString

          public static void setEnvFromInputString(Map<String,String> env,
                                                   String envString,
                                                   org.apache.hadoop.conf.Configuration conf)

    - #### setEnvFromInputProperty

          public static void setEnvFromInputProperty(Map<String,String> env,
                                                     String propName,
                                                     String defaultPropValue,
                                                     org.apache.hadoop.conf.Configuration conf)

    - #### addToEnvironment

          @InterfaceAudience.Public
           @InterfaceStability.Unstable
          public static void addToEnvironment(Map<String,String> environment,
                                                                                                      String variable,
                                                                                                      String value,
                                                                                                      org.apache.hadoop.conf.Configuration conf)

    - #### crossPlatformifyMREnv

          public static String crossPlatformifyMREnv(org.apache.hadoop.conf.Configuration conf,
                                                     org.apache.hadoop.yarn.api.ApplicationConstants.Environment env)

Copyright © 2008–2022 Apache Software Foundation. All rights reserved.