org.apache.hadoop.mapred.pipes

## Class Submitter

- java.lang.Object
- - org.apache.hadoop.conf.Configured
  - - org.apache.hadoop.mapred.pipes.Submitter

- All Implemented Interfaces:
    org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

      @InterfaceAudience.Public
       @InterfaceStability.Stable
      public class Submitter
      extends org.apache.hadoop.conf.Configured
      implements org.apache.hadoop.util.Tool

  The main entry point and job submitter. It may either be used as a command line-based or API-based method to launch Pipes jobs.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static String` | `EXECUTABLE`
    `static String` | `INPUT_FORMAT`
    `static String` | `INTERPRETOR`
    `static String` | `IS_JAVA_MAP`
    `static String` | `IS_JAVA_REDUCE`
    `static String` | `IS_JAVA_RR`
    `static String` | `IS_JAVA_RW`
    `protected static org.slf4j.Logger` | `LOG`
    `static String` | `PARTITIONER`
    `static String` | `PORT`
    `static String` | `PRESERVE_COMMANDFILE`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `Submitter()`
    `Submitter(org.apache.hadoop.conf.Configuration conf)`

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods
    Modifier and Type | Method and Description
    --- | ---
    `static String` | `getExecutable(JobConf conf)` Get the URI of the application's executable.
    `static boolean` | `getIsJavaMapper(JobConf conf)` Check whether the job is using a Java Mapper.
    `static boolean` | `getIsJavaRecordReader(JobConf conf)` Check whether the job is using a Java RecordReader
    `static boolean` | `getIsJavaRecordWriter(JobConf conf)` Will the reduce use a Java RecordWriter?
    `static boolean` | `getIsJavaReducer(JobConf conf)` Check whether the job is using a Java Reducer.
    `static boolean` | `getKeepCommandFile(JobConf conf)` Does the user want to keep the command file for debugging? If this is true, pipes will write a copy of the command data to a file in the task directory named "downlink.data", which may be used to run the C++ program under the debugger.
    `static RunningJob` | `jobSubmit(JobConf conf)` Submit a job to the Map-Reduce framework.
    `static void` | `main(String[] args)` Submit a pipes job based on the command line arguments.
    `int` | `run(String[] args)`
    `static RunningJob` | `runJob(JobConf conf)` Submit a job to the map/reduce cluster.
    `static void` | `setExecutable(JobConf conf, String executable)` Set the URI for the application's executable.
    `static void` | `setIsJavaMapper(JobConf conf, boolean value)` Set whether the Mapper is written in Java.
    `static void` | `setIsJavaRecordReader(JobConf conf, boolean value)` Set whether the job is using a Java RecordReader.
    `static void` | `setIsJavaRecordWriter(JobConf conf, boolean value)` Set whether the job will use a Java RecordWriter.
    `static void` | `setIsJavaReducer(JobConf conf, boolean value)` Set whether the Reducer is written in Java.
    `static void` | `setKeepCommandFile(JobConf conf, boolean keep)` Set whether to keep the command file for debugging
    `static RunningJob` | `submitJob(JobConf conf)` Deprecated. Use `runJob(JobConf)`

    - ### Methods inherited from class org.apache.hadoop.conf.Configured

      `getConf, setConf`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

    - ### Methods inherited from interface org.apache.hadoop.conf.Configurable

      `getConf, setConf`

- - ### Field Detail

    - #### LOG

          protected static final org.slf4j.Logger LOG

    - #### PRESERVE_COMMANDFILE

          public static final String PRESERVE_COMMANDFILE

      See Also:
        Constant Field Values

    - #### EXECUTABLE

          public static final String EXECUTABLE

      See Also:
        Constant Field Values

    - #### INTERPRETOR

          public static final String INTERPRETOR

      See Also:
        Constant Field Values

    - #### IS_JAVA_MAP

          public static final String IS_JAVA_MAP

      See Also:
        Constant Field Values

    - #### IS_JAVA_RR

          public static final String IS_JAVA_RR

      See Also:
        Constant Field Values

    - #### IS_JAVA_RW

          public static final String IS_JAVA_RW

      See Also:
        Constant Field Values

    - #### IS_JAVA_REDUCE

          public static final String IS_JAVA_REDUCE

      See Also:
        Constant Field Values

    - #### PARTITIONER

          public static final String PARTITIONER

      See Also:
        Constant Field Values

    - #### INPUT_FORMAT

          public static final String INPUT_FORMAT

      See Also:
        Constant Field Values

    - #### PORT

          public static final String PORT

      See Also:
        Constant Field Values

  - ### Constructor Detail

    - #### Submitter

          public Submitter()

    - #### Submitter

          public Submitter(org.apache.hadoop.conf.Configuration conf)

  - ### Method Detail

    - #### getExecutable

          public static String getExecutable(JobConf conf)

      Get the URI of the application's executable.
      Parameters:
        `conf` -
      Returns:
        the URI where the application's executable is located

    - #### setExecutable

          public static void setExecutable(JobConf conf,
                                           String executable)

      Set the URI for the application's executable. Normally this is a hdfs: location.
      Parameters:
        `conf` -
        `executable` - The URI of the application's executable.

    - #### setIsJavaRecordReader

          public static void setIsJavaRecordReader(JobConf conf,
                                                   boolean value)

      Set whether the job is using a Java RecordReader.
      Parameters:
        `conf` - the configuration to modify
        `value` - the new value

    - #### getIsJavaRecordReader

          public static boolean getIsJavaRecordReader(JobConf conf)

      Check whether the job is using a Java RecordReader
      Parameters:
        `conf` - the configuration to check
      Returns:
        is it a Java RecordReader?

    - #### setIsJavaMapper

          public static void setIsJavaMapper(JobConf conf,
                                             boolean value)

      Set whether the Mapper is written in Java.
      Parameters:
        `conf` - the configuration to modify
        `value` - the new value

    - #### getIsJavaMapper

          public static boolean getIsJavaMapper(JobConf conf)

      Check whether the job is using a Java Mapper.
      Parameters:
        `conf` - the configuration to check
      Returns:
        is it a Java Mapper?

    - #### setIsJavaReducer

          public static void setIsJavaReducer(JobConf conf,
                                              boolean value)

      Set whether the Reducer is written in Java.
      Parameters:
        `conf` - the configuration to modify
        `value` - the new value

    - #### getIsJavaReducer

          public static boolean getIsJavaReducer(JobConf conf)

      Check whether the job is using a Java Reducer.
      Parameters:
        `conf` - the configuration to check
      Returns:
        is it a Java Reducer?

    - #### setIsJavaRecordWriter

          public static void setIsJavaRecordWriter(JobConf conf,
                                                   boolean value)

      Set whether the job will use a Java RecordWriter.
      Parameters:
        `conf` - the configuration to modify
        `value` - the new value to set

    - #### getIsJavaRecordWriter

          public static boolean getIsJavaRecordWriter(JobConf conf)

      Will the reduce use a Java RecordWriter?
      Parameters:
        `conf` - the configuration to check
      Returns:
        true, if the output of the job will be written by Java

    - #### getKeepCommandFile

          public static boolean getKeepCommandFile(JobConf conf)

      Does the user want to keep the command file for debugging? If this is true, pipes will write a copy of the command data to a file in the task directory named "downlink.data", which may be used to run the C++ program under the debugger. You probably also want to set JobConf.setKeepFailedTaskFiles(true) to keep the entire directory from being deleted. To run using the data file, set the environment variable "mapreduce.pipes.commandfile" to point to the file.
      Parameters:
        `conf` - the configuration to check
      Returns:
        will the framework save the command file?

    - #### setKeepCommandFile

          public static void setKeepCommandFile(JobConf conf,
                                                boolean keep)

      Set whether to keep the command file for debugging
      Parameters:
        `conf` - the configuration to modify
        `keep` - the new value

    - #### submitJob

          @Deprecated
          public static RunningJob submitJob(JobConf conf)
                                                  throws IOException

      Deprecated. Use `runJob(JobConf)`
      Submit a job to the map/reduce cluster. All of the necessary modifications to the job to run under pipes are made to the configuration.
      Parameters:
        `conf` - the job to submit to the cluster (MODIFIED)
      Throws:
        `IOException`

    - #### runJob

          public static RunningJob runJob(JobConf conf)
                                   throws IOException

      Submit a job to the map/reduce cluster. All of the necessary modifications to the job to run under pipes are made to the configuration.
      Parameters:
        `conf` - the job to submit to the cluster (MODIFIED)
      Throws:
        `IOException`

    - #### jobSubmit

          public static RunningJob jobSubmit(JobConf conf)
                                      throws IOException

      Submit a job to the Map-Reduce framework. This returns a handle to the `RunningJob` which can be used to track the running-job.
      Parameters:
        `conf` - the job configuration.
      Returns:
        a handle to the `RunningJob` which can be used to track the running-job.
      Throws:
        `IOException`

    - #### run

          public int run(String[] args)
                  throws Exception

      Specified by:
        `run` in interface `org.apache.hadoop.util.Tool`
      Throws:
        `Exception`

    - #### main

          public static void main(String[] args)
                           throws Exception

      Submit a pipes job based on the command line arguments.
      Parameters:
        `args` -
      Throws:
        `Exception`

Copyright © 2008–2022 Apache Software Foundation. All rights reserved.