org.apache.hadoop.mapreduce.lib.chain

## Class Chain

- java.lang.Object
- - org.apache.hadoop.mapreduce.lib.chain.Chain

-     @InterfaceAudience.Private
       @InterfaceStability.Unstable
      public class Chain
      extends Object

  The Chain class provides all the common functionality for the `ChainMapper` and the `ChainReducer` classes.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected static String` | `CHAIN_MAPPER`
    `protected static String` | `CHAIN_MAPPER_CLASS`
    `protected static String` | `CHAIN_MAPPER_CONFIG`
    `protected static String` | `CHAIN_MAPPER_SIZE`
    `protected static String` | `CHAIN_REDUCER`
    `protected static String` | `CHAIN_REDUCER_CLASS`
    `protected static String` | `CHAIN_REDUCER_CONFIG`
    `protected boolean` | `isMap`
    `protected static String` | `MAPPER_INPUT_KEY_CLASS`
    `protected static String` | `MAPPER_INPUT_VALUE_CLASS`
    `protected static String` | `MAPPER_OUTPUT_KEY_CLASS`
    `protected static String` | `MAPPER_OUTPUT_VALUE_CLASS`
    `protected static String` | `REDUCER_INPUT_KEY_CLASS`
    `protected static String` | `REDUCER_INPUT_VALUE_CLASS`
    `protected static String` | `REDUCER_OUTPUT_KEY_CLASS`
    `protected static String` | `REDUCER_OUTPUT_VALUE_CLASS`

  - ### Constructor Summary

    Constructors
    Modifier | Constructor and Description
    --- | ---
    `protected` | `Chain(boolean isMap)` Creates a Chain instance configured for a Mapper or a Reducer.

  - ### Method Summary

    All Methods Static Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `protected static void` | `addMapper(boolean isMap, Job job, Class<? extends Mapper> klass, Class<?> inputKeyClass, Class<?> inputValueClass, Class<?> outputKeyClass, Class<?> outputValueClass, org.apache.hadoop.conf.Configuration mapperConf)` Adds a Mapper class to the chain job.
    `protected static void` | `checkReducerAlreadySet(boolean isMap, org.apache.hadoop.conf.Configuration jobConf, String prefix, boolean shouldSet)`
    `protected static org.apache.hadoop.conf.Configuration` | `getChainElementConf(org.apache.hadoop.conf.Configuration jobConf, String confKey)` Creates a `Configuration` for the Map or Reduce in the chain.
    `protected static int` | `getIndex(org.apache.hadoop.conf.Configuration conf, String prefix)`
    `protected static String` | `getPrefix(boolean isMap)` Returns the prefix to use for the configuration of the chain depending if it is for a Mapper or a Reducer.
    `protected static void` | `setMapperConf(boolean isMap, org.apache.hadoop.conf.Configuration jobConf, Class<?> inputKeyClass, Class<?> inputValueClass, Class<?> outputKeyClass, Class<?> outputValueClass, org.apache.hadoop.conf.Configuration mapperConf, int index, String prefix)`
    `protected static void` | `setReducer(Job job, Class<? extends Reducer> klass, Class<?> inputKeyClass, Class<?> inputValueClass, Class<?> outputKeyClass, Class<?> outputValueClass, org.apache.hadoop.conf.Configuration reducerConf)` Sets the Reducer class to the chain job.
    `protected static void` | `setReducerConf(org.apache.hadoop.conf.Configuration jobConf, Class<?> inputKeyClass, Class<?> inputValueClass, Class<?> outputKeyClass, Class<?> outputValueClass, org.apache.hadoop.conf.Configuration reducerConf, String prefix)`
    `protected static void` | `validateKeyValueTypes(boolean isMap, org.apache.hadoop.conf.Configuration jobConf, Class<?> inputKeyClass, Class<?> inputValueClass, Class<?> outputKeyClass, Class<?> outputValueClass, int index, String prefix)`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### CHAIN_MAPPER

          protected static final String CHAIN_MAPPER

      See Also:
        Constant Field Values

    - #### CHAIN_REDUCER

          protected static final String CHAIN_REDUCER

      See Also:
        Constant Field Values

    - #### CHAIN_MAPPER_SIZE

          protected static final String CHAIN_MAPPER_SIZE

      See Also:
        Constant Field Values

    - #### CHAIN_MAPPER_CLASS

          protected static final String CHAIN_MAPPER_CLASS

      See Also:
        Constant Field Values

    - #### CHAIN_MAPPER_CONFIG

          protected static final String CHAIN_MAPPER_CONFIG

      See Also:
        Constant Field Values

    - #### CHAIN_REDUCER_CLASS

          protected static final String CHAIN_REDUCER_CLASS

      See Also:
        Constant Field Values

    - #### CHAIN_REDUCER_CONFIG

          protected static final String CHAIN_REDUCER_CONFIG

      See Also:
        Constant Field Values

    - #### MAPPER_INPUT_KEY_CLASS

          protected static final String MAPPER_INPUT_KEY_CLASS

      See Also:
        Constant Field Values

    - #### MAPPER_INPUT_VALUE_CLASS

          protected static final String MAPPER_INPUT_VALUE_CLASS

      See Also:
        Constant Field Values

    - #### MAPPER_OUTPUT_KEY_CLASS

          protected static final String MAPPER_OUTPUT_KEY_CLASS

      See Also:
        Constant Field Values

    - #### MAPPER_OUTPUT_VALUE_CLASS

          protected static final String MAPPER_OUTPUT_VALUE_CLASS

      See Also:
        Constant Field Values

    - #### REDUCER_INPUT_KEY_CLASS

          protected static final String REDUCER_INPUT_KEY_CLASS

      See Also:
        Constant Field Values

    - #### REDUCER_INPUT_VALUE_CLASS

          protected static final String REDUCER_INPUT_VALUE_CLASS

      See Also:
        Constant Field Values

    - #### REDUCER_OUTPUT_KEY_CLASS

          protected static final String REDUCER_OUTPUT_KEY_CLASS

      See Also:
        Constant Field Values

    - #### REDUCER_OUTPUT_VALUE_CLASS

          protected static final String REDUCER_OUTPUT_VALUE_CLASS

      See Also:
        Constant Field Values

    - #### isMap

          protected boolean isMap

  - ### Constructor Detail

    - #### Chain

          protected Chain(boolean isMap)

      Creates a Chain instance configured for a Mapper or a Reducer.
      Parameters:
        `isMap` - TRUE indicates the chain is for a Mapper, FALSE that is for a Reducer.

  - ### Method Detail

    - #### getPrefix

          protected static String getPrefix(boolean isMap)

      Returns the prefix to use for the configuration of the chain depending if it is for a Mapper or a Reducer.
      Parameters:
        `isMap` - TRUE for Mapper, FALSE for Reducer.
      Returns:
        the prefix to use.

    - #### getIndex

          protected static int getIndex(org.apache.hadoop.conf.Configuration conf,
                                        String prefix)

    - #### getChainElementConf

          protected static org.apache.hadoop.conf.Configuration getChainElementConf(org.apache.hadoop.conf.Configuration jobConf,
                                                                                    String confKey)

      Creates a `Configuration` for the Map or Reduce in the chain.

      It creates a new Configuration using the chain job's Configuration as base and adds to it the configuration properties for the chain element. The keys of the chain element Configuration have precedence over the given Configuration.

      Parameters:
        `jobConf` - the chain job's Configuration.
        `confKey` - the key for chain element configuration serialized in the chain job's Configuration.
      Returns:
        a new Configuration aggregating the chain job's Configuration with the chain element configuration properties.

    - #### addMapper

          protected static void addMapper(boolean isMap,
                                          Job job,
                                          Class<? extends Mapper> klass,
                                          Class<?> inputKeyClass,
                                          Class<?> inputValueClass,
                                          Class<?> outputKeyClass,
                                          Class<?> outputValueClass,
                                          org.apache.hadoop.conf.Configuration mapperConf)

      Adds a Mapper class to the chain job.

      The configuration properties of the chain job have precedence over the configuration properties of the Mapper.

      Parameters:
        `isMap` - indicates if the Chain is for a Mapper or for a Reducer.
        `job` - chain job.
        `klass` - the Mapper class to add.
        `inputKeyClass` - mapper input key class.
        `inputValueClass` - mapper input value class.
        `outputKeyClass` - mapper output key class.
        `outputValueClass` - mapper output value class.
        `mapperConf` - a configuration for the Mapper class. It is recommended to use a Configuration without default values using the `Configuration(boolean loadDefaults)` constructor with FALSE.

    - #### checkReducerAlreadySet

          protected static void checkReducerAlreadySet(boolean isMap,
                                                       org.apache.hadoop.conf.Configuration jobConf,
                                                       String prefix,
                                                       boolean shouldSet)

    - #### validateKeyValueTypes

          protected static void validateKeyValueTypes(boolean isMap,
                                                      org.apache.hadoop.conf.Configuration jobConf,
                                                      Class<?> inputKeyClass,
                                                      Class<?> inputValueClass,
                                                      Class<?> outputKeyClass,
                                                      Class<?> outputValueClass,
                                                      int index,
                                                      String prefix)

    - #### setMapperConf

          protected static void setMapperConf(boolean isMap,
                                              org.apache.hadoop.conf.Configuration jobConf,
                                              Class<?> inputKeyClass,
                                              Class<?> inputValueClass,
                                              Class<?> outputKeyClass,
                                              Class<?> outputValueClass,
                                              org.apache.hadoop.conf.Configuration mapperConf,
                                              int index,
                                              String prefix)

    - #### setReducer

          protected static void setReducer(Job job,
                                           Class<? extends Reducer> klass,
                                           Class<?> inputKeyClass,
                                           Class<?> inputValueClass,
                                           Class<?> outputKeyClass,
                                           Class<?> outputValueClass,
                                           org.apache.hadoop.conf.Configuration reducerConf)

      Sets the Reducer class to the chain job.

      The configuration properties of the chain job have precedence over the configuration properties of the Reducer.

      Parameters:
        `job` - the chain job.
        `klass` - the Reducer class to add.
        `inputKeyClass` - reducer input key class.
        `inputValueClass` - reducer input value class.
        `outputKeyClass` - reducer output key class.
        `outputValueClass` - reducer output value class.
        `reducerConf` - a configuration for the Reducer class. It is recommended to use a Configuration without default values using the `Configuration(boolean loadDefaults)` constructor with FALSE.

    - #### setReducerConf

          protected static void setReducerConf(org.apache.hadoop.conf.Configuration jobConf,
                                               Class<?> inputKeyClass,
                                               Class<?> inputValueClass,
                                               Class<?> outputKeyClass,
                                               Class<?> outputValueClass,
                                               org.apache.hadoop.conf.Configuration reducerConf,
                                               String prefix)

Copyright © 2008–2022 Apache Software Foundation. All rights reserved.