org.apache.hadoop.mapreduce.lib.output

## Class PathOutputCommitterFactory

- java.lang.Object
- - org.apache.hadoop.conf.Configured
  - - org.apache.hadoop.mapreduce.lib.output.PathOutputCommitterFactory

- All Implemented Interfaces:
    org.apache.hadoop.conf.Configurable
  Direct Known Subclasses:
    FileOutputCommitterFactory, NamedCommitterFactory

      @InterfaceAudience.Private
       @InterfaceStability.Unstable
      public class PathOutputCommitterFactory
      extends org.apache.hadoop.conf.Configured

  A factory for committers implementing the `PathOutputCommitter` methods, and so can be used from `FileOutputFormat`. The base implementation returns `FileOutputCommitter` instances. Algorithm:
  1. If an explicit committer factory is named, it is used.
  2. The output path is examined. If is non null and there is an explicit schema for that filesystem, its factory is instantiated.
  3. Otherwise, an instance of `FileOutputCommitter` is created.
  In `FileOutputFormat`, the created factory has its method `createOutputCommitter(Path, TaskAttemptContext)` with a task attempt context and a possibly null path.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static String` | `COMMITTER_FACTORY_CLASS` Name of the configuration option used to configure the output committer factory to use unless there is a specific one for a schema.
    `static String` | `COMMITTER_FACTORY_DEFAULT` Default committer factory name: "org.apache.hadoop.mapreduce.lib.output.FileOutputCommitterFactory".
    `static String` | `COMMITTER_FACTORY_SCHEME` Scheme prefix for per-filesystem scheme committers.
    `static String` | `COMMITTER_FACTORY_SCHEME_PATTERN` String format pattern for per-filesystem scheme committers.
    `static String` | `FILE_COMMITTER_FACTORY` The `FileOutputCommitter` factory.
    `static String` | `NAMED_COMMITTER_CLASS` The named output committer.
    `static String` | `NAMED_COMMITTER_FACTORY` The `FileOutputCommitter` factory.

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `PathOutputCommitterFactory()`

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `static PathOutputCommitter` | `createCommitter(org.apache.hadoop.fs.Path outputPath, TaskAttemptContext context)` Create the committer factory for a task attempt and destination, then create the committer from it.
    `protected PathOutputCommitter` | `createFileOutputCommitter(org.apache.hadoop.fs.Path outputPath, TaskAttemptContext context)` Create an instance of the default committer, a `FileOutputCommitter` for a task.
    `PathOutputCommitter` | `createOutputCommitter(org.apache.hadoop.fs.Path outputPath, TaskAttemptContext context)` Create an output committer for a task attempt.
    `static PathOutputCommitterFactory` | `getCommitterFactory(org.apache.hadoop.fs.Path outputPath, org.apache.hadoop.conf.Configuration conf)` Get the committer factory for a configuration.

    - ### Methods inherited from class org.apache.hadoop.conf.Configured

      `getConf, setConf`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### COMMITTER_FACTORY_CLASS

          public static final String COMMITTER_FACTORY_CLASS

      Name of the configuration option used to configure the output committer factory to use unless there is a specific one for a schema.
      See Also:
        Constant Field Values

    - #### COMMITTER_FACTORY_SCHEME

          public static final String COMMITTER_FACTORY_SCHEME

      Scheme prefix for per-filesystem scheme committers.
      See Also:
        Constant Field Values

    - #### COMMITTER_FACTORY_SCHEME_PATTERN

          public static final String COMMITTER_FACTORY_SCHEME_PATTERN

      String format pattern for per-filesystem scheme committers.
      See Also:
        Constant Field Values

    - #### FILE_COMMITTER_FACTORY

          public static final String FILE_COMMITTER_FACTORY

      The `FileOutputCommitter` factory.
      See Also:
        Constant Field Values

    - #### NAMED_COMMITTER_FACTORY

          public static final String NAMED_COMMITTER_FACTORY

      The `FileOutputCommitter` factory.
      See Also:
        Constant Field Values

    - #### NAMED_COMMITTER_CLASS

          public static final String NAMED_COMMITTER_CLASS

      The named output committer. Creates any committer listed in
      See Also:
        Constant Field Values

    - #### COMMITTER_FACTORY_DEFAULT

          public static final String COMMITTER_FACTORY_DEFAULT

      Default committer factory name: "org.apache.hadoop.mapreduce.lib.output.FileOutputCommitterFactory".
      See Also:
        Constant Field Values

  - ### Constructor Detail

    - #### PathOutputCommitterFactory

          public PathOutputCommitterFactory()

  - ### Method Detail

    - #### createOutputCommitter

          public PathOutputCommitter createOutputCommitter(org.apache.hadoop.fs.Path outputPath,
                                                           TaskAttemptContext context)
                                                    throws IOException

      Create an output committer for a task attempt.
      Parameters:
        `outputPath` - output path. This may be null.
        `context` - context
      Returns:
        a new committer
      Throws:
        `IOException` - problems instantiating the committer

    - #### createFileOutputCommitter

          protected final PathOutputCommitter createFileOutputCommitter(org.apache.hadoop.fs.Path outputPath,
                                                                        TaskAttemptContext context)
                                                                 throws IOException

      Create an instance of the default committer, a `FileOutputCommitter` for a task.
      Parameters:
        `outputPath` - the task's output path, or or null if no output path has been defined.
        `context` - the task attempt context
      Returns:
        the committer to use
      Throws:
        `IOException` - problems instantiating the committer

    - #### getCommitterFactory

          public static PathOutputCommitterFactory getCommitterFactory(org.apache.hadoop.fs.Path outputPath,
                                                                       org.apache.hadoop.conf.Configuration conf)

      Get the committer factory for a configuration.
      Parameters:
        `outputPath` - the job's output path. If null, it means that the schema is unknown and a per-schema factory cannot be determined.
        `conf` - configuration
      Returns:
        an instantiated committer factory

    - #### createCommitter

          public static PathOutputCommitter createCommitter(org.apache.hadoop.fs.Path outputPath,
                                                            TaskAttemptContext context)
                                                     throws IOException

      Create the committer factory for a task attempt and destination, then create the committer from it.
      Parameters:
        `outputPath` - the task's output path, or or null if no output path has been defined.
        `context` - the task attempt context
      Returns:
        the committer to use
      Throws:
        `IOException` - problems instantiating the committer

Copyright © 2008–2022 Apache Software Foundation. All rights reserved.