org.apache.hadoop.mapred

## Class YarnOutputFiles

- java.lang.Object
- - org.apache.hadoop.mapred.MapOutputFile
  - - org.apache.hadoop.mapred.YarnOutputFiles

- All Implemented Interfaces:
    org.apache.hadoop.conf.Configurable

      @InterfaceAudience.Private
       @InterfaceStability.Unstable
      public class YarnOutputFiles
      extends org.apache.hadoop.mapred.MapOutputFile

  Manipulate the working area for the transient store for maps and reduces. This class is used by map and reduce tasks to identify the directories that they need to write to/read from for intermediate files. The callers of these methods are from child space.

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `YarnOutputFiles()`

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `org.apache.hadoop.conf.Configuration` | `getConf()`
    `org.apache.hadoop.fs.Path` | `getInputFile(int mapId)` Return a local reduce input file created earlier
    `org.apache.hadoop.fs.Path` | `getInputFileForWrite(org.apache.hadoop.mapreduce.TaskID mapId, long size)` Create a local reduce input file name.
    `org.apache.hadoop.fs.Path` | `getOutputFile()` Return the path to local map output file created earlier
    `org.apache.hadoop.fs.Path` | `getOutputFileForWrite(long size)` Create a local map output file name.
    `org.apache.hadoop.fs.Path` | `getOutputFileForWriteInVolume(org.apache.hadoop.fs.Path existing)` Create a local map output file name on the same volume.
    `org.apache.hadoop.fs.Path` | `getOutputIndexFile()` Return the path to a local map output index file created earlier
    `org.apache.hadoop.fs.Path` | `getOutputIndexFileForWrite(long size)` Create a local map output index file name.
    `org.apache.hadoop.fs.Path` | `getOutputIndexFileForWriteInVolume(org.apache.hadoop.fs.Path existing)` Create a local map output index file name on the same volume.
    `org.apache.hadoop.fs.Path` | `getSpillFile(int spillNumber)` Return a local map spill file created earlier.
    `org.apache.hadoop.fs.Path` | `getSpillFileForWrite(int spillNumber, long size)` Create a local map spill file name.
    `org.apache.hadoop.fs.Path` | `getSpillIndexFile(int spillNumber)` Return a local map spill index file created earlier
    `org.apache.hadoop.fs.Path` | `getSpillIndexFileForWrite(int spillNumber, long size)` Create a local map spill index file name.
    `void` | `removeAll()` Removes all of the files related to a task.
    `void` | `setConf(org.apache.hadoop.conf.Configuration conf)`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### YarnOutputFiles

          public YarnOutputFiles()

  - ### Method Detail

    - #### getOutputFile

          public org.apache.hadoop.fs.Path getOutputFile()
                                                  throws IOException

      Return the path to local map output file created earlier
      Specified by:
        `getOutputFile` in class `org.apache.hadoop.mapred.MapOutputFile`
      Returns:
        path
      Throws:
        `IOException`

    - #### getOutputFileForWrite

          public org.apache.hadoop.fs.Path getOutputFileForWrite(long size)
                                                          throws IOException

      Create a local map output file name.
      Specified by:
        `getOutputFileForWrite` in class `org.apache.hadoop.mapred.MapOutputFile`
      Parameters:
        `size` - the size of the file
      Returns:
        path
      Throws:
        `IOException`

    - #### getOutputFileForWriteInVolume

          public org.apache.hadoop.fs.Path getOutputFileForWriteInVolume(org.apache.hadoop.fs.Path existing)

      Create a local map output file name on the same volume.
      Specified by:
        `getOutputFileForWriteInVolume` in class `org.apache.hadoop.mapred.MapOutputFile`

    - #### getOutputIndexFile

          public org.apache.hadoop.fs.Path getOutputIndexFile()
                                                       throws IOException

      Return the path to a local map output index file created earlier
      Specified by:
        `getOutputIndexFile` in class `org.apache.hadoop.mapred.MapOutputFile`
      Returns:
        path
      Throws:
        `IOException`

    - #### getOutputIndexFileForWrite

          public org.apache.hadoop.fs.Path getOutputIndexFileForWrite(long size)
                                                               throws IOException

      Create a local map output index file name.
      Specified by:
        `getOutputIndexFileForWrite` in class `org.apache.hadoop.mapred.MapOutputFile`
      Parameters:
        `size` - the size of the file
      Returns:
        path
      Throws:
        `IOException`

    - #### getOutputIndexFileForWriteInVolume

          public org.apache.hadoop.fs.Path getOutputIndexFileForWriteInVolume(org.apache.hadoop.fs.Path existing)

      Create a local map output index file name on the same volume.
      Specified by:
        `getOutputIndexFileForWriteInVolume` in class `org.apache.hadoop.mapred.MapOutputFile`

    - #### getSpillFile

          public org.apache.hadoop.fs.Path getSpillFile(int spillNumber)
                                                 throws IOException

      Return a local map spill file created earlier.
      Specified by:
        `getSpillFile` in class `org.apache.hadoop.mapred.MapOutputFile`
      Parameters:
        `spillNumber` - the number
      Returns:
        path
      Throws:
        `IOException`

    - #### getSpillFileForWrite

          public org.apache.hadoop.fs.Path getSpillFileForWrite(int spillNumber,
                                                                long size)
                                                         throws IOException

      Create a local map spill file name.
      Specified by:
        `getSpillFileForWrite` in class `org.apache.hadoop.mapred.MapOutputFile`
      Parameters:
        `spillNumber` - the number
        `size` - the size of the file
      Returns:
        path
      Throws:
        `IOException`

    - #### getSpillIndexFile

          public org.apache.hadoop.fs.Path getSpillIndexFile(int spillNumber)
                                                      throws IOException

      Return a local map spill index file created earlier
      Specified by:
        `getSpillIndexFile` in class `org.apache.hadoop.mapred.MapOutputFile`
      Parameters:
        `spillNumber` - the number
      Returns:
        path
      Throws:
        `IOException`

    - #### getSpillIndexFileForWrite

          public org.apache.hadoop.fs.Path getSpillIndexFileForWrite(int spillNumber,
                                                                     long size)
                                                              throws IOException

      Create a local map spill index file name.
      Specified by:
        `getSpillIndexFileForWrite` in class `org.apache.hadoop.mapred.MapOutputFile`
      Parameters:
        `spillNumber` - the number
        `size` - the size of the file
      Returns:
        path
      Throws:
        `IOException`

    - #### getInputFile

          public org.apache.hadoop.fs.Path getInputFile(int mapId)
                                                 throws IOException

      Return a local reduce input file created earlier
      Specified by:
        `getInputFile` in class `org.apache.hadoop.mapred.MapOutputFile`
      Parameters:
        `mapId` - a map task id
      Returns:
        path
      Throws:
        `IOException`

    - #### getInputFileForWrite

          public org.apache.hadoop.fs.Path getInputFileForWrite(org.apache.hadoop.mapreduce.TaskID mapId,
                                                                long size)
                                                         throws IOException

      Create a local reduce input file name.
      Specified by:
        `getInputFileForWrite` in class `org.apache.hadoop.mapred.MapOutputFile`
      Parameters:
        `mapId` - a map task id
        `size` - the size of the file
      Returns:
        path
      Throws:
        `IOException`

    - #### removeAll

          public void removeAll()
                         throws IOException

      Removes all of the files related to a task.
      Specified by:
        `removeAll` in class `org.apache.hadoop.mapred.MapOutputFile`
      Throws:
        `IOException`

    - #### setConf

          public void setConf(org.apache.hadoop.conf.Configuration conf)

      Specified by:
        `setConf` in interface `org.apache.hadoop.conf.Configurable`
      Overrides:
        `setConf` in class `org.apache.hadoop.mapred.MapOutputFile`

    - #### getConf

          public org.apache.hadoop.conf.Configuration getConf()

      Specified by:
        `getConf` in interface `org.apache.hadoop.conf.Configurable`
      Overrides:
        `getConf` in class `org.apache.hadoop.mapred.MapOutputFile`

Copyright © 2008–2022 Apache Software Foundation. All rights reserved.