org.apache.avro.mapred

## Class AvroOutputFormat<T>

- java.lang.Object
- - org.apache.hadoop.mapred.FileOutputFormat<AvroWrapper<T>,NullWritable>
  - - org.apache.avro.mapred.AvroOutputFormat<T>

- All Implemented Interfaces:
    OutputFormat<AvroWrapper<T>,NullWritable>

      public class AvroOutputFormat<T>
      extends FileOutputFormat<AvroWrapper<T>,NullWritable>

  An `OutputFormat` for Avro data files.

  You can specify various options using Job Configuration properties. Look at the fields in `AvroJob` as well as this class to get an overview of the supported options.

- - ### Nested Class Summary

    - ### Nested classes/interfaces inherited from class org.apache.hadoop.mapred.FileOutputFormat

      `FileOutputFormat.Counter`

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static String` | `DEFLATE_LEVEL_KEY` The configuration key for Avro deflate level.
    `static String` | `EXT` The file name extension for avro data files.
    `static String` | `SYNC_INTERVAL_KEY` The configuration key for Avro sync interval.
    `static String` | `XZ_LEVEL_KEY` The configuration key for Avro XZ level.
    `static String` | `ZSTD_BUFFERPOOL_KEY` The configuration key for Avro ZSTD buffer pool.
    `static String` | `ZSTD_LEVEL_KEY` The configuration key for Avro ZSTD level.

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `AvroOutputFormat()`

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `RecordWriter<AvroWrapper<T>,NullWritable>` | `getRecordWriter(FileSystem ignore, JobConf job, String name, Progressable prog)`
    `static void` | `setDeflateLevel(JobConf job, int level)` Enable output compression using the deflate codec and specify its level.
    `static void` | `setSyncInterval(JobConf job, int syncIntervalInBytes)` Set the sync interval to be used by the underlying `DataFileWriter`.

    - ### Methods inherited from class org.apache.hadoop.mapred.FileOutputFormat

      `checkOutputSpecs, getCompressOutput, getOutputCompressorClass, getOutputPath, getPathForCustomFile, getTaskOutputPath, getUniqueName, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputPath, setWorkOutputPath`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### EXT

          public static final String EXT

      The file name extension for avro data files.
      See Also:
        Constant Field Values

    - #### DEFLATE_LEVEL_KEY

          public static final String DEFLATE_LEVEL_KEY

      The configuration key for Avro deflate level.
      See Also:
        Constant Field Values

    - #### XZ_LEVEL_KEY

          public static final String XZ_LEVEL_KEY

      The configuration key for Avro XZ level.
      See Also:
        Constant Field Values

    - #### ZSTD_LEVEL_KEY

          public static final String ZSTD_LEVEL_KEY

      The configuration key for Avro ZSTD level.
      See Also:
        Constant Field Values

    - #### ZSTD_BUFFERPOOL_KEY

          public static final String ZSTD_BUFFERPOOL_KEY

      The configuration key for Avro ZSTD buffer pool.
      See Also:
        Constant Field Values

    - #### SYNC_INTERVAL_KEY

          public static final String SYNC_INTERVAL_KEY

      The configuration key for Avro sync interval.
      See Also:
        Constant Field Values

  - ### Constructor Detail

    - #### AvroOutputFormat

          public AvroOutputFormat()

  - ### Method Detail

    - #### setDeflateLevel

          public static void setDeflateLevel(JobConf job,
                                             int level)

      Enable output compression using the deflate codec and specify its level.

    - #### setSyncInterval

          public static void setSyncInterval(JobConf job,
                                             int syncIntervalInBytes)

      Set the sync interval to be used by the underlying `DataFileWriter`.

    - #### getRecordWriter

          public RecordWriter<AvroWrapper<T>,NullWritable> getRecordWriter(FileSystem ignore,
                                                                           JobConf job,
                                                                           String name,
                                                                           Progressable prog)
                                                                    throws IOException

      Specified by:
        `getRecordWriter` in interface `OutputFormat<AvroWrapper<T>,NullWritable>`
      Specified by:
        `getRecordWriter` in class `FileOutputFormat<AvroWrapper<T>,NullWritable>`
      Throws:
        `IOException`

Copyright © 2009–2024 The Apache Software Foundation. All rights reserved.