org.apache.avro.mapred

## Class AvroMultipleInputs

- java.lang.Object
- - org.apache.avro.mapred.AvroMultipleInputs

-     public class AvroMultipleInputs
      extends Object

  This class supports Avro-MapReduce jobs that have multiple input paths with a different `Schema` and `AvroMapper` for each path.

  Usage:

  Case 1: (ReflectData based inputs)

       // Enable ReflectData usage across job.
       AvroJob.setReflect(job);

       Schema type1Schema = ReflectData.get().getSchema(Type1Record.class)
       AvroMultipleInputs.addInputPath(job, inputPath1, type1Schema, Type1AvroMapper.class);

  Where Type1AvroMapper would be implemented as

        class Type1AvroMapper extends AvroMapper<Type1Record, Pair<ComparingKeyRecord, CommonValueRecord>>

       Schema type2Schema = ReflectData.get().getSchema(Type2Record.class)
       AvroMultipleInputs.addInputPath(job, inputPath2, type2Schema, Type2AvroMapper.class);

  Where Type2AvroMapper would be implemented as

        class Type2AvroMapper extends AvroMapper<Type2Record, Pair<ComparingKeyRecord, CommonValueRecord>>

  Case 2: (SpecificData based inputs)

       Schema type1Schema = Type1Record.SCHEMA$;
       AvroMultipleInputs.addInputPath(job, inputPath1, type1Schema, Type1AvroMapper.class);

  Where Type1AvroMapper would be implemented as

        class Type1AvroMapper extends AvroMapper<Type1Record, Pair<ComparingKeyRecord, CommonValueRecord>>

       Schema type2Schema = Type2Record.SCHEMA$;
       AvroMultipleInputs.addInputPath(job, inputPath2, type2Schema, Type2AvroMapper.class);

  Where Type2AvroMapper would be implemented as

        class Type2AvroMapper extends AvroMapper<Type2Record, Pair<ComparingKeyRecord, CommonValueRecord>>

  Note on InputFormat: The InputFormat used will always be `AvroInputFormat` when using this class.

  Note on collector outputs: When using this class, you will need to ensure that the mapper implementations involved must all emit the same Key type and Value record types, as set by `AvroJob.setOutputSchema(JobConf, Schema)` or `AvroJob.setMapOutputSchema(JobConf, Schema)`.

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `AvroMultipleInputs()`

  - ### Method Summary

    All Methods Static Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `static void` | `addInputPath(JobConf conf, Path path, Class<? extends AvroMapper> mapperClass, Schema inputSchema)` Add a `Path` with a custom `Schema` and `AvroMapper` to the list of inputs for the map-reduce job.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### AvroMultipleInputs

          public AvroMultipleInputs()

  - ### Method Detail

    - #### addInputPath

          public static void addInputPath(JobConf conf,
                                          Path path,
                                          Class<? extends AvroMapper> mapperClass,
                                          Schema inputSchema)

      Add a `Path` with a custom `Schema` and `AvroMapper` to the list of inputs for the map-reduce job.
      Parameters:
        `conf` - The configuration of the job
        `path` - `Path` to be added to the list of inputs for the job
        `inputSchema` - `Schema` to use for this path
        `mapperClass` - `AvroMapper` class to use for this path

Copyright © 2009–2024 The Apache Software Foundation. All rights reserved.