org.apache.tez.mapreduce.input

Class MultiMRInput

java.lang.Object

org.apache.tez.runtime.api.AbstractLogicalInput

org.apache.tez.mapreduce.input.base.MRInputBase

org.apache.tez.mapreduce.input.MultiMRInput

All Implemented Interfaces:

org.apache.tez.runtime.api.Input, org.apache.tez.runtime.api.InputFrameworkInterface, org.apache.tez.runtime.api.LogicalInput, org.apache.tez.runtime.api.LogicalInputFrameworkInterface

@InterfaceAudience.Public
@InterfaceStability.Evolving
public class MultiMRInput
extends org.apache.tez.mapreduce.input.base.MRInputBase

Field Summary

Fields inherited from class org.apache.tez.mapreduce.input.base.MRInputBase

inputRecordCounter, jobConf

Constructor Summary

Constructors

Constructor and Description
MultiMRInput(org.apache.tez.runtime.api.InputContext inputContext, int numPhysicalInputs)

Method Summary

Methods

Modifier and Type	Method and Description
List<org.apache.tez.runtime.api.Event>	close()
static org.apache.tez.mapreduce.input.MultiMRInput.MultiMRInputConfigBuilder	createConfigBuilder(org.apache.hadoop.conf.Configuration conf, Class<?> inputFormat) Create an MultiMRInput.MultiMRInputConfigBuilder to configure a MultiMRInput
Collection<org.apache.tez.runtime.library.api.KeyValueReader>	getKeyValueReaders()
org.apache.tez.runtime.api.Reader	getReader()
void	handleEvents(List<org.apache.tez.runtime.api.Event> inputEvents)
List<org.apache.tez.runtime.api.Event>	initialize()
void	start()

Methods inherited from class org.apache.tez.runtime.api.AbstractLogicalInput

getContext, getNumPhysicalInputs

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MultiMRInput

public MultiMRInput(org.apache.tez.runtime.api.InputContext inputContext,
            int numPhysicalInputs)

Method Detail

createConfigBuilder

public static org.apache.tez.mapreduce.input.MultiMRInput.MultiMRInputConfigBuilder createConfigBuilder(org.apache.hadoop.conf.Configuration conf,
                                                                                        @Nullable
                                                                                        Class<?> inputFormat)

Create an MultiMRInput.MultiMRInputConfigBuilder to configure a MultiMRInput

The preferred usage model is to provide all of the parameters, and use methods to configure the Input.

For legacy applications, which may already have a fully configured Configuration instance, the inputFormat can be specified as null

Typically, this will be used along with a custom VertexManagerPlugin or InputInitializer to generate the multiple inputs to be used by each task. If this is not setup, this will work the same as MRInput

Grouping of splits is disabled by default.

Parameters:

conf - Configuration for the MRInput. This configuration instance will be modified in place

inputFormat - InputFormat derived class. This can be null. If the InputFormat specified is null, the provided configuration should be complete.

Returns:

MultiMRInput.MultiMRInputConfigBuilder

initialize

public List<org.apache.tez.runtime.api.Event> initialize()
                                                  throws IOException

Specified by:

initialize in interface org.apache.tez.runtime.api.InputFrameworkInterface

Overrides:

initialize in class org.apache.tez.mapreduce.input.base.MRInputBase

Throws:

IOException

getKeyValueReaders

public Collection<org.apache.tez.runtime.library.api.KeyValueReader> getKeyValueReaders()
                                                                                 throws InterruptedException,
                                                                                        IOException

Throws:

InterruptedException

IOException

getReader

public org.apache.tez.runtime.api.Reader getReader()
                                            throws Exception

Specified by:

getReader in interface org.apache.tez.runtime.api.Input

Overrides:

getReader in class org.apache.tez.mapreduce.input.base.MRInputBase

Throws:

Exception

handleEvents

public void handleEvents(List<org.apache.tez.runtime.api.Event> inputEvents)
                  throws Exception

Throws:

Exception

close

public List<org.apache.tez.runtime.api.Event> close()
                                             throws Exception

Throws:

Exception

start

public void start()
           throws Exception

Throws:

Exception