org.apache.tez.mapreduce.input

Class MRInput.MRInputConfigBuilder

java.lang.Object

org.apache.tez.mapreduce.input.MRInput.MRInputConfigBuilder

Direct Known Subclasses:

MultiMRInput.MultiMRInputConfigBuilder

Enclosing class:

MRInput

public static class MRInput.MRInputConfigBuilder
extends Object

Helper class to configure MRInput

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.tez.dag.api.DataSourceDescriptor	build() Create the DataSourceDescriptor
MRInput.MRInputConfigBuilder	generateSplitsInAM(boolean value) Set whether splits should be generated in the Tez App Master (default true)
MRInput.MRInputConfigBuilder	getCredentialsForSourceFileSystem(boolean value) Get the credentials for the inputPaths from their FileSystems Use the method to turn this off when not using a FileSystem or when Credentials are not supported
MRInput.MRInputConfigBuilder	groupSplits(boolean value) Set whether splits should be grouped (default true)
MRInput.MRInputConfigBuilder	setCustomInitializerDescriptor(org.apache.tez.dag.api.InputInitializerDescriptor customInitializerDescriptor) This method is intended to be used in case a custom InputInitializer is being used along with MRInput.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

groupSplits

public MRInput.MRInputConfigBuilder groupSplits(boolean value)

Set whether splits should be grouped (default true)

Parameters:

value - whether to group splits in the AM or not

Returns:

MRInput.MRInputConfigBuilder

generateSplitsInAM

public MRInput.MRInputConfigBuilder generateSplitsInAM(boolean value)

Set whether splits should be generated in the Tez App Master (default true)

Parameters:

value - whether to generate splits in the AM or not

Returns:

MRInput.MRInputConfigBuilder

getCredentialsForSourceFileSystem

public MRInput.MRInputConfigBuilder getCredentialsForSourceFileSystem(boolean value)

Get the credentials for the inputPaths from their FileSystems Use the method to turn this off when not using a FileSystem or when Credentials are not supported

Parameters:

value - whether to get credentials or not. (true by default)

Returns:

MRInput.MRInputConfigBuilder

setCustomInitializerDescriptor

public MRInput.MRInputConfigBuilder setCustomInitializerDescriptor(org.apache.tez.dag.api.InputInitializerDescriptor customInitializerDescriptor)

This method is intended to be used in case a custom InputInitializer is being used along with MRInput. If a custom descriptor is used, the config builder will not be able to setup location hints, parallelism, etc, and configuring the Vertex on which this Input is used is the responsibility of the user. Credential fetching can be controlled via the getCredentialsForSourceFilesystem method. Whether grouping is enabled or not can be controlled via groupSplitsInAM method.

Parameters:

customInitializerDescriptor - the initializer descriptor

Returns:

MRInput.MRInputConfigBuilder

build

public org.apache.tez.dag.api.DataSourceDescriptor build()

Create the DataSourceDescriptor

Returns:

DataSourceDescriptor