org.apache.tez.runtime.library.conf

Interface UnorderedKVInputConfig.SpecificConfigBuilder<T>

All Known Implementing Classes:

UnorderedKVInputConfig.Builder, UnorderedKVInputConfig.SpecificBuilder

Enclosing class:

UnorderedKVInputConfig

@InterfaceAudience.Private
public static interface UnorderedKVInputConfig.SpecificConfigBuilder<T>

Configure parameters which are specific to the Input.

Method Summary

Methods

Modifier and Type	Method and Description
T	setAdditionalConfiguration(Map<String,String> confMap) Used to set additional configuration parameters which are not set via API methods.
T	setAdditionalConfiguration(String key, String value) Used to set additional configuration parameters which are not set via API methods.
T	setFromConfiguration(org.apache.hadoop.conf.Configuration conf) Used to build out a configuration from an existing Hadoop Configuration.
T	setMaxSingleMemorySegmentFraction(float maxSingleSegmentFraction) Sets a size limit on the maximum segment size to be shuffled to disk.
T	setMergeFraction(float mergeFraction) Configure the point at which in memory segments will be merged and written out to a single large disk segment.
T	setShuffleBufferFraction(float shuffleBufferFraction) Sets the buffer fraction, as a fraction of container size, to be used while fetching remote data.

Method Detail

setShuffleBufferFraction

T setShuffleBufferFraction(float shuffleBufferFraction)

Sets the buffer fraction, as a fraction of container size, to be used while fetching remote data.

Parameters:

shuffleBufferFraction - fraction of container size

Returns:

instance of the current builder

setMaxSingleMemorySegmentFraction

T setMaxSingleMemorySegmentFraction(float maxSingleSegmentFraction)

Sets a size limit on the maximum segment size to be shuffled to disk. This is a fraction of the shuffle buffer.

Parameters:

maxSingleSegmentFraction - fraction of memory determined by ShuffleBufferFraction

Returns:

instance of the current builder

setMergeFraction

T setMergeFraction(float mergeFraction)

Configure the point at which in memory segments will be merged and written out to a single large disk segment. This is specified as a fraction of the shuffle buffer.

Has no affect at the moment.

Parameters:

mergeFraction - fraction of memory determined by ShuffleBufferFraction, which when filled, will trigger a merge

Returns:

instance of the current builder

setAdditionalConfiguration

T setAdditionalConfiguration(String key,
                           String value)

Used to set additional configuration parameters which are not set via API methods. This is primarily meant for rarely used configuration options such as IFile read-ahead, configuring the number of parallel files to merge etc.

Parameters:

key - the key to set

value - the corresponding value

Returns:

this object for further chained method calls

setAdditionalConfiguration

T setAdditionalConfiguration(Map<String,String> confMap)

Used to set additional configuration parameters which are not set via API methods. This is primarily meant for rarely used configuration options such as IFile read-ahead, configuring the number of parallel files to merge etc.

Additionally keys set via this method are made available to the combiner.

Parameters:

confMap - map of configuration key-value pairs

Returns:

this object for further chained method calls

setFromConfiguration

@InterfaceAudience.LimitedPrivate(value="hive, pig")
T setFromConfiguration(org.apache.hadoop.conf.Configuration conf)

Used to build out a configuration from an existing Hadoop Configuration. This is a private API is present only for compatibility and ease of use for existing systems which rely heavily on Configuration.

Parameters:

conf -

Returns:

this object for further chained method calls