org.apache.tez.runtime.library.conf

Interface OrderedPartitionedKVOutputConfig.SpecificConfigBuilder<T>

All Known Implementing Classes:

OrderedPartitionedKVOutputConfig.Builder, OrderedPartitionedKVOutputConfig.SpecificBuilder

Enclosing class:

OrderedPartitionedKVOutputConfig

@InterfaceAudience.Private
public static interface OrderedPartitionedKVOutputConfig.SpecificConfigBuilder<T>

Configure parameters which are specific to the Output.

Method Summary

Methods

Modifier and Type	Method and Description
T	setAdditionalConfiguration(Map<String,String> confMap) Used to set additional configuration parameters which are not set via API methods.
T	setAdditionalConfiguration(String key, String value) Used to set additional configuration parameters which are not set via API methods.
T	setCombiner(String combinerClassName) Configure the combiner class
T	setCombiner(String combinerClassName, Map<String,String> combinerConf) Configure the combiner class and it's associated configuration (specified as key-value pairs).
T	setFromConfiguration(org.apache.hadoop.conf.Configuration conf) Used to build out a configuration from an existing Hadoop Configuration.
T	setSortBufferSize(int sortBufferSize) Set the buffer size to use when sort the output
T	setSorter(OrderedPartitionedKVOutputConfig.SorterImpl sorterImpl) Configure which sorter implementation to be used
T	setSorterNumThreads(int numThreads) Configure the number of threads to be used by the sorter

Method Detail

setSortBufferSize

T setSortBufferSize(int sortBufferSize)

Set the buffer size to use when sort the output

Parameters:

sortBufferSize - the size of the buffer in MB

Returns:

instance of the current builder

setCombiner

T setCombiner(String combinerClassName)

Configure the combiner class

Parameters:

combinerClassName - the combiner class name

Returns:

instance of the current builder

setCombiner

T setCombiner(String combinerClassName,
            @Nullable
            Map<String,String> combinerConf)

Configure the combiner class and it's associated configuration (specified as key-value pairs). This method should only be used if the combiner requires some specific configuration. setCombiner(String) is the preferred method for setting a combiner.

Parameters:

combinerClassName - the combiner class name

combinerConf - the combiner configuration. This can be null, and otherwise is a Map of key-value pairs. The keys should be limited to the ones required by the combiner.

Returns:

instance of the current builder

setSorterNumThreads

T setSorterNumThreads(int numThreads)

Configure the number of threads to be used by the sorter

Parameters:

numThreads - the number of threads

Returns:

instance of the current builder

setSorter

T setSorter(OrderedPartitionedKVOutputConfig.SorterImpl sorterImpl)

Configure which sorter implementation to be used

Parameters:

sorterImpl - Use an in-built sorter implementations.

Returns:

instance of the current builder

setAdditionalConfiguration

T setAdditionalConfiguration(String key,
                           String value)

Used to set additional configuration parameters which are not set via API methods. This is primarily meant for rarely used configuration options such as IFile read-ahead, configuring the number of parallel files to merge etc.

Parameters:

key - the key to set

value - the corresponding value

Returns:

this object for further chained method calls

setAdditionalConfiguration

T setAdditionalConfiguration(Map<String,String> confMap)

Used to set additional configuration parameters which are not set via API methods. This is primarily meant for rarely used configuration options such as IFile read-ahead, configuring the number of parallel files to merge etc.

Additionally keys set via this method are made available to the combiner.

Parameters:

confMap - map of configuration key-value pairs

Returns:

this object for further chained method calls

setFromConfiguration

@InterfaceAudience.LimitedPrivate(value="hive, pig")
T setFromConfiguration(org.apache.hadoop.conf.Configuration conf)

Used to build out a configuration from an existing Hadoop Configuration. This is a private API is present only for compatibility and ease of use for existing systems which rely heavily on Configuration.

Parameters:

conf -

Returns:

this object for further chained method calls