org.apache.tez.runtime.library.conf

Class OrderedPartitionedKVOutputConfig.Builder

java.lang.Object

org.apache.tez.runtime.library.conf.OrderedPartitionedKVOutputConfig.Builder

All Implemented Interfaces:

org.apache.tez.runtime.library.conf.OrderedPartitionedKVOutputConfig.SpecificConfigBuilder<OrderedPartitionedKVOutputConfig.Builder>

Enclosing class:

OrderedPartitionedKVOutputConfig

@InterfaceAudience.Public
 @InterfaceStability.Evolving
public static class OrderedPartitionedKVOutputConfig.Builder
extends Object
implements org.apache.tez.runtime.library.conf.OrderedPartitionedKVOutputConfig.SpecificConfigBuilder<OrderedPartitionedKVOutputConfig.Builder>

Method Summary

Modifier and Type	Method and Description
OrderedPartitionedKVOutputConfig	build() Create the actual configuration instance.
OrderedPartitionedKVOutputConfig.Builder	setAdditionalConfiguration(Map<String,String> confMap) Used to set additional configuration parameters which are not set via API methods.
OrderedPartitionedKVOutputConfig.Builder	setAdditionalConfiguration(String key, String value) Used to set additional configuration parameters which are not set via API methods.
OrderedPartitionedKVOutputConfig.Builder	setCombiner(String combinerClassName) Configure the combiner class
OrderedPartitionedKVOutputConfig.Builder	setCombiner(String combinerClassName, Map<String,String> combinerConf) Configure the combiner class and it's associated configuration (specified as key-value pairs).
OrderedPartitionedKVOutputConfig.Builder	setCompression(boolean enabled, String compressionCodec, Map<String,String> codecConf)
OrderedPartitionedKVOutputConfig.Builder	setFromConfiguration(org.apache.hadoop.conf.Configuration conf)
OrderedPartitionedKVOutputConfig.Builder	setFromConfigurationUnfiltered(org.apache.hadoop.conf.Configuration conf)
OrderedPartitionedKVOutputConfig.Builder	setKeyComparatorClass(String comparatorClassName) Set the key comparator class
OrderedPartitionedKVOutputConfig.Builder	setKeyComparatorClass(String comparatorClassName, Map<String,String> comparatorConf) Set the key comparator class and it's associated configuration.
OrderedPartitionedKVOutputConfig.Builder	setKeySerializationClass(String serializationClassName, String comparatorClassName, Map<String,String> serializerConf) Set serialization class and the relevant comparator to be used for sorting.
OrderedPartitionedKVOutputConfig.Builder	setSortBufferSize(int sortBufferSize) Set the buffer size to use when sort the output
OrderedPartitionedKVOutputConfig.Builder	setSorter(org.apache.tez.runtime.library.conf.OrderedPartitionedKVOutputConfig.SorterImpl sorterImpl) Configure which sorter implementation to be used
OrderedPartitionedKVOutputConfig.Builder	setSorterNumThreads(int numThreads) Configure the number of threads to be used by the sorter
OrderedPartitionedKVOutputConfig.Builder	setValueSerializationClass(String serializationClassName, Map<String,String> serializerConf) Set serialization class responsible for providing serializer/deserializer for values.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

setSortBufferSize

public OrderedPartitionedKVOutputConfig.Builder setSortBufferSize(int sortBufferSize)

Description copied from interface: org.apache.tez.runtime.library.conf.OrderedPartitionedKVOutputConfig.SpecificConfigBuilder

Set the buffer size to use when sort the output

Specified by:

setSortBufferSize in interface org.apache.tez.runtime.library.conf.OrderedPartitionedKVOutputConfig.SpecificConfigBuilder<OrderedPartitionedKVOutputConfig.Builder>

Parameters:

sortBufferSize - the size of the buffer in MB

Returns:

instance of the current builder

setCombiner

public OrderedPartitionedKVOutputConfig.Builder setCombiner(String combinerClassName)

Description copied from interface: org.apache.tez.runtime.library.conf.OrderedPartitionedKVOutputConfig.SpecificConfigBuilder

Configure the combiner class

Specified by:

setCombiner in interface org.apache.tez.runtime.library.conf.OrderedPartitionedKVOutputConfig.SpecificConfigBuilder<OrderedPartitionedKVOutputConfig.Builder>

Parameters:

combinerClassName - the combiner class name

Returns:

instance of the current builder

setCombiner

public OrderedPartitionedKVOutputConfig.Builder setCombiner(String combinerClassName,
                                                            Map<String,String> combinerConf)

Description copied from interface: org.apache.tez.runtime.library.conf.OrderedPartitionedKVOutputConfig.SpecificConfigBuilder

Configure the combiner class and it's associated configuration (specified as key-value pairs). This method should only be used if the combiner requires some specific configuration. OrderedPartitionedKVOutputConfig.SpecificConfigBuilder.setCombiner(String) is the preferred method for setting a combiner.

Specified by:

setCombiner in interface org.apache.tez.runtime.library.conf.OrderedPartitionedKVOutputConfig.SpecificConfigBuilder<OrderedPartitionedKVOutputConfig.Builder>

Parameters:

combinerClassName - the combiner class name

combinerConf - the combiner configuration. This can be null, and otherwise is a Map of key-value pairs. The keys should be limited to the ones required by the combiner.

Returns:

instance of the current builder

setSorterNumThreads

public OrderedPartitionedKVOutputConfig.Builder setSorterNumThreads(int numThreads)

Description copied from interface: org.apache.tez.runtime.library.conf.OrderedPartitionedKVOutputConfig.SpecificConfigBuilder

Configure the number of threads to be used by the sorter

Specified by:

setSorterNumThreads in interface org.apache.tez.runtime.library.conf.OrderedPartitionedKVOutputConfig.SpecificConfigBuilder<OrderedPartitionedKVOutputConfig.Builder>

Parameters:

numThreads - the number of threads

Returns:

instance of the current builder

setSorter

public OrderedPartitionedKVOutputConfig.Builder setSorter(org.apache.tez.runtime.library.conf.OrderedPartitionedKVOutputConfig.SorterImpl sorterImpl)

Description copied from interface: org.apache.tez.runtime.library.conf.OrderedPartitionedKVOutputConfig.SpecificConfigBuilder

Configure which sorter implementation to be used

Specified by:

setSorter in interface org.apache.tez.runtime.library.conf.OrderedPartitionedKVOutputConfig.SpecificConfigBuilder<OrderedPartitionedKVOutputConfig.Builder>

Parameters:

sorterImpl - Use an in-built sorter implementations.

Returns:

instance of the current builder

setAdditionalConfiguration

public OrderedPartitionedKVOutputConfig.Builder setAdditionalConfiguration(String key,
                                                                           String value)

Used to set additional configuration parameters which are not set via API methods. This is primarily meant for rarely used configuration options such as IFile read-ahead, configuring the number of parallel files to merge etc.

Parameters:

key - the key to set

value - the corresponding value

Returns:

this object for further chained method calls

setAdditionalConfiguration

public OrderedPartitionedKVOutputConfig.Builder setAdditionalConfiguration(Map<String,String> confMap)

Used to set additional configuration parameters which are not set via API methods. This is primarily meant for rarely used configuration options such as IFile read-ahead, configuring the number of parallel files to merge etc.

Additionally keys set via this method are made available to the combiner.

Parameters:

confMap - map of configuration key-value pairs

Returns:

this object for further chained method calls

setFromConfiguration

public OrderedPartitionedKVOutputConfig.Builder setFromConfiguration(org.apache.hadoop.conf.Configuration conf)

setFromConfigurationUnfiltered

public OrderedPartitionedKVOutputConfig.Builder setFromConfigurationUnfiltered(org.apache.hadoop.conf.Configuration conf)

setKeyComparatorClass

public OrderedPartitionedKVOutputConfig.Builder setKeyComparatorClass(String comparatorClassName)

Set the key comparator class

Parameters:

comparatorClassName - the key comparator class name

Returns:

instance of the current builder

setKeyComparatorClass

public OrderedPartitionedKVOutputConfig.Builder setKeyComparatorClass(String comparatorClassName,
                                                                      @Nullable
                                                                      Map<String,String> comparatorConf)

Set the key comparator class and it's associated configuration. This method should only be used if the comparator requires some specific configuration, which is typically not the case. setKeyComparatorClass(String) is the preferred method for setting a comparator.

Parameters:

comparatorClassName - the key comparator class name

comparatorConf - the comparator configuration. This can be null, and is a Map of key-value pairs. The keys should be limited to the ones required by the comparator.

Returns:

instance of the current builder

Throws:

NullPointerException - if comparatorClassName is null

setCompression

public OrderedPartitionedKVOutputConfig.Builder setCompression(boolean enabled,
                                                               @Nullable
                                                               String compressionCodec,
                                                               @Nullable
                                                               Map<String,String> codecConf)

setKeySerializationClass

public OrderedPartitionedKVOutputConfig.Builder setKeySerializationClass(String serializationClassName,
                                                                         String comparatorClassName,
                                                                         @Nullable
                                                                         Map<String,String> serializerConf)

Set serialization class and the relevant comparator to be used for sorting. Providing custom serialization class could change the way, keys needs to be compared in sorting. Providing invalid comparator here could create invalid results.

Parameters:

serializationClassName -

comparatorClassName -

serializerConf - the serializer configuration. This can be null, and is a Map of key-value pairs. The keys should be limited to the ones required by the comparator.

Returns:

this object for further chained method calls

Throws:

NullPointerException - if serializationClassName or comparatorClassName is null

setValueSerializationClass

public OrderedPartitionedKVOutputConfig.Builder setValueSerializationClass(String serializationClassName,
                                                                           @Nullable
                                                                           Map<String,String> serializerConf)

Set serialization class responsible for providing serializer/deserializer for values.

Parameters:

serializationClassName -

serializerConf - the serializer configuration. This can be null, and is a Map of key-value pairs. The keys should be limited to the ones required by the comparator.

Returns:

this object for further chained method calls

Throws:

NullPointerException - if serializationClassName is null

build

public OrderedPartitionedKVOutputConfig build()

Create the actual configuration instance.

Returns:

an instance of the Configuration