org.apache.hadoop.mapred.split

Class TezMapredSplitsGrouper

java.lang.Object

org.apache.tez.mapreduce.grouper.TezSplitGrouper

org.apache.hadoop.mapred.split.TezMapredSplitsGrouper

@InterfaceAudience.Public
@InterfaceStability.Evolving
public class TezMapredSplitsGrouper
extends TezSplitGrouper

A Helper that provides grouping logic to group InputSplits using various parameters. A TezGroupedSplit is used to wrap the real InputSplits in a group.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.tez.mapreduce.grouper.TezSplitGrouper

TezSplitGrouper.TezMRSplitsGrouperConfigBuilder

Field Summary

Fields inherited from class org.apache.tez.mapreduce.grouper.TezSplitGrouper

TEZ_GROUPING_NODE_LOCAL_ONLY, TEZ_GROUPING_NODE_LOCAL_ONLY_DEFAULT, TEZ_GROUPING_RACK_SPLIT_SIZE_REDUCTION, TEZ_GROUPING_RACK_SPLIT_SIZE_REDUCTION_DEFAULT, TEZ_GROUPING_REPEATABLE, TEZ_GROUPING_REPEATABLE_DEFAULT, TEZ_GROUPING_SPLIT_BY_COUNT, TEZ_GROUPING_SPLIT_BY_COUNT_DEFAULT, TEZ_GROUPING_SPLIT_BY_LENGTH, TEZ_GROUPING_SPLIT_BY_LENGTH_DEFAULT, TEZ_GROUPING_SPLIT_COUNT, TEZ_GROUPING_SPLIT_MAX_SIZE, TEZ_GROUPING_SPLIT_MAX_SIZE_DEFAULT, TEZ_GROUPING_SPLIT_MIN_SIZE, TEZ_GROUPING_SPLIT_MIN_SIZE_DEFAULT, TEZ_GROUPING_SPLIT_WAVES, TEZ_GROUPING_SPLIT_WAVES_DEFAULT

Constructor Summary

Constructors

Constructor and Description
TezMapredSplitsGrouper()

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.hadoop.mapred.InputSplit[]	getGroupedSplits(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.mapred.InputSplit[] originalSplits, int desiredNumSplits, String wrappedInputFormatName)
org.apache.hadoop.mapred.InputSplit[]	getGroupedSplits(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.mapred.InputSplit[] originalSplits, int desiredNumSplits, String wrappedInputFormatName, SplitSizeEstimator estimator)
org.apache.hadoop.mapred.InputSplit[]	getGroupedSplits(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.mapred.InputSplit[] originalSplits, int desiredNumSplits, String wrappedInputFormatName, SplitSizeEstimator estimator, SplitLocationProvider locationProvider)

Methods inherited from class org.apache.tez.mapreduce.grouper.TezSplitGrouper

getGroupedSplits, newConfigBuilder

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TezMapredSplitsGrouper

public TezMapredSplitsGrouper()

Method Detail

getGroupedSplits

public org.apache.hadoop.mapred.InputSplit[] getGroupedSplits(org.apache.hadoop.conf.Configuration conf,
                                                     org.apache.hadoop.mapred.InputSplit[] originalSplits,
                                                     int desiredNumSplits,
                                                     String wrappedInputFormatName)
                                                       throws IOException

Throws:

IOException

getGroupedSplits

public org.apache.hadoop.mapred.InputSplit[] getGroupedSplits(org.apache.hadoop.conf.Configuration conf,
                                                     org.apache.hadoop.mapred.InputSplit[] originalSplits,
                                                     int desiredNumSplits,
                                                     String wrappedInputFormatName,
                                                     SplitSizeEstimator estimator)
                                                       throws IOException

Throws:

IOException

getGroupedSplits

public org.apache.hadoop.mapred.InputSplit[] getGroupedSplits(org.apache.hadoop.conf.Configuration conf,
                                                     org.apache.hadoop.mapred.InputSplit[] originalSplits,
                                                     int desiredNumSplits,
                                                     String wrappedInputFormatName,
                                                     SplitSizeEstimator estimator,
                                                     SplitLocationProvider locationProvider)
                                                       throws IOException

Throws:

IOException