org.apache.tez.runtime.library.common.shuffle.orderedgrouped

Class Shuffle

java.lang.Object

org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle

@InterfaceAudience.Private
@InterfaceStability.Unstable
public class Shuffle
extends Object

Usage: Create instance, setInitialMemoryAllocated(long), run()

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Shuffle.ShuffleError

Constructor Summary

Constructors

Constructor and Description
Shuffle(org.apache.tez.runtime.api.InputContext inputContext, org.apache.hadoop.conf.Configuration conf, int numInputs, long initialMemoryAvailable)

Method Summary

Methods

Modifier and Type	Method and Description
static long	getInitialMemoryRequirement(org.apache.hadoop.conf.Configuration conf, long maxAvailableTaskMemory)
void	handleEvents(List<org.apache.tez.runtime.api.Event> events)
boolean	isInputReady() Indicates whether the Shuffle and Merge processing is complete.
void	reportException(Throwable t)
void	run()
void	shutdown()
TezRawKeyValueIterator	waitForInput() Waits for the Shuffle and Merge to complete, and returns an iterator over the input.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Shuffle

public Shuffle(org.apache.tez.runtime.api.InputContext inputContext,
       org.apache.hadoop.conf.Configuration conf,
       int numInputs,
       long initialMemoryAvailable)
        throws IOException

Throws:

IOException

Method Detail

handleEvents

public void handleEvents(List<org.apache.tez.runtime.api.Event> events)
                  throws IOException

Throws:

IOException

isInputReady

public boolean isInputReady()
                     throws IOException,
                            InterruptedException,
                            org.apache.tez.dag.api.TezException

Indicates whether the Shuffle and Merge processing is complete.

Returns:

false if not complete, true if complete or if an error occurred.

Throws:

InterruptedException

IOException

InputAlreadyClosedException

org.apache.tez.dag.api.TezException

waitForInput

public TezRawKeyValueIterator waitForInput()
                                    throws IOException,
                                           InterruptedException,
                                           org.apache.tez.dag.api.TezException

Waits for the Shuffle and Merge to complete, and returns an iterator over the input.

Returns:

an iterator over the fetched input.

Throws:

IOException

InterruptedException

org.apache.tez.dag.api.TezException

run

public void run()
         throws IOException

Throws:

IOException

shutdown

public void shutdown()

reportException

@InterfaceAudience.Private
public void reportException(Throwable t)

getInitialMemoryRequirement

@InterfaceAudience.Private
public static long getInitialMemoryRequirement(org.apache.hadoop.conf.Configuration conf,
                                                         long maxAvailableTaskMemory)