org.apache.tez.runtime.library.conf

Class UnorderedPartitionedKVEdgeConfig

java.lang.Object

org.apache.tez.runtime.library.conf.UnorderedPartitionedKVEdgeConfig

@InterfaceAudience.Public
@InterfaceStability.Evolving
public class UnorderedPartitionedKVEdgeConfig
extends Object

Configure payloads for the UnorderedPartitionedKVOutput and UnorderedKVInput pair

Values will be picked up from tez-site if not specified, otherwise defaults from TezRuntimeConfiguration will be used.

Property Summary

Properties

Type	Property and Description
org.apache.tez.dag.api.EdgeProperty	createDefaultEdge This is a convenience method for the typical usage of this edge, and creates an instance of EdgeProperty which is likely to be used.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	UnorderedPartitionedKVEdgeConfig.Builder

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.tez.dag.api.EdgeProperty	createDefaultCustomEdgeProperty(org.apache.tez.dag.api.EdgeManagerPluginDescriptor edgeManagerDescriptor) This is a convenience method for creating an Edge descriptor based on the specified EdgeManagerDescriptor.
org.apache.tez.dag.api.EdgeProperty	createDefaultEdgeProperty() This is a convenience method for the typical usage of this edge, and creates an instance of EdgeProperty which is likely to be used.
String	getInputClassName() Get the input class name
String	getInputHistoryText() Get the history text for the configured Input
org.apache.tez.dag.api.UserPayload	getInputPayload() Get the payload for the configured Input
String	getOutputClassName() Get the output class name
String	getOutputHistoryText() Get the history text for the configured Output
org.apache.tez.dag.api.UserPayload	getOutputPayload() Get the payload for the configured Output
static UnorderedPartitionedKVEdgeConfig.Builder	newBuilder(String keyClassName, String valueClassName, String partitionerClassName) Create a builder to configure the relevant Input and Output
static UnorderedPartitionedKVEdgeConfig.Builder	newBuilder(String keyClassName, String valueClassName, String partitionerClassName, Map<String,String> partitionerConf) Create a builder to configure the relevant Input and Output.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Property Detail

createDefaultEdge

This is a convenience method for the typical usage of this edge, and creates an instance of EdgeProperty which is likely to be used.

If custom edge properties are required, the methods to get the relevant payloads should be used.

* In this case - DataMovementType.SCATTER_GATHER, EdgeProperty.DataSourceType.PERSISTED, EdgeProperty.SchedulingType.SEQUENTIAL

Returns:

an EdgeProperty instance

Method Detail

newBuilder

public static UnorderedPartitionedKVEdgeConfig.Builder newBuilder(String keyClassName,
                                                  String valueClassName,
                                                  String partitionerClassName,
                                                  @Nullable
                                                  Map<String,String> partitionerConf)

Create a builder to configure the relevant Input and Output.

This method should only be used when using a custom Partitioner which requires specific Configuration. newBuilder(String, String, String) is the preferred method to crate an instance of the Builder

Parameters:

keyClassName - the key class name

valueClassName - the value class name

partitionerClassName - the partitioner class name

partitionerConf - the partitioner configuration. This can be null, and is a Map of key-value pairs. The keys should be limited to the ones required by the partitioner.

Returns:

a builder to configure the edge

newBuilder

public static UnorderedPartitionedKVEdgeConfig.Builder newBuilder(String keyClassName,
                                                  String valueClassName,
                                                  String partitionerClassName)

Create a builder to configure the relevant Input and Output

Parameters:

keyClassName - the key class name

valueClassName - the value class name

partitionerClassName - the partitioner class name

Returns:

a builder to configure the edge

getOutputPayload

public org.apache.tez.dag.api.UserPayload getOutputPayload()

Get the payload for the configured Output

Returns:

output configuration as a byte array

getOutputClassName

public String getOutputClassName()

Get the output class name

Returns:

the output class name

getInputPayload

public org.apache.tez.dag.api.UserPayload getInputPayload()

Get the payload for the configured Input

Returns:

input configuration as a byte array

getOutputHistoryText

public String getOutputHistoryText()

Get the history text for the configured Output

Returns:

output configuration as a string in json format

getInputHistoryText

public String getInputHistoryText()

Get the history text for the configured Input

Returns:

input configuration as a string in json format

getInputClassName

public String getInputClassName()

Get the input class name

Returns:

the input class name

createDefaultEdgeProperty

public org.apache.tez.dag.api.EdgeProperty createDefaultEdgeProperty()

This is a convenience method for the typical usage of this edge, and creates an instance of EdgeProperty which is likely to be used.

If custom edge properties are required, the methods to get the relevant payloads should be used.

* In this case - DataMovementType.SCATTER_GATHER, EdgeProperty.DataSourceType.PERSISTED, EdgeProperty.SchedulingType.SEQUENTIAL

Returns:

an EdgeProperty instance

createDefaultCustomEdgeProperty

public org.apache.tez.dag.api.EdgeProperty createDefaultCustomEdgeProperty(org.apache.tez.dag.api.EdgeManagerPluginDescriptor edgeManagerDescriptor)

This is a convenience method for creating an Edge descriptor based on the specified EdgeManagerDescriptor.

Parameters:

edgeManagerDescriptor - the custom edge specification

Returns:

an EdgeProperty instance