org.apache.tez.dag.api

Interface VertexManagerPluginContext

@InterfaceStability.Unstable
@InterfaceAudience.Public
public interface VertexManagerPluginContext

Object with API's to interact with the Tez execution engine

Method Summary

Methods

Modifier and Type	Method and Description
void	addRootInputEvents(String inputName, Collection<InputDataInformationEvent> events) Allows a VertexManagerPlugin to assign Events for Root Inputs For regular Event Routing changes - the EdgeManager should be configured via the setVertexParallelism method
void	doneReconfiguringVertex() Optional API.
int	getDAGAttemptNumber()
Map<String,EdgeProperty>	getInputVertexEdgeProperties() Get the edge properties on the input edges of this vertex.
int	getNumClusterNodes() Get the number of nodes in the cluster
org.apache.hadoop.yarn.api.records.Container	getTaskContainer(String vertexName, Integer taskIndex) Get the container for the successful attempt of the task
org.apache.hadoop.yarn.api.records.Resource	getTotalAvailableResource() Get the total resource allocated to this vertex.
UserPayload	getUserPayload() Get the payload set for the plugin
Set<String>	getVertexInputNames() Get the names of the non-vertex inputs of this vertex.
String	getVertexName() Get the name of the vertex
int	getVertexNumTasks(String vertexName) Get the number of tasks in the given vertex
org.apache.hadoop.yarn.api.records.Resource	getVertexTaskResource() Get the resource allocated to a task of this vertex
void	registerForVertexStateUpdates(String vertexName, Set<VertexState> stateSet) Register to get notifications on updates to the specified vertex.
void	scheduleVertexTasks(List<org.apache.tez.dag.api.VertexManagerPluginContext.TaskWithLocationHint> tasks) Notify the vertex to start the given tasks
void	setVertexLocationHint(VertexLocationHint locationHint) Set the placement hint for tasks in this vertex
void	setVertexParallelism(int parallelism, VertexLocationHint locationHint, Map<String,EdgeManagerPluginDescriptor> sourceEdgeManagers, Map<String,InputSpecUpdate> rootInputSpecUpdate) Set the new parallelism (number of tasks) of this vertex, Map of source (input) vertices and edge managers to change the event routing between the source tasks and the new destination tasks and the number of physical inputs for root inputs.
void	vertexReconfigurationPlanned() Optional API.

Method Detail

getInputVertexEdgeProperties

Map<String,EdgeProperty> getInputVertexEdgeProperties()

Get the edge properties on the input edges of this vertex. The input edge is represented by the source vertex name

Returns:

Map of source vertex name and edge property

getVertexName

String getVertexName()

Get the name of the vertex

Returns:

Vertex name

getUserPayload

UserPayload getUserPayload()

Get the payload set for the plugin

Returns:

user payload

getVertexNumTasks

int getVertexNumTasks(String vertexName)

Get the number of tasks in the given vertex

Parameters:

vertexName -

Returns:

Total number of tasks in this vertex

getVertexTaskResource

org.apache.hadoop.yarn.api.records.Resource getVertexTaskResource()

Get the resource allocated to a task of this vertex

Returns:

Resource

getTaskContainer

org.apache.hadoop.yarn.api.records.Container getTaskContainer(String vertexName,
                                                            Integer taskIndex)

Get the container for the successful attempt of the task

Returns:

YARN container for the successful task. Maybe null if there is no successful task.

getTotalAvailableResource

org.apache.hadoop.yarn.api.records.Resource getTotalAvailableResource()

Get the total resource allocated to this vertex. If the DAG is running in a busy cluster then it may have no resources available dedicated to it. The DAG may divide its available resource among member vertices.

Returns:

Resource

getNumClusterNodes

int getNumClusterNodes()

Get the number of nodes in the cluster

Returns:

Number of nodes

setVertexParallelism

void setVertexParallelism(int parallelism,
                        @Nullable
                        VertexLocationHint locationHint,
                        @Nullable
                        Map<String,EdgeManagerPluginDescriptor> sourceEdgeManagers,
                        @Nullable
                        Map<String,InputSpecUpdate> rootInputSpecUpdate)

Set the new parallelism (number of tasks) of this vertex, Map of source (input) vertices and edge managers to change the event routing between the source tasks and the new destination tasks and the number of physical inputs for root inputs. This API can change the parallelism only once. Subsequent attempts will be disallowed

Parameters:

parallelism - New number of tasks in the vertex

locationHint - the placement policy for tasks.

sourceEdgeManagers - Edge Managers to be updated

rootInputSpecUpdate - Updated Root Input specifications, if any. If none specified, a default of 1 physical input is used

addRootInputEvents

void addRootInputEvents(String inputName,
                      Collection<InputDataInformationEvent> events)

Allows a VertexManagerPlugin to assign Events for Root Inputs For regular Event Routing changes - the EdgeManager should be configured via the setVertexParallelism method

Parameters:

inputName - The input name associated with the event

events - The list of Events to be assigned to various tasks belonging to the Vertex. The target index on individual events represents the task to which events need to be sent.

scheduleVertexTasks

void scheduleVertexTasks(List<org.apache.tez.dag.api.VertexManagerPluginContext.TaskWithLocationHint> tasks)

Notify the vertex to start the given tasks

Parameters:

tasks - Indices of the tasks to be started

getVertexInputNames

@Nullable
Set<String> getVertexInputNames()

Get the names of the non-vertex inputs of this vertex. These are primary sources of data.

Returns:

Names of inputs to this vertex. Maybe null if there are no inputs

setVertexLocationHint

void setVertexLocationHint(VertexLocationHint locationHint)

Set the placement hint for tasks in this vertex

Parameters:

locationHint -

getDAGAttemptNumber

int getDAGAttemptNumber()

Returns:

DAG Attempt number

registerForVertexStateUpdates

void registerForVertexStateUpdates(String vertexName,
                                 @Nullable
                                 Set<VertexState> stateSet)

Register to get notifications on updates to the specified vertex. Notifications will be sent via VertexManagerPlugin.onVertexStateUpdated(org.apache.tez.dag.api.event.VertexStateUpdate) This method can only be invoked once. Duplicate invocations will result in an error.

Parameters:

vertexName - the vertex name for which notifications are required.

stateSet - the set of states for which notifications are required. null implies all

vertexReconfigurationPlanned

void vertexReconfigurationPlanned()

Optional API. No need to call this when the vertex is not fully defined to start with. E.g. vertex parallelism is not defined, or edges are not configured. In that case, Tez will assume that the vertex needs reconfiguration. If the vertex is already fully defined, but the VertexManagerPlugin wants to reconfigure the vertex, then it must use this API to inform Tez about its intention. Without invoking this method, it is invalid to re-configure the vertex, e.g. via the setVertexParallelism(int, VertexLocationHint, Map, Map) method if the vertex is already fully defined. This can be invoked at any time until VertexManagerPlugin.initialize() has completed. Its invalid to invoke this method after VertexManagerPlugin.initialize() has completed
If this API is invoked, then doneReconfiguringVertex() must be invoked after the VertexManagerPlugin is done reconfiguring the vertex, . Actions like scheduling tasks or sending events do not count as reconfiguration.

doneReconfiguringVertex

void doneReconfiguringVertex()

Optional API. This needs to be called only if vertexReconfigurationPlanned() has been invoked. This must be called after vertexReconfigurationPlanned() is called.