org.apache.tez.client

Class TezClientUtils

java.lang.Object

org.apache.tez.client.TezClientUtils

@InterfaceAudience.Private
public class TezClientUtils
extends Object

Constructor Summary

Constructors

Constructor and Description
TezClientUtils()

Method Summary

Methods

Modifier and Type	Method and Description
static String	addDefaultsToTaskLaunchCmdOpts(String vOpts, org.apache.hadoop.conf.Configuration conf)
static String	addDefaultsToTaskLaunchCmdOpts(String vOpts, org.apache.hadoop.conf.Configuration conf, JavaOptsChecker javaOptsChecker)
static void	addFileSystemCredentialsFromURIs(Collection<URI> uris, org.apache.hadoop.security.Credentials credentials, org.apache.hadoop.conf.Configuration conf) Populate Credentials for the URI's to access them from their FileSystems
static void	addLog4jSystemProperties(String logLevel, List<String> vargs)
static void	addLogParamsToEnv(Map<String,String> environment, String[] logParams)
static org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext	createApplicationSubmissionContext(org.apache.hadoop.yarn.api.records.ApplicationId appId, DAG dag, String amName, org.apache.tez.client.AMConfiguration amConfig, Map<String,org.apache.hadoop.yarn.api.records.LocalResource> tezJarResources, org.apache.hadoop.security.Credentials sessionCreds, boolean tezLrsAsArchive, TezApiVersionInfo apiVersionInfo, HistoryACLPolicyManager historyACLPolicyManager, ServicePluginsDescriptor servicePluginsDescriptor, JavaOptsChecker javaOptsChecker) Create an ApplicationSubmissionContext to launch a Tez AM
static org.apache.hadoop.fs.FileSystem	ensureStagingDirExists(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path stagingArea) Verify or create the Staging area directory on the configured Filesystem
static DAGClientAMProtocolBlockingPB	getAMProxy(org.apache.hadoop.conf.Configuration conf, String amHost, int amRpcPort, org.apache.hadoop.yarn.api.records.Token clientToAMToken)
static String	maybeAddDefaultLoggingJavaOpts(String logLevel, String javaOpts)
static String	maybeAddDefaultMemoryJavaOpts(String javaOpts, org.apache.hadoop.yarn.api.records.Resource resource, double maxHeapFactor)
static String[]	parseLogParams(String logConfig)
static void	setApplicationPriority(org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext context, org.apache.tez.client.AMConfiguration amConfig)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TezClientUtils

public TezClientUtils()

Method Detail

ensureStagingDirExists

public static org.apache.hadoop.fs.FileSystem ensureStagingDirExists(org.apache.hadoop.conf.Configuration conf,
                                                     org.apache.hadoop.fs.Path stagingArea)
                                                              throws IOException

Verify or create the Staging area directory on the configured Filesystem

Parameters:

stagingArea - Staging area directory path

Returns:

the FileSytem for the staging area directory

Throws:

IOException

addFileSystemCredentialsFromURIs

public static void addFileSystemCredentialsFromURIs(Collection<URI> uris,
                                    org.apache.hadoop.security.Credentials credentials,
                                    org.apache.hadoop.conf.Configuration conf)
                                             throws IOException

Populate Credentials for the URI's to access them from their FileSystems

Parameters:

uris - URIs that need to be accessed

credentials - Credentials object into which to add the credentials

conf - Configuration to access the FileSystem

Throws:

IOException

createApplicationSubmissionContext

@InterfaceAudience.Private
public static org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext createApplicationSubmissionContext(org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                                                                                                           DAG dag,
                                                                                                                           String amName,
                                                                                                                           org.apache.tez.client.AMConfiguration amConfig,
                                                                                                                           Map<String,org.apache.hadoop.yarn.api.records.LocalResource> tezJarResources,
                                                                                                                           org.apache.hadoop.security.Credentials sessionCreds,
                                                                                                                           boolean tezLrsAsArchive,
                                                                                                                           TezApiVersionInfo apiVersionInfo,
                                                                                                                           HistoryACLPolicyManager historyACLPolicyManager,
                                                                                                                           ServicePluginsDescriptor servicePluginsDescriptor,
                                                                                                                           JavaOptsChecker javaOptsChecker)
                                                                                                          throws IOException,
                                                                                                                 org.apache.hadoop.yarn.exceptions.YarnException

Create an ApplicationSubmissionContext to launch a Tez AM

Parameters:

appId - Application Id

dag - DAG to be submitted

amName - Name for the application

amConfig - AM Configuration

tezJarResources - Resources to be used by the AM

sessionCreds - the credential object which will be populated with session specific

historyACLPolicyManager -

servicePluginsDescriptor - descriptor for services which may be running in the AM

Returns:

an ApplicationSubmissionContext to launch a Tez AM

Throws:

IOException

org.apache.hadoop.yarn.exceptions.YarnException

maybeAddDefaultLoggingJavaOpts

@InterfaceAudience.Private
public static String maybeAddDefaultLoggingJavaOpts(String logLevel,
                                                              String javaOpts)

addDefaultsToTaskLaunchCmdOpts

@InterfaceAudience.Private
public static String addDefaultsToTaskLaunchCmdOpts(String vOpts,
                                                              org.apache.hadoop.conf.Configuration conf)
                                             throws TezException

Throws:

TezException

addDefaultsToTaskLaunchCmdOpts

@InterfaceAudience.Private
public static String addDefaultsToTaskLaunchCmdOpts(String vOpts,
                                                              org.apache.hadoop.conf.Configuration conf,
                                                              JavaOptsChecker javaOptsChecker)
                                             throws TezException

Throws:

TezException

addLog4jSystemProperties

@InterfaceAudience.Private
public static void addLog4jSystemProperties(String logLevel,
                                                      List<String> vargs)

getAMProxy

@InterfaceAudience.Private
public static DAGClientAMProtocolBlockingPB getAMProxy(org.apache.hadoop.conf.Configuration conf,
                                                                 String amHost,
                                                                 int amRpcPort,
                                                                 org.apache.hadoop.yarn.api.records.Token clientToAMToken)
                                                throws IOException

Throws:

IOException

maybeAddDefaultMemoryJavaOpts

@InterfaceAudience.Private
public static String maybeAddDefaultMemoryJavaOpts(String javaOpts,
                                                             org.apache.hadoop.yarn.api.records.Resource resource,
                                                             double maxHeapFactor)

addLogParamsToEnv

@InterfaceAudience.Private
public static void addLogParamsToEnv(Map<String,String> environment,
                                               String[] logParams)

parseLogParams

@InterfaceAudience.Private
public static String[] parseLogParams(String logConfig)

setApplicationPriority

public static void setApplicationPriority(org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext context,
                          org.apache.tez.client.AMConfiguration amConfig)