org.apache.tez.common

Class TezUtils

java.lang.Object

org.apache.tez.common.TezUtils

@InterfaceAudience.Public
public final class TezUtils
extends Object

Utility methods for setting up a DAG. Has helpers for setting up log4j configuration, converting Configuration to UserPayload etc.

Method Summary

Modifier and Type	Method and Description
static void	addLog4jSystemProperties(String logLevel, List<String> vargs) Allows changing the log level for task / AM logging.
static void	addToConfFromByteString(org.apache.hadoop.conf.Configuration configuration, com.google.protobuf.ByteString byteString)
static String	convertToHistoryText(org.apache.hadoop.conf.Configuration conf)
static String	convertToHistoryText(String description, org.apache.hadoop.conf.Configuration conf)
static com.google.protobuf.ByteString	createByteStringFromConf(org.apache.hadoop.conf.Configuration conf) Convert a Configuration to compressed ByteString using Protocol buffer
static org.apache.hadoop.conf.Configuration	createConfFromBaseConfAndPayload(TaskContext context)
static org.apache.hadoop.conf.Configuration	createConfFromByteString(com.google.protobuf.ByteString byteString) Convert a byte string to a Configuration object
static org.apache.hadoop.conf.Configuration	createConfFromUserPayload(UserPayload payload) Convert an instance of UserPayload to Configuration
static UserPayload	createUserPayloadFromConf(org.apache.hadoop.conf.Configuration conf) Convert a Configuration to a UserPayload
static void	populateConfProtoFromEntries(Iterable<Map.Entry<String,String>> params, org.apache.tez.dag.api.records.DAGProtos.ConfigurationProto.Builder confBuilder)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

addLog4jSystemProperties

public static void addLog4jSystemProperties(String logLevel,
                                            List<String> vargs)

Allows changing the log level for task / AM logging.

Adds the JVM system properties necessary to configure ContainerLogAppender.

Parameters:

logLevel - the desired log level (eg INFO/WARN/DEBUG)

vargs - the argument list to append to

createByteStringFromConf

public static com.google.protobuf.ByteString createByteStringFromConf(org.apache.hadoop.conf.Configuration conf)
                                                               throws IOException

Convert a Configuration to compressed ByteString using Protocol buffer

Parameters:

conf - : Configuration to be converted

Returns:

PB ByteString (compressed)

Throws:

IOException

createUserPayloadFromConf

public static UserPayload createUserPayloadFromConf(org.apache.hadoop.conf.Configuration conf)
                                             throws IOException

Convert a Configuration to a UserPayload

Parameters:

conf - configuration to be converted

Returns:

an instance of UserPayload

Throws:

IOException

createConfFromByteString

public static org.apache.hadoop.conf.Configuration createConfFromByteString(com.google.protobuf.ByteString byteString)
                                                                     throws IOException

Convert a byte string to a Configuration object

Parameters:

byteString - byteString representation of the conf created using createByteStringFromConf(org.apache.hadoop.conf.Configuration)

Returns:

Configuration

Throws:

IOException

createConfFromBaseConfAndPayload

public static org.apache.hadoop.conf.Configuration createConfFromBaseConfAndPayload(TaskContext context)
                                                                             throws IOException

Throws:

IOException

addToConfFromByteString

public static void addToConfFromByteString(org.apache.hadoop.conf.Configuration configuration,
                                           com.google.protobuf.ByteString byteString)
                                    throws IOException

Throws:

IOException

createConfFromUserPayload

public static org.apache.hadoop.conf.Configuration createConfFromUserPayload(UserPayload payload)
                                                                      throws IOException

Convert an instance of UserPayload to Configuration

Parameters:

payload - UserPayload created using createUserPayloadFromConf(org.apache.hadoop.conf.Configuration)

Returns:

Configuration

Throws:

IOException

convertToHistoryText

public static String convertToHistoryText(String description,
                                          org.apache.hadoop.conf.Configuration conf)

convertToHistoryText

public static String convertToHistoryText(org.apache.hadoop.conf.Configuration conf)

populateConfProtoFromEntries

public static void populateConfProtoFromEntries(Iterable<Map.Entry<String,String>> params,
                                                org.apache.tez.dag.api.records.DAGProtos.ConfigurationProto.Builder confBuilder)