TaskCounter (tez-api 0.8.3 API)

java.lang.Object
- java.lang.Enum<TaskCounter>
- - org.apache.tez.common.counters.TaskCounter

All Implemented Interfaces:: Serializable, Comparable<TaskCounter>

@InterfaceAudience.Private
public enum TaskCounter
extends Enum<TaskCounter>

Enum Constant Summary

Enum Constants
Enum Constant and Description
`ADDITIONAL_SPILL_COUNT` Spills that were generated & read by the same task (unnecessary spills due to lac of adequate memory).
`ADDITIONAL_SPILLS_BYTES_READ` Bytes read from disk due to previous spills (lac of adequate memory).
`ADDITIONAL_SPILLS_BYTES_WRITTEN` Bytes written to disk due to unnecessary spills (lac of adequate memory).
`COMBINE_INPUT_RECORDS`
`COMBINE_OUTPUT_RECORDS`
`COMMITTED_HEAP_BYTES`
`CPU_MILLISECONDS`
`FIRST_EVENT_RECEIVED` First event received from source relative to task start time.
`GC_TIME_MILLIS`
`INPUT_GROUPS`
`INPUT_RECORDS_PROCESSED` Represents the number of Input Records that were actually processed.
`LAST_EVENT_RECEIVED` Last event received from source relative to task start time.
`MERGE_PHASE_TIME` Time taken to merge data retrieved during shuffle.
`MERGED_MAP_OUTPUTS`
`NUM_DISK_TO_DISK_MERGES` Number of disk to disk merges performed during the sort-merge
`NUM_FAILED_SHUFFLE_INPUTS` Number of failed copy attempts (physical inputs)
`NUM_MEM_TO_DISK_MERGES` Number of Memory to Disk merges performed during sort-merge.
`NUM_SHUFFLED_INPUTS` Number of Inputs from which data is copied.
`NUM_SKIPPED_INPUTS` Number of Inputs from which data was not copied - typically due to an empty Input
`NUM_SPECULATIONS`
`OUTPUT_BYTES` Represents the serialized output size (uncompressed) of data being written.
`OUTPUT_BYTES_PHYSICAL` Represents the actual physical size of the Output generated.
`OUTPUT_BYTES_WITH_OVERHEAD` Represents serialized output size (uncompressed) along with any overhead added by the format being used.
`OUTPUT_LARGE_RECORDS` Represent the number of large records in the output - typically, records which are spilled directly
`OUTPUT_RECORDS` Represents the number of actual output records.
`PHYSICAL_MEMORY_BYTES`
`REDUCE_INPUT_GROUPS` Number of Input Groups seen by ShuffledMergedInput.
`REDUCE_INPUT_RECORDS` Number of records (across all Groups) seen by ShuffledMergedInput Alternately number of records seen by a ReduceProcessor
`REDUCE_OUTPUT_RECORDS`
`REDUCE_SKIPPED_GROUPS`
`REDUCE_SKIPPED_RECORDS`
`SHUFFLE_BYTES` Amount of physical data moved over the wire.
`SHUFFLE_BYTES_DECOMPRESSED` Uncompressed size of the data being processed by the relevant Shuffle.
`SHUFFLE_BYTES_DISK_DIRECT` Number of bytes which were read directly from local disk
`SHUFFLE_BYTES_TO_DISK` Number of bytes which were shuffled directly to disk
`SHUFFLE_BYTES_TO_MEM` Number of bytes which were shuffled directly to memory.
`SHUFFLE_CHUNK_COUNT` Number of spill files being offered via shuffle-handler.
`SHUFFLE_PHASE_TIME` Time taken to shuffle data.
`SKIPPED_RECORDS`
`SPILLED_RECORDS` Number of records written to disk in case of OnFileSortedOutput.
`SPLIT_RAW_BYTES`
`VIRTUAL_MEMORY_BYTES`

Method Summary

Methods
Modifier and Type	Method and Description
`static TaskCounter`	`valueOf(String name)` Returns the enum constant of this type with the specified name.
`static TaskCounter[]`	`values()` Returns an array containing the constants of this enum type, in the order they are declared.

Methods inherited from class java.lang.Enum
clone, compareTo, equals, finalize, getDeclaringClass, hashCode, name, ordinal, toString, valueOf

Methods inherited from class java.lang.Object
getClass, notify, notifyAll, wait, wait, wait

- Enum Constant Detail
  - NUM_SPECULATIONS
```
public static final TaskCounter NUM_SPECULATIONS
```
  - REDUCE_INPUT_GROUPS
```
public static final TaskCounter REDUCE_INPUT_GROUPS
```
    Number of Input Groups seen by ShuffledMergedInput. Alternately the number of Input Groups seen by a Reduce task.
  - REDUCE_INPUT_RECORDS
```
public static final TaskCounter REDUCE_INPUT_RECORDS
```
    Number of records (across all Groups) seen by ShuffledMergedInput Alternately number of records seen by a ReduceProcessor
  - REDUCE_OUTPUT_RECORDS
```
public static final TaskCounter REDUCE_OUTPUT_RECORDS
```
  - REDUCE_SKIPPED_GROUPS
```
public static final TaskCounter REDUCE_SKIPPED_GROUPS
```
  - REDUCE_SKIPPED_RECORDS
```
public static final TaskCounter REDUCE_SKIPPED_RECORDS
```
  - SPLIT_RAW_BYTES
```
public static final TaskCounter SPLIT_RAW_BYTES
```
  - COMBINE_INPUT_RECORDS
```
public static final TaskCounter COMBINE_INPUT_RECORDS
```
  - COMBINE_OUTPUT_RECORDS
```
public static final TaskCounter COMBINE_OUTPUT_RECORDS
```
  - SPILLED_RECORDS
```
public static final TaskCounter SPILLED_RECORDS
```
    Number of records written to disk in case of OnFileSortedOutput. Number of additional records writtent out to disk in case of ShuffledMergedInput; this represents the number of unnecessary spills to disk caused by lac of memory.
  - NUM_SHUFFLED_INPUTS
```
public static final TaskCounter NUM_SHUFFLED_INPUTS
```
    Number of Inputs from which data is copied. Represents physical Inputs.
  - NUM_SKIPPED_INPUTS
```
public static final TaskCounter NUM_SKIPPED_INPUTS
```
    Number of Inputs from which data was not copied - typically due to an empty Input
  - NUM_FAILED_SHUFFLE_INPUTS
```
public static final TaskCounter NUM_FAILED_SHUFFLE_INPUTS
```
    Number of failed copy attempts (physical inputs)
  - MERGED_MAP_OUTPUTS
```
public static final TaskCounter MERGED_MAP_OUTPUTS
```
  - GC_TIME_MILLIS
```
public static final TaskCounter GC_TIME_MILLIS
```
  - CPU_MILLISECONDS
```
public static final TaskCounter CPU_MILLISECONDS
```
  - PHYSICAL_MEMORY_BYTES
```
public static final TaskCounter PHYSICAL_MEMORY_BYTES
```
  - VIRTUAL_MEMORY_BYTES
```
public static final TaskCounter VIRTUAL_MEMORY_BYTES
```
  - COMMITTED_HEAP_BYTES
```
public static final TaskCounter COMMITTED_HEAP_BYTES
```
  - INPUT_RECORDS_PROCESSED
```
public static final TaskCounter INPUT_RECORDS_PROCESSED
```
    Represents the number of Input Records that were actually processed. Used by MRInput and ShuffledUnorderedKVInput
  - OUTPUT_RECORDS
```
public static final TaskCounter OUTPUT_RECORDS
```
    Represents the number of actual output records. Used by MROutput, OnFileSortedOutput, and OnFileUnorderedKVOutput
  - OUTPUT_LARGE_RECORDS
```
public static final TaskCounter OUTPUT_LARGE_RECORDS
```
    Represent the number of large records in the output - typically, records which are spilled directly
  - SKIPPED_RECORDS
```
public static final TaskCounter SKIPPED_RECORDS
```
  - OUTPUT_BYTES
```
public static final TaskCounter OUTPUT_BYTES
```
    Represents the serialized output size (uncompressed) of data being written.
  - OUTPUT_BYTES_WITH_OVERHEAD
```
public static final TaskCounter OUTPUT_BYTES_WITH_OVERHEAD
```
    Represents serialized output size (uncompressed) along with any overhead added by the format being used.
  - OUTPUT_BYTES_PHYSICAL
```
public static final TaskCounter OUTPUT_BYTES_PHYSICAL
```
    Represents the actual physical size of the Output generated. This factors in Compression if it is enabled. (Will include actual serialized output size + overhead)
  - ADDITIONAL_SPILLS_BYTES_WRITTEN
```
public static final TaskCounter ADDITIONAL_SPILLS_BYTES_WRITTEN
```
    Bytes written to disk due to unnecessary spills (lac of adequate memory). Used by OnFileSortedOutput and ShuffledMergedInput
  - ADDITIONAL_SPILLS_BYTES_READ
```
public static final TaskCounter ADDITIONAL_SPILLS_BYTES_READ
```
    Bytes read from disk due to previous spills (lac of adequate memory). Used by OnFileSortedOutput and ShuffledMergedInput
  - ADDITIONAL_SPILL_COUNT
```
public static final TaskCounter ADDITIONAL_SPILL_COUNT
```
    Spills that were generated & read by the same task (unnecessary spills due to lac of adequate memory). Used by OnFileSortedOutput
  - SHUFFLE_CHUNK_COUNT
```
public static final TaskCounter SHUFFLE_CHUNK_COUNT
```
    Number of spill files being offered via shuffle-handler. e.g Without pipelined shuffle, this would be 1. With pipelined shuffle, this could be many as final merge is avoided.
  - INPUT_GROUPS
```
public static final TaskCounter INPUT_GROUPS
```
  - SHUFFLE_BYTES
```
public static final TaskCounter SHUFFLE_BYTES
```
    Amount of physical data moved over the wire. Used by Shuffled*Input. Should be a combination of SHUFFLE_BYTES_TO_MEM and SHUFFLE_BYTES_TO_DISK
  - SHUFFLE_BYTES_DECOMPRESSED
```
public static final TaskCounter SHUFFLE_BYTES_DECOMPRESSED
```
    Uncompressed size of the data being processed by the relevant Shuffle. Includes serialization, file format etc overheads.
  - SHUFFLE_BYTES_TO_MEM
```
public static final TaskCounter SHUFFLE_BYTES_TO_MEM
```
    Number of bytes which were shuffled directly to memory.
  - SHUFFLE_BYTES_TO_DISK
```
public static final TaskCounter SHUFFLE_BYTES_TO_DISK
```
    Number of bytes which were shuffled directly to disk
  - SHUFFLE_BYTES_DISK_DIRECT
```
public static final TaskCounter SHUFFLE_BYTES_DISK_DIRECT
```
    Number of bytes which were read directly from local disk
  - NUM_MEM_TO_DISK_MERGES
```
public static final TaskCounter NUM_MEM_TO_DISK_MERGES
```
    Number of Memory to Disk merges performed during sort-merge. Used by ShuffledMergedInput
  - NUM_DISK_TO_DISK_MERGES
```
public static final TaskCounter NUM_DISK_TO_DISK_MERGES
```
    Number of disk to disk merges performed during the sort-merge
  - SHUFFLE_PHASE_TIME
```
public static final TaskCounter SHUFFLE_PHASE_TIME
```
    Time taken to shuffle data. This includes time taken to fetch the data & merging the data in parallel to fetching when needed. This also includes any waiting time related to event delays from source. Represented in milliseconds.
  - MERGE_PHASE_TIME
```
public static final TaskCounter MERGE_PHASE_TIME
```
    Time taken to merge data retrieved during shuffle. Relative to task start time and expressed in milliseconds.
  - FIRST_EVENT_RECEIVED
```
public static final TaskCounter FIRST_EVENT_RECEIVED
```
    First event received from source relative to task start time. Represented in milliseconds
  - LAST_EVENT_RECEIVED
```
public static final TaskCounter LAST_EVENT_RECEIVED
```
    Last event received from source relative to task start time. Represented in milliseconds
- Method Detail
  - values
```
public static TaskCounter[] values()
```
    Returns an array containing the constants of this enum type, in the order they are declared. This method may be used to iterate over the constants as follows:
```
for (TaskCounter c : TaskCounter.values())
    System.out.println(c);
```
    Returns:
    an array containing the constants of this enum type, in the order they are declared
  - valueOf
```
public static TaskCounter valueOf(String name)
```
    Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)
    
    Parameters:
    name - the name of the enum constant to be returned.
    
    Returns:
    the enum constant with the specified name
    
    Throws:
    
    IllegalArgumentException - if this enum type has no constant with the specified name
    
    NullPointerException - if the argument is null

Enum TaskCounter

Enum Constant Summary

Method Summary

Methods inherited from class java.lang.Enum

Methods inherited from class java.lang.Object

Enum Constant Detail

NUM_SPECULATIONS

REDUCE_INPUT_GROUPS

REDUCE_INPUT_RECORDS

REDUCE_OUTPUT_RECORDS

REDUCE_SKIPPED_GROUPS

REDUCE_SKIPPED_RECORDS

SPLIT_RAW_BYTES

COMBINE_INPUT_RECORDS

COMBINE_OUTPUT_RECORDS

SPILLED_RECORDS

NUM_SHUFFLED_INPUTS

NUM_SKIPPED_INPUTS

NUM_FAILED_SHUFFLE_INPUTS

MERGED_MAP_OUTPUTS

GC_TIME_MILLIS

CPU_MILLISECONDS

PHYSICAL_MEMORY_BYTES

VIRTUAL_MEMORY_BYTES

COMMITTED_HEAP_BYTES

INPUT_RECORDS_PROCESSED

OUTPUT_RECORDS

OUTPUT_LARGE_RECORDS

SKIPPED_RECORDS

OUTPUT_BYTES

OUTPUT_BYTES_WITH_OVERHEAD

OUTPUT_BYTES_PHYSICAL

ADDITIONAL_SPILLS_BYTES_WRITTEN

ADDITIONAL_SPILLS_BYTES_READ

ADDITIONAL_SPILL_COUNT

SHUFFLE_CHUNK_COUNT

INPUT_GROUPS

SHUFFLE_BYTES

SHUFFLE_BYTES_DECOMPRESSED

SHUFFLE_BYTES_TO_MEM

SHUFFLE_BYTES_TO_DISK

SHUFFLE_BYTES_DISK_DIRECT

NUM_MEM_TO_DISK_MERGES

NUM_DISK_TO_DISK_MERGES

SHUFFLE_PHASE_TIME

MERGE_PHASE_TIME

FIRST_EVENT_RECEIVED

LAST_EVENT_RECEIVED

Method Detail

values

valueOf