Initial support for YARN Timeline was introduced in Apache Hadoop 2.4.0. Support for ACLs in Timeline was introduced in Apache Hadoop 2.6.0. Support for Timeline was introduced in Tez in 0.5.x ( with some experimental support in 0.4.x ). However, Tez ACLs integration with Timeline is only available from Tez 0.6.0 onwards.
Tez uses YARN Timeline as its application history store. Tez stores most of its lifecycle information into this history store such as: - DAG information such as: - DAG Plan - DAG Submission, Start and End times - DAG Counters - Final status of the DAG and additional diagnostics - Vertex, Task and Task Attempt Information - Start and End times - Counters - Diagnostics
Using the above information, a user can analyze a Tez DAG while it is running and after it has completed.
Given that the support for YARN Timeline with full security was only realized in Apache Hadoop 2.6.0, some features may or may not be supported depending on which version of Apache Hadoop is used.
|Hadoop 2.2.x, 2.3.x||Hadoop 2.4.x, 2.5.x||Hadoop 2.6.x and higher|
|Timeline with ACLs Support||No||No||Yes|
By default, Tez writes its history data into a file on HDFS. To use Timeline, add the following property into your tez-site.xml:
For Tez 0.4.x, the above property is not respected. For 0.4.x, please set the following property:
When using Tez with Apache Hadoop 2.4.x or 2.5.x, given that these versions are not fully secure, the following property also needs to be enabled: