org.apache.tez.dag.app

Class RecoveryParser

java.lang.Object

org.apache.tez.dag.app.RecoveryParser

public class RecoveryParser
extends Object

RecoverParser is mainly for Tez AM Recovery. It would read the recovery events. (summary & non-summary)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	RecoveryParser.DAGRecoveryData
static class	RecoveryParser.TaskAttemptRecoveryData
static class	RecoveryParser.TaskRecoveryData
static class	RecoveryParser.VertexRecoveryData

Constructor Summary

Constructors

Constructor and Description
RecoveryParser(DAGAppMaster dagAppMaster, org.apache.hadoop.fs.FileSystem recoveryFS, org.apache.hadoop.fs.Path recoveryDataDir, int currentAttemptId)

Method Summary

Modifier and Type	Method and Description
static void	main(String[] argv)
static List<HistoryEvent>	parseDAGRecoveryFile(org.apache.hadoop.fs.FSDataInputStream inputStream)
RecoveryParser.DAGRecoveryData	parseRecoveryData() 1.
static List<HistoryEvent>	readRecoveryEvents(org.apache.tez.dag.api.TezConfiguration tezConf, org.apache.hadoop.yarn.api.records.ApplicationId appId, int attempt)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RecoveryParser

public RecoveryParser(DAGAppMaster dagAppMaster,
                      org.apache.hadoop.fs.FileSystem recoveryFS,
                      org.apache.hadoop.fs.Path recoveryDataDir,
                      int currentAttemptId)
               throws IOException

Throws:

IOException

Method Detail

parseDAGRecoveryFile

public static List<HistoryEvent> parseDAGRecoveryFile(org.apache.hadoop.fs.FSDataInputStream inputStream)
                                               throws IOException

Throws:

IOException

readRecoveryEvents

public static List<HistoryEvent> readRecoveryEvents(org.apache.tez.dag.api.TezConfiguration tezConf,
                                                    org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                                    int attempt)
                                             throws IOException

Throws:

IOException

main

public static void main(String[] argv)
                 throws IOException

Throws:

IOException

parseRecoveryData

public RecoveryParser.DAGRecoveryData parseRecoveryData()
                                                 throws IOException

1. Read Summary Recovery file and build DAGSummaryData Check whether it is recoverable based on the summary file (whether dag is in the middle of committing) 2. Read the non-Summary Recovery file and build DAGRecoveryData Check whether it is recoverable based on both the summary file and non-summary file (whether vertex has completed its committing, but its full non-summary recovery events are not seen)

Returns:

DAGRecoveryData

Throws:

IOException