org.apache.tez.examples

Class TezExampleBase

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.tez.examples.TezExampleBase

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

Direct Known Subclasses:

CartesianProduct, HashJoinExample, JoinDataGen, JoinValidate, OrderedWordCount, SimpleSessionExample, SortMergeJoinExample, WordCount

@InterfaceAudience.Private
public abstract class TezExampleBase
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool

Field Summary

Fields

Modifier and Type	Field and Description
protected static String	COUNTER_LOG
protected static String	DISABLE_SPLIT_GROUPING
protected static String	GENERATE_SPLIT_IN_CLIENT
protected static String	LEAVE_AM_RUNNING
protected static String	LOCAL_MODE
protected static String	RECONNECT_APP_ID

Constructor Summary

Constructors

Constructor and Description
TezExampleBase()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.yarn.api.records.ApplicationId	getAppId()
protected boolean	isCountersLog()
protected boolean	isDisableSplitGrouping()
protected boolean	isGenerateSplitInClient()
protected void	printExtraOptionsUsage(PrintStream ps)
protected abstract void	printUsage() Print usage instructions for this example
int	run(String[] args)
int	run(TezConfiguration conf, String[] args, TezClient tezClient) Utility method to use the example from within code or a test.
int	runDag(DAG dag, boolean printCounters, org.slf4j.Logger logger)
protected abstract int	runJob(String[] args, TezConfiguration tezConf, TezClient tezClient) Create and execute the actual DAG for the example
protected abstract int	validateArgs(String[] otherArgs) Validate the arguments

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Field Detail

DISABLE_SPLIT_GROUPING

protected static final String DISABLE_SPLIT_GROUPING

See Also:

Constant Field Values

LOCAL_MODE

protected static final String LOCAL_MODE

See Also:

Constant Field Values

COUNTER_LOG

protected static final String COUNTER_LOG

See Also:

Constant Field Values

GENERATE_SPLIT_IN_CLIENT

protected static final String GENERATE_SPLIT_IN_CLIENT

See Also:

Constant Field Values

LEAVE_AM_RUNNING

protected static final String LEAVE_AM_RUNNING

See Also:

Constant Field Values

RECONNECT_APP_ID

protected static final String RECONNECT_APP_ID

See Also:

Constant Field Values

Constructor Detail

TezExampleBase

public TezExampleBase()

Method Detail

isCountersLog

protected boolean isCountersLog()

isDisableSplitGrouping

protected boolean isDisableSplitGrouping()

isGenerateSplitInClient

protected boolean isGenerateSplitInClient()

run

public final int run(String[] args)
              throws Exception

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

Exception

run

public int run(TezConfiguration conf,
               String[] args,
               @Nullable
               TezClient tezClient)
        throws Exception

Utility method to use the example from within code or a test.

Parameters:

conf - the tez configuration instance which will be used to crate the DAG and possible the Tez Client.

args - arguments to the example

tezClient - an existing running TezClient instance if one exists. If no TezClient is specified (null), one will be created based on the provided configuration. If TezClient is specified, local mode option can not been specified in arguments, it takes no effect.

Returns:

Zero indicates success, non-zero indicates failure

Throws:

Exception

runDag

public int runDag(DAG dag,
                  boolean printCounters,
                  org.slf4j.Logger logger)
           throws TezException,
                  InterruptedException,
                  IOException

Parameters:

dag - the dag to execute

printCounters - whether to print counters or not

logger - the logger to use while printing diagnostics

Returns:

Zero indicates success, non-zero indicates failure

Throws:

TezException

InterruptedException

IOException

printUsage

protected abstract void printUsage()

Print usage instructions for this example

printExtraOptionsUsage

protected void printExtraOptionsUsage(PrintStream ps)

validateArgs

protected abstract int validateArgs(String[] otherArgs)

Validate the arguments

Parameters:

otherArgs - arguments, if any

Returns:

Zero indicates success, non-zero indicates failure

runJob

protected abstract int runJob(String[] args,
                              TezConfiguration tezConf,
                              TezClient tezClient)
                       throws Exception

Create and execute the actual DAG for the example

Parameters:

args - arguments for execution

tezConf - the tez configuration instance to be used while processing the DAG

tezClient - the tez client instance to use to run the DAG if any custom monitoring is required. Otherwise the utility method runDag(org.apache.tez.dag.api.DAG, boolean, org.slf4j.Logger) should be used

Returns:

Zero indicates success, non-zero indicates failure

Throws:

IOException

TezException

Exception

getAppId

@InterfaceAudience.Private
public org.apache.hadoop.yarn.api.records.ApplicationId getAppId()