org.apache.tez.mapreduce.examples

Class MRRSleepJob

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.tez.mapreduce.examples.MRRSleepJob

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class MRRSleepJob
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool

Dummy class for testing MR framefork. Sleeps for a defined period of time in mapper and reducer. Generates fake input for map / reduce jobs. Note that generated number of input pairs is in the order of numMappers * mapSleepTime / 100, so the job uses some disk space.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MRRSleepJob.EmptySplit
static class	MRRSleepJob.ISleepReducer
static class	MRRSleepJob.MRRSleepJobPartitioner
static class	MRRSleepJob.SleepInputFormat
static class	MRRSleepJob.SleepMapper
static class	MRRSleepJob.SleepReducer

Field Summary

Fields

Modifier and Type	Field and Description
static String	IREDUCE_SLEEP_COUNT
static String	IREDUCE_SLEEP_TIME
static String	IREDUCE_STAGES_COUNT
static String	IREDUCE_TASKS_COUNT
static String	MAP_ERROR_TASK_IDS
static String	MAP_FATAL_ERROR
static String	MAP_SLEEP_COUNT
static String	MAP_SLEEP_TIME
static String	MAP_THROW_ERROR
static String	REDUCE_SLEEP_COUNT
static String	REDUCE_SLEEP_TIME

Constructor Summary

Constructors

Constructor and Description
MRRSleepJob()

Method Summary

Modifier and Type	Method and Description
org.apache.tez.dag.api.DAG	createDAG(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path stagingDir, int numMapper, int numReducer, int iReduceStagesCount, int numIReducer, long mapSleepTime, int mapSleepCount, long reduceSleepTime, int reduceSleepCount, long iReduceSleepTime, int iReduceSleepCount, boolean writeSplitsToDFS, boolean generateSplitsInAM)
org.apache.hadoop.mapreduce.Job	createJob(int numMapper, int numReducer, int iReduceStagesCount, int numIReducer, long mapSleepTime, int mapSleepCount, long reduceSleepTime, int reduceSleepCount, long iReduceSleepTime, int iReduceSleepCount)
static void	main(String[] args)
int	run(String[] args)

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Field Detail

MAP_SLEEP_COUNT

public static final String MAP_SLEEP_COUNT

See Also:

Constant Field Values

REDUCE_SLEEP_COUNT

public static final String REDUCE_SLEEP_COUNT

See Also:

Constant Field Values

MAP_SLEEP_TIME

public static final String MAP_SLEEP_TIME

See Also:

Constant Field Values

REDUCE_SLEEP_TIME

public static final String REDUCE_SLEEP_TIME

See Also:

Constant Field Values

IREDUCE_SLEEP_COUNT

public static final String IREDUCE_SLEEP_COUNT

See Also:

Constant Field Values

IREDUCE_SLEEP_TIME

public static final String IREDUCE_SLEEP_TIME

See Also:

Constant Field Values

IREDUCE_STAGES_COUNT

public static final String IREDUCE_STAGES_COUNT

See Also:

Constant Field Values

IREDUCE_TASKS_COUNT

public static final String IREDUCE_TASKS_COUNT

See Also:

Constant Field Values

MAP_THROW_ERROR

public static final String MAP_THROW_ERROR

See Also:

Constant Field Values

MAP_FATAL_ERROR

public static final String MAP_FATAL_ERROR

See Also:

Constant Field Values

MAP_ERROR_TASK_IDS

public static final String MAP_ERROR_TASK_IDS

See Also:

Constant Field Values

Constructor Detail

MRRSleepJob

public MRRSleepJob()

Method Detail

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception

createDAG

public org.apache.tez.dag.api.DAG createDAG(org.apache.hadoop.conf.Configuration conf,
                                            org.apache.hadoop.fs.Path stagingDir,
                                            int numMapper,
                                            int numReducer,
                                            int iReduceStagesCount,
                                            int numIReducer,
                                            long mapSleepTime,
                                            int mapSleepCount,
                                            long reduceSleepTime,
                                            int reduceSleepCount,
                                            long iReduceSleepTime,
                                            int iReduceSleepCount,
                                            boolean writeSplitsToDFS,
                                            boolean generateSplitsInAM)
                                     throws IOException,
                                            org.apache.hadoop.yarn.exceptions.YarnException

Throws:

IOException

org.apache.hadoop.yarn.exceptions.YarnException

createJob

public org.apache.hadoop.mapreduce.Job createJob(int numMapper,
                                                 int numReducer,
                                                 int iReduceStagesCount,
                                                 int numIReducer,
                                                 long mapSleepTime,
                                                 int mapSleepCount,
                                                 long reduceSleepTime,
                                                 int reduceSleepCount,
                                                 long iReduceSleepTime,
                                                 int iReduceSleepCount)
                                          throws IOException

Throws:

IOException

run

public int run(String[] args)
        throws Exception

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

Exception