org.apache.tez.dag.app.dag.speculation.legacy

Class LegacySpeculator

java.lang.Object

org.apache.tez.dag.app.dag.speculation.legacy.LegacySpeculator

public class LegacySpeculator
extends Object

Maintains runtime estimation statistics. Makes periodic updates estimates based on progress and decides on when to trigger a speculative attempt. Speculation attempts are triggered when the estimated runtime is more than a threshold beyond the mean runtime and the original task still has enough estimated runtime left that the speculative version is expected to finish sooner than that. If the original is close to completion then we dont start a speculation because it may be likely a wasted attempt. There is a delay between successive speculations.

Constructor Summary

Constructors

Constructor and Description
LegacySpeculator(org.apache.hadoop.conf.Configuration conf, AppContext context, Vertex vertex)
LegacySpeculator(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.util.Clock clock, Vertex vertex)
LegacySpeculator(org.apache.hadoop.conf.Configuration conf, TaskRuntimeEstimator estimator, org.apache.hadoop.yarn.util.Clock clock, Vertex vertex)

Method Summary

Modifier and Type	Method and Description
protected void	addSpeculativeAttempt(org.apache.tez.dag.records.TezTaskID taskID)
int	getMinimumAllowedSpeculativeTasks()
double	getProportionRunningTasksSpeculatable()
double	getProportionTotalTasksSpeculatable()
long	getSoonestRetryAfterNoSpeculate()
long	getSoonestRetryAfterSpeculate()
void	handle(SpeculatorEvent event)
void	notifyAttemptStarted(org.apache.tez.dag.records.TezTaskAttemptID taId, long timestamp)
void	notifyAttemptStatusUpdate(org.apache.tez.dag.records.TezTaskAttemptID taId, TaskAttemptState reportedState, long timestamp)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LegacySpeculator

public LegacySpeculator(org.apache.hadoop.conf.Configuration conf,
                        AppContext context,
                        Vertex vertex)

LegacySpeculator

public LegacySpeculator(org.apache.hadoop.conf.Configuration conf,
                        org.apache.hadoop.yarn.util.Clock clock,
                        Vertex vertex)

LegacySpeculator

public LegacySpeculator(org.apache.hadoop.conf.Configuration conf,
                        TaskRuntimeEstimator estimator,
                        org.apache.hadoop.yarn.util.Clock clock,
                        Vertex vertex)

Method Detail

getMinimumAllowedSpeculativeTasks

public int getMinimumAllowedSpeculativeTasks()

getProportionTotalTasksSpeculatable

public double getProportionTotalTasksSpeculatable()

getProportionRunningTasksSpeculatable

public double getProportionRunningTasksSpeculatable()

getSoonestRetryAfterNoSpeculate

public long getSoonestRetryAfterNoSpeculate()

getSoonestRetryAfterSpeculate

public long getSoonestRetryAfterSpeculate()

notifyAttemptStarted

public void notifyAttemptStarted(org.apache.tez.dag.records.TezTaskAttemptID taId,
                                 long timestamp)

notifyAttemptStatusUpdate

public void notifyAttemptStatusUpdate(org.apache.tez.dag.records.TezTaskAttemptID taId,
                                      TaskAttemptState reportedState,
                                      long timestamp)

handle

public void handle(SpeculatorEvent event)

addSpeculativeAttempt

protected void addSpeculativeAttempt(org.apache.tez.dag.records.TezTaskID taskID)