## cascading.stats.hadoop
Class BaseHadoopStepStats

    java.lang.Object
      cascading.stats.CascadingStats
          cascading.stats.FlowStepStats
              cascading.stats.hadoop.BaseHadoopStepStats

All Implemented Interfaces:
  Serializable

Direct Known Subclasses:
  HadoopStepStats

    public abstract class BaseHadoopStepStats

extends FlowStepStats

Class BaseHadoopStepStats is a base class to Hadoop specific statistics and methods to underlying Hadoop facilities.

See Also:
  Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class cascading.stats.CascadingStats
`CascadingStats.Status`

Field Summary
`static String` `COUNTER_TIMEOUT_PROPERTY`
`static int` `TIMEOUT_MAX`

Fields inherited from class cascading.stats.CascadingStats
`clientState, STATS_STORE_INTERVAL`

Constructor Summary
`protected` `BaseHadoopStepStats(FlowStep<JobConf> flowStep, ClientState clientState)`

Method Summary
`protected abstract void` `addAttemptsToTaskStats(Map<String,HadoopSliceStats> taskStats, boolean captureAttempts)`
`protected abstract void` `addTaskStats(Map<String,HadoopSliceStats> taskStats, HadoopSliceStats.Kind kind, boolean skipLast)`
`protected Counters` `cachedCounters()`
`protected Counters` `cachedCounters(boolean force)`
`void` `captureDetail()` Method captureDetail captures statistics task details and completion events.
`void` `captureDetail(boolean captureAttempts)`
`Set<String>` `getChildIDs()`
`Collection` `getChildren()` Method getChildren returns the children of this HadoopStepStats object.
`Collection<String>` `getCounterGroups()` Method getCounterGroups returns all of the Hadoop counter groups.
`Collection<String>` `getCounterGroupsMatching(String regex)` Method getCounterGroupsMatching returns all the Hadoop counter groups that match the give regex pattern.
`Collection<String>` `getCountersFor(String group)` Method getCountersFor returns the Hadoop counters for the given group.
`long` `getCounterValue(Enum counter)` Method getCounterValue returns the Hadoop counter value for the given counter enum.
`long` `getCounterValue(String group, String counter)` Method getCounterValue returns the Hadoop counter value for the given group and counter name.
`abstract JobClient` `getJobClient()` Method getJobClient returns the Hadoop `JobClient` managing this Hadoop job.
`String` `getJobID()` Method getJobID returns the Hadoop running job JobID.
`float` `getMapProgress()` Returns the underlying Map tasks progress percentage.
`int` `getNumMapTasks()` Method getNumMapTasks returns the numMapTasks from the Hadoop job file.
`int` `getNumReduceTasks()` Method getNumReduceTasks returns the numReducerTasks from the Hadoop job file.
`float` `getReduceProgress()` Returns the underlying Reduce tasks progress percentage.
`abstract RunningJob` `getRunningJob()` Method getRunningJob returns the Hadoop `RunningJob` managing this Hadoop job.
`String` `getStatusURL()`
`Map<String,HadoopSliceStats>` `getTaskStats()` Method getTaskStats returns the taskStats of this HadoopStepStats object.
`void` `recordChildStats()` Synchronized to prevent state changes mid record, #stop may be called out of band
`protected void` `setTaskStats(Map<String,HadoopSliceStats> taskStats)`

Methods inherited from class cascading.stats.FlowStepStats
`getFlowStep, getID, recordInfo, toString`

Methods inherited from class cascading.stats.CascadingStats

`cleanup, getCountersFor, getCurrentDuration, getDuration, getFinishedTime, getName, getPendingTime, getRunTime, getStartTime, getStatsString, getStatus, getSubmitTime, getThrowable, isEngaged, isFailed, isFinished, isPending, isRunning, isSkipped, isStarted, isStopped, isSubmitted, isSuccessful, markFailed, markPending, markPendingTime, markRunning, markRunTime, markSkipped, markStarted, markStartedThenRunning, markStartTime, markStartToRunTime, markStopped, markSubmitted, markSubmitTime, markSuccessful, prepare, recordStats, setStatsStoreInterval`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Field Detail

### COUNTER_TIMEOUT_PROPERTY

    public static final String COUNTER_TIMEOUT_PROPERTY

See Also:
  Constant Field Values

### TIMEOUT_MAX

    public static final int TIMEOUT_MAX

See Also:
  Constant Field Values

Constructor Detail

### BaseHadoopStepStats

    protected BaseHadoopStepStats(FlowStep<JobConf> flowStep,
                                  ClientState clientState)

Method Detail

### getTaskStats

    public Map<String,HadoopSliceStats> getTaskStats()

  Method getTaskStats returns the taskStats of this HadoopStepStats object.

  Returns:
    the taskStats (type ArrayList<HadoopTaskStats>) of this HadoopStepStats object.

### setTaskStats

    protected void setTaskStats(Map<String,HadoopSliceStats> taskStats)

### getNumMapTasks

    public int getNumMapTasks()

  Method getNumMapTasks returns the numMapTasks from the Hadoop job file.

  Returns:
    the numMapTasks (type int) of this HadoopStepStats object.

### getNumReduceTasks

    public int getNumReduceTasks()

  Method getNumReduceTasks returns the numReducerTasks from the Hadoop job file.

  Returns:
    the numReducerTasks (type int) of this HadoopStepStats object.

### getJobID

    public String getJobID()

  Method getJobID returns the Hadoop running job JobID.

  Returns:
    the jobID (type String) of this HadoopStepStats object.

### getJobClient

    public abstract JobClient getJobClient()

  Method getJobClient returns the Hadoop `JobClient` managing this Hadoop job.

  Returns:
    the jobClient (type JobClient) of this HadoopStepStats object.

### getRunningJob

    public abstract RunningJob getRunningJob()

  Method getRunningJob returns the Hadoop `RunningJob` managing this Hadoop job.

  Returns:
    the runningJob (type RunningJob) of this HadoopStepStats object.

### getCounterGroups

    public Collection<String> getCounterGroups()

  Method getCounterGroups returns all of the Hadoop counter groups.

  Specified by:
    `getCounterGroups` in class `CascadingStats`
  Returns:
    the counterGroups (type Collection<String>) of this HadoopStepStats object.

### getCounterGroupsMatching

    public Collection<String> getCounterGroupsMatching(String regex)

  Method getCounterGroupsMatching returns all the Hadoop counter groups that match the give regex pattern.

  Specified by:
    `getCounterGroupsMatching` in class `CascadingStats`
  Parameters:
    `regex` - of String
  Returns:
    Collection<String>

### getCountersFor

    public Collection<String> getCountersFor(String group)

  Method getCountersFor returns the Hadoop counters for the given group.

  Specified by:
    `getCountersFor` in class `CascadingStats`
  Parameters:
    `group` - of String
  Returns:
    Collection<String>

### getCounterValue

    public long getCounterValue(Enum counter)

  Method getCounterValue returns the Hadoop counter value for the given counter enum.

  Specified by:
    `getCounterValue` in class `CascadingStats`
  Parameters:
    `counter` - of Enum
  Returns:
    long

### getCounterValue

    public long getCounterValue(String group,
                                String counter)

  Method getCounterValue returns the Hadoop counter value for the given group and counter name.

  Specified by:
    `getCounterValue` in class `CascadingStats`
  Parameters:
    `group` - of String
    `counter` - of String
  Returns:
    long

### cachedCounters

    protected Counters cachedCounters()

### cachedCounters

    protected Counters cachedCounters(boolean force)

### getMapProgress

    public float getMapProgress()

  Returns the underlying Map tasks progress percentage.

  This method is experimental.

  Returns:
    float

### getReduceProgress

    public float getReduceProgress()

  Returns the underlying Reduce tasks progress percentage.

  This method is experimental.

  Returns:
    float

### getStatusURL

    public String getStatusURL()

### getChildren

    public Collection getChildren()

  Method getChildren returns the children of this HadoopStepStats object.

  Specified by:
    `getChildren` in class `CascadingStats`
  Returns:
    the children (type Collection) of this HadoopStepStats object.

### getChildIDs

    public Set<String> getChildIDs()

### recordChildStats

    public void recordChildStats()

  Synchronized to prevent state changes mid record, #stop may be called out of band

  Specified by:
    `recordChildStats` in class `FlowStepStats`

### captureDetail

    public void captureDetail()

  Method captureDetail captures statistics task details and completion events.

  Specified by:
    `captureDetail` in class `CascadingStats`

### captureDetail

    public void captureDetail(boolean captureAttempts)

### addTaskStats

    protected abstract void addTaskStats(Map<String,HadoopSliceStats> taskStats,
                                         HadoopSliceStats.Kind kind,
                                         boolean skipLast)
                                  throws IOException

  Throws:
    `IOException`

### addAttemptsToTaskStats

    protected abstract void addAttemptsToTaskStats(Map<String,HadoopSliceStats> taskStats,
                                                   boolean captureAttempts)

Copyright © 2007-2014 Concurrent, Inc. All Rights Reserved.