Overview | Package | Class | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## cascading.flow.planner
Class FlowPlanner<F extends Flow,Config>

    java.lang.Object
      cascading.flow.planner.FlowPlanner<F,Config>

Direct Known Subclasses:
  HadoopPlanner, LocalPlanner

    public abstract class FlowPlanner<F extends Flow,Config>

extends Object

Class FlowPlanner is the base class for all planner implementations.

Field Summary
`protected AssertionLevel` | `assertionLevel` Field assertionLevel
`protected String` | `checkpointRootPath`
`protected DebugLevel` | `debugLevel` Field debugLevel
`protected Map<Object,Object>` | `properties` Field properties

Constructor Summary
`FlowPlanner()`

Method Summary
`abstract F` | `buildFlow(FlowDef flowDef)` Method buildFlow renders the actual Flow instance.
`protected ElementGraph` | `createElementGraph(FlowDef flowDef, Pipe[] flowTails)`
`protected abstract Flow` | `createFlow(FlowDef flowDef)`
`protected void` | `failOnGroupEverySplit(ElementGraph elementGraph)`
`protected void` | `failOnLoneGroupAssertion(ElementGraph elementGraph)` Verifies that there are not only GroupAssertions following any given Group instance.
`protected void` | `failOnMissingGroup(ElementGraph elementGraph)`
`protected void` | `failOnMisusedBuffer(ElementGraph elementGraph)`
`abstract Config` | `getConfig()`
`abstract PlatformInfo` | `getPlatformInfo()`
`Map<Object,Object>` | `getProperties()`
`protected PlannerException` | `handleExceptionDuringPlanning(Exception exception, ElementGraph elementGraph)`
`protected void` | `handleJobPartitioning(ElementGraph elementGraph)` Inserts a temporary Tap between logical MR jobs.
`protected void` | `handleJoins(ElementGraph elementGraph)` Prevent leftmost sources from sourcing a downstream join on the rightmost side intra-task by inserting a temp tap between the left-sourced join and right-sourced join.
`protected void` | `handleNonSafeOperations(ElementGraph elementGraph)`
`void` | `initialize(FlowConnector flowConnector, Map<Object,Object> properties)`
`protected void` | `insertTempTapAfter(ElementGraph graph, Pipe pipe)` Method insertTapAfter ...
`protected Tap` | `makeTempTap(String name)`
`protected abstract Tap` | `makeTempTap(String prefix, String name)`
`protected Pipe[]` | `resolveAssemblyPlanners(FlowDef flowDef, Flow flow, Pipe[] pipes)`
`protected Pipe[]` | `resolveTails(FlowDef flowDef, Flow<Config> flow)`
`protected void` | `verifyAllTaps(FlowDef flowDef)`
`protected void` | `verifyAssembly(FlowDef flowDef, Pipe[] tails)`
`protected void` | `verifyCheckpoints(FlowDef flowDef, Pipe[] flowTails)`
`protected void` | `verifyPipeAssemblyEndPoints(FlowDef flowDef, Pipe[] flowTails)` Method verifyEndPoints verifies there aren't dupe names in heads or tails.
`protected void` | `verifySourceNotSinks(Map<String,Tap> sources, Map<String,Tap> sinks)`
`protected void` | `verifyTaps(Map<String,Tap> taps, boolean areSources, boolean mayNotBeEmpty)` Method verifyTaps ...
`protected void` | `verifyTraps(FlowDef flowDef, Pipe[] flowTails)`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

### properties

    protected Map<Object,Object> properties

  Field properties

### checkpointRootPath

    protected String checkpointRootPath

### assertionLevel

    protected AssertionLevel assertionLevel

  Field assertionLevel

### debugLevel

    protected DebugLevel debugLevel

  Field debugLevel

Constructor Detail

### FlowPlanner

    public FlowPlanner()

Method Detail

### getProperties

    public Map<Object,Object> getProperties()

### getConfig

    public abstract Config getConfig()

### getPlatformInfo

    public abstract PlatformInfo getPlatformInfo()

### initialize

    public void initialize(FlowConnector flowConnector,
                           Map<Object,Object> properties)

### createFlow

    protected abstract Flow createFlow(FlowDef flowDef)

### buildFlow

    public abstract F buildFlow(FlowDef flowDef)

  Method buildFlow renders the actual Flow instance.

  Parameters:
    `flowDef` -
  Returns:
    Flow

### resolveTails

    protected Pipe[] resolveTails(FlowDef flowDef,
                                  Flow<Config> flow)

### resolveAssemblyPlanners

    protected Pipe[] resolveAssemblyPlanners(FlowDef flowDef,
                                             Flow flow,
                                             Pipe[] pipes)

### verifyAssembly

    protected void verifyAssembly(FlowDef flowDef,
                                  Pipe[] tails)

### verifyAllTaps

    protected void verifyAllTaps(FlowDef flowDef)

### createElementGraph

    protected ElementGraph createElementGraph(FlowDef flowDef,
                                              Pipe[] flowTails)

### verifySourceNotSinks

    protected void verifySourceNotSinks(Map<String,Tap> sources,
                                        Map<String,Tap> sinks)

### verifyTaps

    protected void verifyTaps(Map<String,Tap> taps,
                              boolean areSources,
                              boolean mayNotBeEmpty)

  Method verifyTaps ...

  Parameters:
    `taps` - of type Map
    `areSources` - of type boolean
    `mayNotBeEmpty` - of type boolean

### verifyPipeAssemblyEndPoints

    protected void verifyPipeAssemblyEndPoints(FlowDef flowDef,
                                               Pipe[] flowTails)

  Method verifyEndPoints verifies

  there aren't dupe names in heads or tails. all the sink and source tap names match up with tail and head pipes

### verifyTraps

    protected void verifyTraps(FlowDef flowDef,
                               Pipe[] flowTails)

### verifyCheckpoints

    protected void verifyCheckpoints(FlowDef flowDef,
                                     Pipe[] flowTails)

### failOnLoneGroupAssertion

    protected void failOnLoneGroupAssertion(ElementGraph elementGraph)

  Verifies that there are not only GroupAssertions following any given Group instance. This will adversely affect the stream entering any subsequent Tap of Each instances.

### failOnMissingGroup

    protected void failOnMissingGroup(ElementGraph elementGraph)

### failOnMisusedBuffer

    protected void failOnMisusedBuffer(ElementGraph elementGraph)

### failOnGroupEverySplit

    protected void failOnGroupEverySplit(ElementGraph elementGraph)

### handleExceptionDuringPlanning

    protected PlannerException handleExceptionDuringPlanning(Exception exception,
                                                             ElementGraph elementGraph)

### handleNonSafeOperations

    protected void handleNonSafeOperations(ElementGraph elementGraph)

### insertTempTapAfter

    protected void insertTempTapAfter(ElementGraph graph,
                                      Pipe pipe)

  Method insertTapAfter ...

  Parameters:
    `graph` - of type PipeGraph
    `pipe` - of type Pipe

### makeTempTap

    protected Tap makeTempTap(String name)

### makeTempTap

    protected abstract Tap makeTempTap(String prefix,
                                       String name)

### handleJobPartitioning

    protected void handleJobPartitioning(ElementGraph elementGraph)

  Inserts a temporary Tap between logical MR jobs.

  Since all joins are at groups or splices, depth first search is safe

  todo: refactor so that rules are applied to path segments bounded by taps todo: this would allow balancing of operations within paths instead of pushing todo: all operations up. may allow for consolidation of rules

  Parameters:
    `elementGraph` - of type PipeGraph

### handleJoins

    protected void handleJoins(ElementGraph elementGraph)

  Prevent leftmost sources from sourcing a downstream join on the rightmost side intra-task by inserting a temp tap between the left-sourced join and right-sourced join.

  Parameters:
    `elementGraph` -

Overview | Package | Class | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2007-2013 Concurrent, Inc. All Rights Reserved.