cascading.flow.tez

## Class Hadoop2TezFlowStep

- java.lang.Object
- - cascading.flow.planner.BaseFlowStep<org.apache.tez.dag.api.TezConfiguration>
  - - cascading.flow.tez.Hadoop2TezFlowStep

- All Implemented Interfaces:
    FlowStep<org.apache.tez.dag.api.TezConfiguration>, ProcessModel, ProcessLogger, Serializable

      public class Hadoop2TezFlowStep
      extends BaseFlowStep<org.apache.tez.dag.api.TezConfiguration>

  See Also:
    Serialized Form

- - ### Field Summary

    - ### Fields inherited from class cascading.flow.planner.BaseFlowStep

      `flowStepStats, sinks, sources, tempSink`

    - ### Fields inherited from interface cascading.flow.FlowStep

      `CASCADING_FLOW_STEP_ID`

    - ### Fields inherited from interface cascading.util.ProcessLogger

      `NULL`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `Hadoop2TezFlowStep(ElementGraph elementGraph, FlowNodeGraph flowNodeGraph)`

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `clean(org.apache.tez.dag.api.TezConfiguration config)`
    `protected FlowStepJob` | `createFlowStepJob(ClientState clientState, FlowProcess<org.apache.tez.dag.api.TezConfiguration> flowProcess, org.apache.tez.dag.api.TezConfiguration initializedStepConfig)`
    `org.apache.tez.dag.api.TezConfiguration` | `createInitializedConfig(FlowProcess<org.apache.tez.dag.api.TezConfiguration> flowProcess, org.apache.tez.dag.api.TezConfiguration parentConfig)` Public for testing.
    `org.apache.tez.dag.api.Vertex` | `createVertex(FlowProcess<org.apache.tez.dag.api.TezConfiguration> flowProcess, org.apache.tez.dag.api.TezConfiguration initializedConfig, FlowNode flowNode)`
    `Map<String,org.apache.hadoop.yarn.api.records.LocalResource>` | `getAllLocalResources()`
    `Map<Object,Object>` | `getConfigAsProperties()` Method getConfigAsProperties converts the internal configuration object into a `Map` of key value pairs.
    `protected String` | `getOutputPath(org.apache.hadoop.conf.Configuration sinkConf)`
    `protected int` | `getParallelism(FlowNode flowNode, org.apache.hadoop.mapred.JobConf conf)`
    `protected Map<FlowElement,org.apache.hadoop.conf.Configuration>` | `initFromSinks(FlowNode flowNode, FlowProcess<? extends org.apache.hadoop.conf.Configuration> flowProcess, org.apache.hadoop.conf.Configuration conf)`
    `protected Map<FlowElement,org.apache.hadoop.conf.Configuration>` | `initFromSources(FlowNode flowNode, FlowProcess<org.apache.tez.dag.api.TezConfiguration> flowProcess, org.apache.hadoop.conf.Configuration conf, Map<String,org.apache.hadoop.yarn.api.records.LocalResource> taskLocalResources)`
    `protected void` | `initFromTraps(FlowNode flowNode, FlowProcess<? extends org.apache.hadoop.conf.Configuration> flowProcess, org.apache.hadoop.conf.Configuration conf)`
    `protected boolean` | `isFileOutputFormat(Class outputFormatClass)`
    `void` | `syncArtifacts()`

    - ### Methods inherited from class cascading.flow.planner.BaseFlowStep

      `addAllGroups, addGroup, addGroups, addListener, addProcessAnnotation, addProcessAnnotation, addSink, addSinks, addSource, addSources, clean, commitSinks, configure, containsPipeNamed, createClientState, equals, fireOnCompleted, fireOnRunning, fireOnStarting, fireOnStopping, fireOnThrowable, getAllAccumulatedSources, getAllOperations, getAnnotations, getConfig, getCreateFlowStepJob, getElementGraph, getFlow, getFlowID, getFlowName, getFlowNodeGraph, getFlowStepDescriptor, getFlowStepJob, getFlowStepStats, getGroup, getGroups, getID, getName, getNextFlowElement, getNextScope, getNodeDisplayName, getNumFlowNodes, getOrdinal, getPreviousScopes, getProcessAnnotations, getSink, getSinkElements, getSinkName, getSinkTaps, getSinkWith, getSourceElements, getSourceName, getSourceTaps, getSourceWith, getStepDisplayName, getStepDisplayName, getSubmitPriority, getTrap, getTrapMap, getTraps, hashCode, hasListeners, initConfFromNodeConfigDef, initConfFromStepConfigDef, isDebugEnabled, isInfoEnabled, logDebug, logError, logError, logInfo, logWarn, logWarn, logWarn, prepareResources, removeListener, rollbackSinks, setConfig, setFlow, setFlowName, setFlowStepDescriptor, setFlowStepStats, setName, setOrdinal, setSubmitPriority, toString`

    - ### Methods inherited from class java.lang.Object

      `clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

- - ### Constructor Detail

    - #### Hadoop2TezFlowStep

          public Hadoop2TezFlowStep(ElementGraph elementGraph,
                            FlowNodeGraph flowNodeGraph)

  - ### Method Detail

    - #### getConfigAsProperties

          public Map<Object,Object> getConfigAsProperties()

      Description copied from interface: `FlowStep`
      Method getConfigAsProperties converts the internal configuration object into a `Map` of key value pairs.
      Specified by:
        `getConfigAsProperties` in interface `FlowStep<org.apache.tez.dag.api.TezConfiguration>`
      Overrides:
        `getConfigAsProperties` in class `BaseFlowStep<org.apache.tez.dag.api.TezConfiguration>`
      Returns:
        a Map of key/value pairs, may return an empty collection if unsupported

    - #### createInitializedConfig

          public org.apache.tez.dag.api.TezConfiguration createInitializedConfig(FlowProcess<org.apache.tez.dag.api.TezConfiguration> flowProcess,
                                                                        org.apache.tez.dag.api.TezConfiguration parentConfig)

      Description copied from class: `BaseFlowStep`
      Public for testing.
      Specified by:
        `createInitializedConfig` in class `BaseFlowStep<org.apache.tez.dag.api.TezConfiguration>`
      Returns:

    - #### createFlowStepJob

          protected FlowStepJob createFlowStepJob(ClientState clientState,
                                      FlowProcess<org.apache.tez.dag.api.TezConfiguration> flowProcess,
                                      org.apache.tez.dag.api.TezConfiguration initializedStepConfig)

      Specified by:
        `createFlowStepJob` in class `BaseFlowStep<org.apache.tez.dag.api.TezConfiguration>`

    - #### createVertex

          public org.apache.tez.dag.api.Vertex createVertex(FlowProcess<org.apache.tez.dag.api.TezConfiguration> flowProcess,
                                                   org.apache.tez.dag.api.TezConfiguration initializedConfig,
                                                   FlowNode flowNode)

    - #### getOutputPath

          protected String getOutputPath(org.apache.hadoop.conf.Configuration sinkConf)

    - #### isFileOutputFormat

          protected boolean isFileOutputFormat(Class outputFormatClass)

    - #### getParallelism

          protected int getParallelism(FlowNode flowNode,
                           org.apache.hadoop.mapred.JobConf conf)

    - #### initFromSources

          protected Map<FlowElement,org.apache.hadoop.conf.Configuration> initFromSources(FlowNode flowNode,
                                                                              FlowProcess<org.apache.tez.dag.api.TezConfiguration> flowProcess,
                                                                              org.apache.hadoop.conf.Configuration conf,
                                                                              Map<String,org.apache.hadoop.yarn.api.records.LocalResource> taskLocalResources)

    - #### initFromSinks

          protected Map<FlowElement,org.apache.hadoop.conf.Configuration> initFromSinks(FlowNode flowNode,
                                                                            FlowProcess<? extends org.apache.hadoop.conf.Configuration> flowProcess,
                                                                            org.apache.hadoop.conf.Configuration conf)

    - #### initFromTraps

          protected void initFromTraps(FlowNode flowNode,
                           FlowProcess<? extends org.apache.hadoop.conf.Configuration> flowProcess,
                           org.apache.hadoop.conf.Configuration conf)

    - #### clean

          public void clean(org.apache.tez.dag.api.TezConfiguration config)

      Specified by:
        `clean` in class `BaseFlowStep<org.apache.tez.dag.api.TezConfiguration>`

    - #### syncArtifacts

          public void syncArtifacts()

    - #### getAllLocalResources

          public Map<String,org.apache.hadoop.yarn.api.records.LocalResource> getAllLocalResources()

Copyright © 2007-2015 Concurrent, Inc. All Rights Reserved.