cascading.flow

## Class FlowProcess<Config>

- java.lang.Object
- - cascading.flow.FlowProcess<Config>

- Direct Known Subclasses:
    FlowProcess.NullFlowProcess, FlowProcessWrapper

      public abstract class FlowProcess<Config>
      extends Object

  FlowProcess implementations provide a call-back interface into the current computing system. Each `Operation` is given a reference to a particular implementation, allowing it to get configuration properties, send a "keep alive" ping, or to set a counter value.

  Depending on the underlying system, FlowProcess instances are not continuous across all operations in a `Flow`. Thus, a call to `increment(Enum, long)` may start incrementing from zero if the operation making the call belongs to a subsequent 'job' or 'step' from any previous operations calling increment.

  A FlowProcess is roughly a child of `FlowSession`. FlowSession is roughly one to one with a particular `Flow`. And every FlowSession will have one or more FlowProcesses.
  See Also:
    `FlowSession`

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `FlowProcess.NullFlowProcess`

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static FlowProcess` | `NULL` Field NULL is a noop implementation of FlowSession.

  - ### Constructor Summary

    Constructors
    Modifier | Constructor and Description
    --- | ---
    `protected` | `FlowProcess()`
    `protected` | `FlowProcess(FlowProcess<Config> flowProcess)` Copy constructor.
    `protected` | `FlowProcess(FlowSession currentSession)`

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `closeTrapCollectors()`
    `abstract <C> C` | `copyConfig(C config)`
    `abstract FlowProcess<Config>` | `copyWith(Config config)`
    `abstract <C> Map<String,String>` | `diffConfigIntoMap(C defaultConfig, C updatedConfig)`
    `Boolean` | `getBooleanProperty(String key)` Method getBooleanProperty should be used to return configuration parameters from the underlying system.
    `boolean` | `getBooleanProperty(String key, boolean defaultValue)` Method getBooleanProperty should be used to return configuration parameters from the underlying system.
    `abstract Config` | `getConfig()` Method getConfig returns the actual instance of the underlying configuration instance.
    `abstract Config` | `getConfigCopy()`
    `abstract long` | `getCounterValue(Enum counter)` Method getCounterValue is used to retrieve a counter value.
    `abstract long` | `getCounterValue(String group, String counter)` Method getCounterValue is used to retrieve a counter value.
    `FlowSession` | `getCurrentSession()` Method getCurrentSession returns the currentSession of this FlowProcess object.
    `abstract int` | `getCurrentSliceNum()` Method getCurrentSliceNum returns an integer representing which slice instance currently running.
    `String` | `getID()` Method getID() returns the current
    `Integer` | `getIntegerProperty(String key)` Method getIntegerProperty should be used to return configuration parameters from the underlying system.
    `int` | `getIntegerProperty(String key, int defaultValue)` Method getIntegerProperty should be used to return configuration parameters from the underlying system.
    `abstract int` | `getNumProcessSlices()` Method getNumProcessSlices returns the number of parallel slices or tasks allocated for this process execution.
    `abstract Object` | `getProperty(String key)` Method getProperty should be used to return configuration parameters from the underlying system.
    `abstract Collection<String>` | `getPropertyKeys()` Method getPropertyKeys returns an immutable collection of all available property key values.
    `String` | `getStringProperty(String key)` Method getStringProperty should be used to return configuration parameters from the underlying system.
    `String` | `getStringProperty(String key, String defaultValue)` Method getStringProperty should be used to return configuration parameters from the underlying system.
    `TupleEntryCollector` | `getTrapCollectorFor(Tap trap)` Method getTrapCollectorFor will return a new `TupleEntryCollector` if one hasn't previously been created for the given trap Tap.
    `protected Map<Tap,TupleEntryCollector>` | `getTrapCollectors()`
    `abstract void` | `increment(Enum counter, long amount)` Method increment is used to increment a custom counter.
    `abstract void` | `increment(String group, String counter, long amount)` Method increment is used to increment a custom counter.
    `abstract boolean` | `isCounterStatusInitialized()` Method isCounterStatusInitialized returns true if it is safe to increment a counter or set a status.
    `abstract void` | `keepAlive()` Method keepAlive notifies the system that the current process is still alive.
    `abstract Config` | `mergeMapIntoConfig(Config defaultConfig, Map<String,String> map)`
    `abstract Object` | `newInstance(String className)` Method newInstance creates a new object instance from the given className argument delegating to any platform specific instantiation and configuration routines.
    `abstract TupleEntryCollector` | `openSystemIntermediateForWrite()`
    `abstract TupleEntryIterator` | `openTapForRead(Tap tap)` Method openTapForRead return a `TupleEntryIterator` for the given Tap instance.
    `abstract TupleEntryCollector` | `openTapForWrite(Tap tap)` Method openTapForWrite returns a (@link TupleCollector} for the given Tap instance.
    `abstract TupleEntryCollector` | `openTrapForWrite(Tap trap)` Method openTrapForWrite returns a (@link TupleCollector} for the given Tap instance.
    `void` | `setCurrentSession(FlowSession currentSession)` Method setCurrentSession sets the currentSession of this FlowProcess object.
    `abstract void` | `setStatus(String status)` Method setStatus is used to set the status of the current operation.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### NULL

          public static FlowProcess NULL

      Field NULL is a noop implementation of FlowSession.

  - ### Constructor Detail

    - #### FlowProcess

          protected FlowProcess()

    - #### FlowProcess

          protected FlowProcess(FlowSession currentSession)

    - #### FlowProcess

          protected FlowProcess(FlowProcess<Config> flowProcess)

      Copy constructor.

      Shares the underlying trap collector collection across copies to avoid a static collection.
      Parameters:
        `flowProcess` -

  - ### Method Detail

    - #### copyWith

          public abstract FlowProcess<Config> copyWith(Config config)

    - #### getID

          public String getID()

      Method getID() returns the current
      Returns:
        of type String

    - #### getCurrentSession

          public FlowSession getCurrentSession()

      Method getCurrentSession returns the currentSession of this FlowProcess object.
      Returns:
        the currentSession (type FlowSession) of this FlowProcess object.

    - #### setCurrentSession

          public void setCurrentSession(FlowSession currentSession)

      Method setCurrentSession sets the currentSession of this FlowProcess object.
      Parameters:
        `currentSession` - the currentSession of this FlowProcess object.

    - #### getNumProcessSlices

          public abstract int getNumProcessSlices()

      Method getNumProcessSlices returns the number of parallel slices or tasks allocated for this process execution.

      For MapReduce platforms, this is the same as the number of tasks for a given MapReduce job.
      Returns:
        an int

    - #### getCurrentSliceNum

          public abstract int getCurrentSliceNum()

      Method getCurrentSliceNum returns an integer representing which slice instance currently running.

      `0` (zero) is the first slice instance.
      Returns:
        an int

    - #### getProperty

          public abstract Object getProperty(String key)

      Method getProperty should be used to return configuration parameters from the underlying system.

      In the case of Hadoop, the current Configuration will be queried.
      Parameters:
        `key` - of type String
      Returns:
        an Object

    - #### getStringProperty

          public String getStringProperty(String key)

      Method getStringProperty should be used to return configuration parameters from the underlying system.

      In the case of Hadoop, the current Configuration will be queried.
      Parameters:
        `key` - of type String,
      Returns:
        null if property is not set

    - #### getStringProperty

          public String getStringProperty(String key,
                                 String defaultValue)

      Method getStringProperty should be used to return configuration parameters from the underlying system.

      In the case of Hadoop, the current Configuration will be queried.
      Parameters:
        `key` - of type String,
        `defaultValue` - of type String,
      Returns:
        `defaultValue` if property is not set

    - #### getIntegerProperty

          public Integer getIntegerProperty(String key)

      Method getIntegerProperty should be used to return configuration parameters from the underlying system.

      In the case of Hadoop, the current Configuration will be queried.
      Parameters:
        `key` - of type String,
      Returns:
        null if property is not set

    - #### getIntegerProperty

          public int getIntegerProperty(String key,
                               int defaultValue)

      Method getIntegerProperty should be used to return configuration parameters from the underlying system.

      In the case of Hadoop, the current Configuration will be queried.
      Parameters:
        `key` - of type String,
        `defaultValue` - of type int,
      Returns:
        `defaultValue` if property is not set

    - #### getBooleanProperty

          public Boolean getBooleanProperty(String key)

      Method getBooleanProperty should be used to return configuration parameters from the underlying system.

      In the case of Hadoop, the current Configuration will be queried.
      Parameters:
        `key` - of type Boolean, null if property is not set
      Returns:
        an Object

    - #### getBooleanProperty

          public boolean getBooleanProperty(String key,
                                   boolean defaultValue)

      Method getBooleanProperty should be used to return configuration parameters from the underlying system.

      In the case of Hadoop, the current Configuration will be queried.
      Parameters:
        `key` - of type String
        `defaultValue` - of type boolean
      Returns:
        `defaultValue` if property is not set

    - #### getPropertyKeys

          public abstract Collection<String> getPropertyKeys()

      Method getPropertyKeys returns an immutable collection of all available property key values.
      Returns:
        a Collection

    - #### newInstance

          public abstract Object newInstance(String className)

      Method newInstance creates a new object instance from the given className argument delegating to any platform specific instantiation and configuration routines.
      Parameters:
        `className` -
      Returns:
        an instance of className

    - #### keepAlive

          public abstract void keepAlive()

      Method keepAlive notifies the system that the current process is still alive. Use this method if a particular `Operation` takes some moments to complete. Each system is different, so calling ping every few seconds to every minute or so would be best.

      This method will fail silently if the underlying mechanism to notify keepAlive status are not initialized.

    - #### increment

          public abstract void increment(Enum counter,
                       long amount)

      Method increment is used to increment a custom counter. Counters must be of type Enum. The amount to increment must be a integer value.

      This method will fail if the underlying counter infrastructure is unavailable. See `isCounterStatusInitialized()`.
      Parameters:
        `counter` - of type Enum
        `amount` - of type int

    - #### increment

          public abstract void increment(String group,
                       String counter,
                       long amount)

      Method increment is used to increment a custom counter. The amount to increment must be a integer value.

      This method will fail if the underlying counter infrastructure is unavailable. See `isCounterStatusInitialized()`.
      Parameters:
        `group` - of type String
        `counter` - of type String
        `amount` - of type int

    - #### getCounterValue

          public abstract long getCounterValue(Enum counter)

      Method getCounterValue is used to retrieve a counter value.

      This method will fail if the underlying counter infrastructure is unavailable. See `isCounterStatusInitialized()`.
      Parameters:
        `counter` - of type Enum

    - #### getCounterValue

          public abstract long getCounterValue(String group,
                             String counter)

      Method getCounterValue is used to retrieve a counter value.

      This method will fail if the underlying counter infrastructure is unavailable. See `isCounterStatusInitialized()`.
      Parameters:
        `group` - of type String
        `counter` - of type String

    - #### setStatus

          public abstract void setStatus(String status)

      Method setStatus is used to set the status of the current operation.

      This method will fail if the underlying counter infrastructure is unavailable. See `isCounterStatusInitialized()`.
      Parameters:
        `status` - of type String

    - #### isCounterStatusInitialized

          public abstract boolean isCounterStatusInitialized()

      Method isCounterStatusInitialized returns true if it is safe to increment a counter or set a status.
      Returns:
        boolean

    - #### openTapForRead

          public abstract TupleEntryIterator openTapForRead(Tap tap)
                                                     throws IOException

      Method openTapForRead return a `TupleEntryIterator` for the given Tap instance.

      Note the returned iterator will return the same instance of `TupleEntry` on every call, thus a copy must be made of either the TupleEntry or the underlying `Tuple` instance if they are to be stored in a Collection.
      Parameters:
        `tap` - of type Tap
      Returns:
        TupleIterator
      Throws:
        `IOException` - when there is a failure opening the resource

    - #### openTapForWrite

          public abstract TupleEntryCollector openTapForWrite(Tap tap)
                                                       throws IOException

      Method openTapForWrite returns a (@link TupleCollector} for the given Tap instance.
      Parameters:
        `tap` - of type Tap
      Returns:
        TupleCollector
      Throws:
        `IOException` - when there is a failure opening the resource

    - #### openTrapForWrite

          public abstract TupleEntryCollector openTrapForWrite(Tap trap)
                                                        throws IOException

      Method openTrapForWrite returns a (@link TupleCollector} for the given Tap instance.
      Parameters:
        `trap` - of type Tap
      Returns:
        TupleCollector
      Throws:
        `IOException` - when there is a failure opening the resource

    - #### openSystemIntermediateForWrite

          public abstract TupleEntryCollector openSystemIntermediateForWrite()
                                                                      throws IOException

      Throws:
        `IOException`

    - #### getConfig

          public abstract Config getConfig()

      Method getConfig returns the actual instance of the underlying configuration instance.

      This instance should not be modified or cached, see `getConfigCopy()` for a modifiable instance.
      Returns:
        an instance of the configuration

    - #### getConfigCopy

          public abstract Config getConfigCopy()

    - #### copyConfig

          public abstract <C> C copyConfig(C config)

    - #### diffConfigIntoMap

          public abstract <C> Map<String,String> diffConfigIntoMap(C defaultConfig,
                                                 C updatedConfig)

    - #### mergeMapIntoConfig

          public abstract Config mergeMapIntoConfig(Config defaultConfig,
                                  Map<String,String> map)

    - #### getTrapCollectorFor

          public TupleEntryCollector getTrapCollectorFor(Tap trap)

      Method getTrapCollectorFor will return a new `TupleEntryCollector` if one hasn't previously been created for the given trap Tap.
      Parameters:
        `trap` -
      Returns:
        TupleEntryCollector

    - #### getTrapCollectors

          protected Map<Tap,TupleEntryCollector> getTrapCollectors()

    - #### closeTrapCollectors

          public void closeTrapCollectors()

Copyright © 2007-2015 Concurrent, Inc. All Rights Reserved.