cascading.tap.hadoop

## Class HfsProps

- java.lang.Object
- - cascading.property.Props
  - - cascading.tap.hadoop.HfsProps

-     public class HfsProps
      extends Props

  Class HfsProps is a fluent helper for setting various Hadoop FS level properties that some `Flow` may or may not be required to have set. These properties are typically passed to a Flow via a `FlowConnector`.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static String` | `COMBINE_INPUT_FILES` Field COMBINE_INPUT_FILES
    `static String` | `COMBINE_INPUT_FILES_SAFE_MODE` Field COMBINE_INPUT_FILES_SAFEMODE
    `static String` | `COMBINE_INPUT_FILES_SIZE_MAX` Field COMBINE_INPUT_FILES_SIZE_MAX
    `protected Long` | `combinedInputMaxSize`
    `protected Boolean` | `combinedInputSafeMode`
    `static String` | `LOCAL_MODE_SCHEME` Fields LOCAL_MODE_SCHEME *
    `protected String` | `localModeScheme`
    `static String` | `TEMPORARY_DIRECTORY` Field TEMPORARY_DIRECTORY
    `protected String` | `temporaryDirectory`
    `protected Boolean` | `useCombinedInput`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `HfsProps()`

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `protected void` | `addPropertiesTo(Properties properties)`
    `Long` | `getCombinedInputMaxSize()`
    `String` | `getLocalModeScheme()`
    `String` | `getTemporaryDirectory()`
    `static HfsProps` | `hfsProps()` Creates a new HfsProps instance.
    `boolean` | `isUseCombinedInput()`
    `boolean` | `isUseCombinedInputSafeMode()`
    `HfsProps` | `setCombinedInputMaxSize(long combinedInputMaxSize)` Method setCombinedInputMaxSize sets the maximum input split size to be used.
    `static void` | `setCombinedInputMaxSize(Map<Object,Object> properties, Long size)` Method setCombinedInputMaxSize sets the maximum input split size to be used.
    `static void` | `setLocalModeScheme(Map<Object,Object> properties, String scheme)` Method setLocalModeScheme provides a means to change the scheme value used to detect when a MapReduce job should be run in Hadoop local mode.
    `HfsProps` | `setLocalModeScheme(String localModeScheme)` Method setLocalModeScheme provides a means to change the scheme value used to detect when a MapReduce job should be run in Hadoop local mode.
    `static void` | `setTemporaryDirectory(Map<Object,Object> properties, String temporaryDirectory)` Method setTemporaryDirectory sets the temporary directory on the given properties object.
    `HfsProps` | `setTemporaryDirectory(String temporaryDirectory)` Method setTemporaryDirectory sets the temporary directory for use on the underlying filesystem.
    `HfsProps` | `setUseCombinedInput(boolean useCombinedInput)` Method setUseCombinedInput provides a means to indicate whether to leverage `CombineFileInputFormat` for the input format.
    `static void` | `setUseCombinedInput(Map<Object,Object> properties, Boolean combine)` Method setUseCombinedInput provides a means to indicate whether to leverage `CombineFileInputFormat` for the input format.
    `HfsProps` | `setUseCombinedInputSafeMode(boolean combinedInputSafeMode)` Method setUseCombinedInputSafeMode toggles safe mode when using `CombineFileInputFormat`.
    `static void` | `setUseCombinedInputSafeMode(Map<Object,Object> properties, Boolean safeMode)` Method setUseCombinedInputSafeMode toggles safe mode when using `CombineFileInputFormat`.

    - ### Methods inherited from class cascading.property.Props

      `buildProperties, buildProperties, buildProperties, buildProperties, setProperties, setProperties`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### TEMPORARY_DIRECTORY

          public static final String TEMPORARY_DIRECTORY

      Field TEMPORARY_DIRECTORY
      See Also:
        Constant Field Values

    - #### LOCAL_MODE_SCHEME

          public static final String LOCAL_MODE_SCHEME

      Fields LOCAL_MODE_SCHEME *
      See Also:
        Constant Field Values

    - #### COMBINE_INPUT_FILES

          public static final String COMBINE_INPUT_FILES

      Field COMBINE_INPUT_FILES
      See Also:
        Constant Field Values

    - #### COMBINE_INPUT_FILES_SAFE_MODE

          public static final String COMBINE_INPUT_FILES_SAFE_MODE

      Field COMBINE_INPUT_FILES_SAFEMODE
      See Also:
        Constant Field Values

    - #### COMBINE_INPUT_FILES_SIZE_MAX

          public static final String COMBINE_INPUT_FILES_SIZE_MAX

      Field COMBINE_INPUT_FILES_SIZE_MAX
      See Also:
        Constant Field Values

    - #### temporaryDirectory

          protected String temporaryDirectory

    - #### localModeScheme

          protected String localModeScheme

    - #### useCombinedInput

          protected Boolean useCombinedInput

    - #### combinedInputMaxSize

          protected Long combinedInputMaxSize

    - #### combinedInputSafeMode

          protected Boolean combinedInputSafeMode

  - ### Constructor Detail

    - #### HfsProps

          public HfsProps()

  - ### Method Detail

    - #### setTemporaryDirectory

          public static void setTemporaryDirectory(Map<Object,Object> properties,
                                   String temporaryDirectory)

      Method setTemporaryDirectory sets the temporary directory on the given properties object.
      Parameters:
        `properties` - of type Map
        `temporaryDirectory` - of type String

    - #### setLocalModeScheme

          public static void setLocalModeScheme(Map<Object,Object> properties,
                                String scheme)

      Method setLocalModeScheme provides a means to change the scheme value used to detect when a MapReduce job should be run in Hadoop local mode. By default the value is `"file"`, set to `"none"` to disable entirely.
      Parameters:
        `properties` - of type Map
        `scheme` - a String

    - #### setUseCombinedInput

          public static void setUseCombinedInput(Map<Object,Object> properties,
                                 Boolean combine)

      Method setUseCombinedInput provides a means to indicate whether to leverage `CombineFileInputFormat` for the input format. By default it is false.

      Use `setCombinedInputMaxSize(long)` to set the max split/combined input size. Other specific properties must be specified directly if needed. Specifically "mapred.min.split.size.per.node" and "mapred.min.split.size.per.rack", which are 0 by default.
      Parameters:
        `properties` - of type Map
        `combine` - a boolean

    - #### setUseCombinedInputSafeMode

          public static void setUseCombinedInputSafeMode(Map<Object,Object> properties,
                                         Boolean safeMode)

      Method setUseCombinedInputSafeMode toggles safe mode when using `CombineFileInputFormat`. Safe mode will throw an exception if the underlying InputFormat is not of type `FileInputFormat`. If safeMode is off a warning will be logged instead. safeMode is on by default.

      Setting this property when not setting `setUseCombinedInput(boolean)` to true has no effect.
      Parameters:
        `properties` - of type Map
        `safeMode` - a boolean

    - #### setCombinedInputMaxSize

          public static void setCombinedInputMaxSize(Map<Object,Object> properties,
                                     Long size)

      Method setCombinedInputMaxSize sets the maximum input split size to be used.

      This property is an alias for the Hadoop property "mapred.max.split.size".
      Parameters:
        `properties` - of type Map
        `size` - of type long

    - #### hfsProps

          public static HfsProps hfsProps()

      Creates a new HfsProps instance.
      Returns:
        HfsProps instance

    - #### getTemporaryDirectory

          public String getTemporaryDirectory()

    - #### setTemporaryDirectory

          public HfsProps setTemporaryDirectory(String temporaryDirectory)

      Method setTemporaryDirectory sets the temporary directory for use on the underlying filesystem.
      Parameters:
        `temporaryDirectory` - of type String
      Returns:
        returns this instance

    - #### getLocalModeScheme

          public String getLocalModeScheme()

    - #### setLocalModeScheme

          public HfsProps setLocalModeScheme(String localModeScheme)

      Method setLocalModeScheme provides a means to change the scheme value used to detect when a MapReduce job should be run in Hadoop local mode. By default the value is `"file"`, set to `"none"` to disable entirely.
      Parameters:
        `localModeScheme` - of type String
      Returns:
        returns this instance

    - #### isUseCombinedInput

          public boolean isUseCombinedInput()

    - #### setUseCombinedInput

          public HfsProps setUseCombinedInput(boolean useCombinedInput)

      Method setUseCombinedInput provides a means to indicate whether to leverage `CombineFileInputFormat` for the input format. By default it is false.
      Parameters:
        `useCombinedInput` - boolean
      Returns:
        returns this instance

    - #### getCombinedInputMaxSize

          public Long getCombinedInputMaxSize()

    - #### setCombinedInputMaxSize

          public HfsProps setCombinedInputMaxSize(long combinedInputMaxSize)

      Method setCombinedInputMaxSize sets the maximum input split size to be used.

      This value is not honored unless `setUseCombinedInput(boolean)` is `true`.
      Parameters:
        `combinedInputMaxSize` - of type long
      Returns:
        returns this instance

    - #### isUseCombinedInputSafeMode

          public boolean isUseCombinedInputSafeMode()

    - #### setUseCombinedInputSafeMode

          public HfsProps setUseCombinedInputSafeMode(boolean combinedInputSafeMode)

      Method setUseCombinedInputSafeMode toggles safe mode when using `CombineFileInputFormat`. Safe mode will throw an exception if the underlying InputFormat is not of type `FileInputFormat`. If safeMode is off a warning will be logged instead. safeMode is on by default.

      Setting this property when not setting `setUseCombinedInput(boolean)` to true has no effect.
      Parameters:
        `combinedInputSafeMode` - boolean
      Returns:
        returns this instance

    - #### addPropertiesTo

          protected void addPropertiesTo(Properties properties)

      Specified by:
        `addPropertiesTo` in class `Props`

Copyright © 2007-2015 Concurrent, Inc. All Rights Reserved.