## cascading.pipe.assembly
Class Unique

    java.lang.Object
      cascading.pipe.Pipe
          cascading.pipe.SubAssembly
              cascading.pipe.assembly.Unique

All Implemented Interfaces:
  FlowElement, Traceable, Serializable

    public class Unique

extends SubAssembly

Class Unique `SubAssembly` is used to filter all duplicates out of a tuple stream.

Typically finding unique value in a tuple stream relies on a `GroupBy` and a `FirstNBuffer` `Buffer` operation.

If the `include` value is set to `Unique.Include.NO_NULLS`, any tuple consisting of only `null` values will be removed from the stream.

This SubAssembly uses the `Unique.FilterPartialDuplicates` `Filter` to remove as many observed duplicates before the GroupBy operator to reduce IO over the network.

This strategy is similar to using `combiners`, except no sorting or serialization is invoked and results in a much simpler mechanism.

The `threshold` value tells the underlying FilterPartialDuplicates how many values to cache for duplicate comparison before dropping values from the LRU cache.

See Also:
  Serialized Form

Nested Class Summary
`static class` `Unique.Cache`
`static class` `Unique.FilterPartialDuplicates` Class FilterPartialDuplicates is a `Filter` that is used to remove observed duplicates from the tuple stream.
`static class` `Unique.Include`

Field Summary

Fields inherited from class cascading.pipe.Pipe
`configDef, name, parent, stepConfigDef`

Constructor Summary
`Unique(Pipe[] pipes, Fields uniqueFields)` Constructor Unique creates a new Unique instance.
`Unique(Pipe[] pipes, Fields uniqueFields, int threshold)` Constructor Unique creates a new Unique instance.
`Unique(Pipe[] pipes, Fields uniqueFields, Unique.Include include)` Constructor Unique creates a new Unique instance.
`Unique(Pipe[] pipes, Fields uniqueFields, Unique.Include include, int threshold)` Constructor Unique creates a new Unique instance.
`Unique(Pipe pipe, Fields uniqueFields)` Constructor Unique creates a new Unique instance.
`Unique(Pipe pipe, Fields uniqueFields, int threshold)` Constructor Unique creates a new Unique instance.
`Unique(Pipe pipe, Fields uniqueFields, Unique.Include include)` Constructor Unique creates a new Unique instance.
`Unique(Pipe pipe, Fields uniqueFields, Unique.Include include, int threshold)` Constructor Unique creates a new Unique instance.
`Unique(String name, Pipe[] pipes, Fields uniqueFields)` Constructor Unique creates a new Unique instance.
`Unique(String name, Pipe[] pipes, Fields uniqueFields, int threshold)` Constructor Unique creates a new Unique instance.
`Unique(String name, Pipe[] pipes, Fields uniqueFields, Unique.Include include)` Constructor Unique creates a new Unique instance.
`Unique(String name, Pipe[] pipes, Fields uniqueFields, Unique.Include include, int threshold)` Constructor Unique creates a new Unique instance.
`Unique(String name, Pipe pipe, Fields uniqueFields)` Constructor Unique creates a new Unique instance.
`Unique(String name, Pipe pipe, Fields uniqueFields, int threshold)` Constructor Unique creates a new Unique instance.
`Unique(String name, Pipe pipe, Fields uniqueFields, Unique.Include include)` Constructor Unique creates a new Unique instance.
`Unique(String name, Pipe pipe, Fields uniqueFields, Unique.Include include, int threshold)` Constructor Unique creates a new Unique instance.

Method Summary

Methods inherited from class cascading.pipe.SubAssembly
`getName, getPrevious, getTailNames, getTails, setPrevious, setTails, unwind`

Methods inherited from class cascading.pipe.Pipe
`equals, getConfigDef, getHeads, getParent, getStepConfigDef, getTrace, hasConfigDef, hashCode, hasStepConfigDef, id, isEquivalentTo, named, names, outgoingScopeFor, pipes, print, printInternal, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, setParent, toString`

Methods inherited from class java.lang.Object
`clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

Constructor Detail

### Unique

    @ConstructorProperties(value={"pipe","uniqueFields"})
    public Unique(Pipe pipe,
                                             Fields uniqueFields)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `pipe` - of type Pipe
    `uniqueFields` - of type Fields

### Unique

    @ConstructorProperties(value={"pipe","uniqueFields","include"})
    public Unique(Pipe pipe,
                                             Fields uniqueFields,
                                             Unique.Include include)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `pipe` - of type Pipe
    `uniqueFields` - of type Fields
    `include` - of type Include

### Unique

    @ConstructorProperties(value={"pipe","uniqueFields","threshold"})
    public Unique(Pipe pipe,
                                             Fields uniqueFields,
                                             int threshold)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `pipe` - of type Pipe
    `uniqueFields` - of type Fields
    `threshold` - of type int

### Unique

    @ConstructorProperties(value={"pipe","uniqueFields","include","threshold"})
    public Unique(Pipe pipe,
                                             Fields uniqueFields,
                                             Unique.Include include,
                                             int threshold)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `pipe` - of type Pipe
    `uniqueFields` - of type Fields
    `include` - of type Include
    `threshold` - of type int

### Unique

    @ConstructorProperties(value={"name","pipe","uniqueFields"})
    public Unique(String name,
                                             Pipe pipe,
                                             Fields uniqueFields)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `name` - of type String
    `pipe` - of type Pipe
    `uniqueFields` - of type Fields

### Unique

    @ConstructorProperties(value={"name","pipe","uniqueFields","include"})
    public Unique(String name,
                                             Pipe pipe,
                                             Fields uniqueFields,
                                             Unique.Include include)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `name` - of type String
    `pipe` - of type Pipe
    `uniqueFields` - of type Fields
    `include` - of type Include

### Unique

    @ConstructorProperties(value={"name","pipe","uniqueFields","threshold"})
    public Unique(String name,
                                             Pipe pipe,
                                             Fields uniqueFields,
                                             int threshold)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `name` - of type String
    `pipe` - of type Pipe
    `uniqueFields` - of type Fields
    `threshold` - of type int

### Unique

    @ConstructorProperties(value={"name","pipe","uniqueFields","include","threshold"})
    public Unique(String name,
                                             Pipe pipe,
                                             Fields uniqueFields,
                                             Unique.Include include,
                                             int threshold)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `name` - of type String
    `pipe` - of type Pipe
    `uniqueFields` - of type Fields
    `include` - of type Include
    `threshold` - of type int

### Unique

    @ConstructorProperties(value={"pipes","uniqueFields"})
    public Unique(Pipe[] pipes,
                                             Fields uniqueFields)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `pipes` - of type Pipe[]
    `uniqueFields` - of type Fields

### Unique

    @ConstructorProperties(value={"pipes","uniqueFields","include"})
    public Unique(Pipe[] pipes,
                                             Fields uniqueFields,
                                             Unique.Include include)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `pipes` - of type Pipe[]
    `uniqueFields` - of type Fields
    `include` - of type Include

### Unique

    @ConstructorProperties(value={"pipes","uniqueFields","threshold"})
    public Unique(Pipe[] pipes,
                                             Fields uniqueFields,
                                             int threshold)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `pipes` - of type Pipe[]
    `uniqueFields` - of type Fields
    `threshold` - of type int

### Unique

    @ConstructorProperties(value={"pipes","uniqueFields","include","threshold"})
    public Unique(Pipe[] pipes,
                                             Fields uniqueFields,
                                             Unique.Include include,
                                             int threshold)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `pipes` - of type Pipe[]
    `uniqueFields` - of type Fields
    `include` - of type Include
    `threshold` - of type int

### Unique

    @ConstructorProperties(value={"name","pipes","uniqueFields"})
    public Unique(String name,
                                             Pipe[] pipes,
                                             Fields uniqueFields)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `name` - of type String
    `pipes` - of type Pipe[]
    `uniqueFields` - of type Fields

### Unique

    @ConstructorProperties(value={"name","pipes","uniqueFields","include"})
    public Unique(String name,
                                             Pipe[] pipes,
                                             Fields uniqueFields,
                                             Unique.Include include)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `name` - of type String
    `pipes` - of type Pipe[]
    `uniqueFields` - of type Fields
    `include` - of type Include

### Unique

    @ConstructorProperties(value={"name","pipes","uniqueFields","threshold"})
    public Unique(String name,
                                             Pipe[] pipes,
                                             Fields uniqueFields,
                                             int threshold)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `name` - of type String
    `pipes` - of type Pipe[]
    `uniqueFields` - of type Fields
    `threshold` - of type int

### Unique

    @ConstructorProperties(value={"name","pipes","uniqueFields","include","threshold"})
    public Unique(String name,
                                             Pipe[] pipes,
                                             Fields uniqueFields,
                                             Unique.Include include,
                                             int threshold)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `name` - of type String
    `pipes` - of type Pipe[]
    `uniqueFields` - of type Fields
    `threshold` - of type int

Copyright © 2007-2014 Concurrent, Inc. All Rights Reserved.