## cascading.pipe.assembly
Class Unique

    java.lang.Object
      cascading.pipe.Pipe
          cascading.pipe.SubAssembly
              cascading.pipe.assembly.Unique

All Implemented Interfaces:
  FlowElement, Serializable

    public class Unique

extends SubAssembly

Class Unique `SubAssembly` is used to filter all duplicates out of a tuple stream.

Typically finding unique value in a tuple stream relies on a `GroupBy` and a `First` `Aggregator` operation.

This SubAssembly also uses the `Unique.FilterPartialDuplicates` `Filter` to remove as many observed duplicates before the GroupBy operator to reduce IO over the network.

This strategy is similar to using `combiners`, except no sorting or serialization is invoked and results in a much simpler mechanism.

The `threshold` value tells the underlying FilterPartialDuplicates how many values to cache for duplicate comparison before dropping values from the LRU cache.

See Also:
  Serialized Form

Nested Class Summary
`static class` `Unique.FilterPartialDuplicates` Class FilterPartialDuplicates is a `Filter` that is used to remove observed duplicates from the tuple stream.

Field Summary

Fields inherited from class cascading.pipe.Pipe
`previous`

Constructor Summary
`Unique(Pipe[] pipes, Fields groupingFields)` Constructor Unique creates a new Unique instance.
`Unique(Pipe[] pipes, Fields groupingFields, int threshold)` Constructor Unique creates a new Unique instance.
`Unique(Pipe pipe, Fields groupingFields)` Constructor Unique creates a new Unique instance.
`Unique(Pipe pipe, Fields groupingFields, int threshold)` Constructor Unique creates a new Unique instance.
`Unique(String name, Pipe[] pipes, Fields groupingFields)` Constructor Unique creates a new Unique instance.
`Unique(String name, Pipe[] pipes, Fields groupingFields, int threshold)` Constructor Unique creates a new Unique instance.
`Unique(String name, Pipe pipe, Fields groupingFields)` Constructor Unique creates a new Unique instance.
`Unique(String name, Pipe pipe, Fields groupingFields, int threshold)` Constructor Unique creates a new Unique instance.

Method Summary

Methods inherited from class cascading.pipe.SubAssembly
`getName, getPrevious, getTailNames, getTails, setTails, unwind`

Methods inherited from class cascading.pipe.Pipe
`equals, getHeads, getTrace, hashCode, isEquivalentTo, names, outgoingScopeFor, pipes, print, printInternal, resolveFields, resolveIncomingOperationFields, toString`

Methods inherited from class java.lang.Object
`clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

Constructor Detail

### Unique

    @ConstructorProperties(value={"pipe","groupingFields"})
    public Unique(Pipe pipe,
                                             Fields groupingFields)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `pipe` - of type Pipe
    `groupingFields` - of type Fields

### Unique

    @ConstructorProperties(value={"pipe","groupingFields","threshold"})
    public Unique(Pipe pipe,
                                             Fields groupingFields,
                                             int threshold)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `pipe` - of type Pipe
    `groupingFields` - of type Fields
    `threshold` - of type int

### Unique

    @ConstructorProperties(value={"name","pipe","groupingFields"})
    public Unique(String name,
                                             Pipe pipe,
                                             Fields groupingFields)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `name` - of type String
    `pipe` - of type Pipe
    `groupingFields` - of type Fields

### Unique

    @ConstructorProperties(value={"name","pipe","groupingFields","threshold"})
    public Unique(String name,
                                             Pipe pipe,
                                             Fields groupingFields,
                                             int threshold)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `name` - of type String
    `pipe` - of type Pipe
    `groupingFields` - of type Fields
    `threshold` - of type int

### Unique

    @ConstructorProperties(value={"pipes","groupingFields"})
    public Unique(Pipe[] pipes,
                                             Fields groupingFields)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `pipes` - of type Pipe[]
    `groupingFields` - of type Fields

### Unique

    @ConstructorProperties(value={"pipes","groupingFields","threshold"})
    public Unique(Pipe[] pipes,
                                             Fields groupingFields,
                                             int threshold)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `pipes` - of type Pipe[]
    `groupingFields` - of type Fields
    `threshold` - of type int

### Unique

    @ConstructorProperties(value={"name","pipes","groupingFields"})
    public Unique(String name,
                                             Pipe[] pipes,
                                             Fields groupingFields)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `name` - of type String
    `pipes` - of type Pipe[]
    `groupingFields` - of type Fields

### Unique

    @ConstructorProperties(value={"name","pipes","groupingFields","threshold"})
    public Unique(String name,
                                             Pipe[] pipes,
                                             Fields groupingFields,
                                             int threshold)

  Constructor Unique creates a new Unique instance.

  Parameters:
    `name` - of type String
    `pipes` - of type Pipe[]
    `groupingFields` - of type Fields
    `threshold` - of type int

Copyright © 2007-2010 Concurrent, Inc. All Rights Reserved.