edu.georgetown.gucs.tokenizers
Class LengthAtLeastTokenizer

java.lang.Object
  extended by edu.georgetown.gucs.tokenizers.Tokenizer
      extended by edu.georgetown.gucs.tokenizers.LengthAtLeastTokenizer

public class LengthAtLeastTokenizer
extends Tokenizer


Field Summary
 
Fields inherited from class edu.georgetown.gucs.tokenizers.Tokenizer
constructor, tokenVector
 
Constructor Summary
LengthAtLeastTokenizer(java.lang.String length)
          Constructor that sets the minimum token length to be considered
 
Method Summary
 void tokenize(java.util.Iterator<java.lang.String> iterator)
          Alters or eliminates certain tokens.
 
Methods inherited from class edu.georgetown.gucs.tokenizers.Tokenizer
getConstructor, iterator, printTokens, tokenize
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

LengthAtLeastTokenizer

public LengthAtLeastTokenizer(java.lang.String length)
Constructor that sets the minimum token length to be considered

Parameters:
length - the string value of the minimum token length to be included
Method Detail

tokenize

public void tokenize(java.util.Iterator<java.lang.String> iterator)
Description copied from class: Tokenizer
Alters or eliminates certain tokens. By default, this method does nothing and must be over-ridden in each specific tokenizer.

Overrides:
tokenize in class Tokenizer
Parameters:
iterator - the string iterator over the token elements