edu.georgetown.gucs.tokenizers
Class LengthLessThanTokenizer

java.lang.Object
  extended by edu.georgetown.gucs.tokenizers.Tokenizer
      extended by edu.georgetown.gucs.tokenizers.LengthLessThanTokenizer

public class LengthLessThanTokenizer
extends Tokenizer


Field Summary
 
Fields inherited from class edu.georgetown.gucs.tokenizers.Tokenizer
constructor, tokenVector
 
Constructor Summary
LengthLessThanTokenizer(java.lang.String length)
           
 
Method Summary
 void tokenize(java.util.Iterator<java.lang.String> iterator)
          Alters or eliminates certain tokens.
 
Methods inherited from class edu.georgetown.gucs.tokenizers.Tokenizer
getConstructor, iterator, printTokens, tokenize
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

LengthLessThanTokenizer

public LengthLessThanTokenizer(java.lang.String length)
Method Detail

tokenize

public void tokenize(java.util.Iterator<java.lang.String> iterator)
Description copied from class: Tokenizer
Alters or eliminates certain tokens. By default, this method does nothing and must be over-ridden in each specific tokenizer.

Overrides:
tokenize in class Tokenizer
Parameters:
iterator - the string iterator over the token elements