public class RemoveNumericTokensTokenizer extends Tokenizer
tokenVectorMap
Constructor and Description |
---|
RemoveNumericTokensTokenizer() |
Modifier and Type | Method and Description |
---|---|
java.util.List<Token> |
tokenize(java.util.List<Token> tokens)
Eliminates tokens that are numbers
|
getTokenVectorMap, iterator, printTokens, tokenize, toString