Package | Description |
---|---|
edu.georgetown.gucs.dictionary | |
edu.georgetown.gucs.fingerprinter | |
edu.georgetown.gucs.tokenizers |
Class and Description |
---|
TokenizerList
An ordered list of edu.georgetown.gucs.tokenizers objects to split a document into tokens and alter the tokens in various
ways.
|
Class and Description |
---|
TokenizerList
An ordered list of edu.georgetown.gucs.tokenizers objects to split a document into tokens and alter the tokens in various
ways.
|
Class and Description |
---|
FileTokenizer
Splits contents of a text file into tokens based on whitespace or by line.
|
Tokenizer |