Package weka.core.tokenizers

Class AlphabeticTokenizer

  • All Implemented Interfaces:
    java.io.Serializable, java.util.Enumeration, OptionHandler, RevisionHandler
    public class AlphabeticTokenizer
extends Tokenizer
    Alphabetic string tokenizer, tokens are to be formed only from contiguous alphabetic sequences.

    Version:
    $Revision: 1.2 $
    Author:
    Asrhaf M. Kibriya (amk14@cs.waikato.ac.nz), FracPete (fracpete at waikato dot ac dot nz)
    See Also:
    Serialized Form

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      java.lang.String getRevision()
      Returns the revision string.
      java.lang.String globalInfo()
      Returns a string describing the stemmer
      boolean hasMoreElements()
      returns whether there are more elements still
      static void main​(java.lang.String[] args)
      Runs the tokenizer with the given options and strings to tokenize.
      java.lang.Object nextElement()
      returns the next element
      void tokenize​(java.lang.String s)
      Sets the string to tokenize.

      • Methods inherited from class java.lang.Object

        equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

      • Methods inherited from interface java.util.Enumeration

        asIterator

    • Constructor Detail

      • AlphabeticTokenizer

        public AlphabeticTokenizer()

    • Method Detail

      • globalInfo

        public java.lang.String globalInfo()
        Returns a string describing the stemmer
        Specified by:
        globalInfo in class Tokenizer
        Returns:
        a description suitable for displaying in the explorer/experimenter gui

      • hasMoreElements

        public boolean hasMoreElements()
        returns whether there are more elements still
        Specified by:
        hasMoreElements in interface java.util.Enumeration
        Specified by:
        hasMoreElements in class Tokenizer
        Returns:
        true if there are still more elements

      • nextElement

        public java.lang.Object nextElement()
        returns the next element
        Specified by:
        nextElement in interface java.util.Enumeration
        Specified by:
        nextElement in class Tokenizer
        Returns:
        the next element

      • tokenize

        public void tokenize​(java.lang.String s)
        Sets the string to tokenize. Tokenization happens immediately.
        Specified by:
        tokenize in class Tokenizer
        Parameters:
        s - the string to tokenize

      • getRevision

        public java.lang.String getRevision()
        Returns the revision string.
        Returns:
        the revision

      • main

        public static void main​(java.lang.String[] args)
        Runs the tokenizer with the given options and strings to tokenize. The tokens are printed to stdout.
        Parameters:
        args - the commandline options and strings to tokenize