Field Detail

• defaultMaxDocFreqPercent

  public static final float defaultMaxDocFreqPercent

  See Also:

  Constant Field Values

Constructor Detail

• QueryAutoStopWordAnalyzer

  public QueryAutoStopWordAnalyzer(Analyzer delegate)

  Deprecated. Use QueryAutoStopWordAnalyzer(Version, Analyzer) instead
  Initializes this analyzer with the Analyzer object that actually produces the tokens

  Parameters:

  delegate - The choice of Analyzer that is used to produce the token stream which needs filtering

• QueryAutoStopWordAnalyzer

  public QueryAutoStopWordAnalyzer(Version matchVersion,
                                   Analyzer delegate)

  Initializes this analyzer with the Analyzer object that actually produces the tokens

  Parameters:

  delegate - The choice of Analyzer that is used to produce the token stream which needs filtering

Method Detail

• addStopWords

  public int addStopWords(IndexReader reader)
                   throws java.io.IOException

  Automatically adds stop words for all fields with terms exceeding the defaultMaxDocFreqPercent

  Parameters:

  reader - The IndexReader which will be consulted to identify potential stop words that exceed the required document frequency

  Returns:

  The number of stop words identified.

  Throws:

  java.io.IOException

• addStopWords

  public int addStopWords(IndexReader reader,
                          int maxDocFreq)
                   throws java.io.IOException

  Automatically adds stop words for all fields with terms exceeding the maxDocFreqPercent

  Parameters:

  reader - The IndexReader which will be consulted to identify potential stop words that exceed the required document frequency

  maxDocFreq - The maximum number of index documents which can contain a term, after which the term is considered to be a stop word

  Returns:

  The number of stop words identified.

  Throws:

  java.io.IOException

• addStopWords

  public int addStopWords(IndexReader reader,
                          float maxPercentDocs)
                   throws java.io.IOException

  Automatically adds stop words for all fields with terms exceeding the maxDocFreqPercent

  Parameters:

  reader - The IndexReader which will be consulted to identify potential stop words that exceed the required document frequency

  maxPercentDocs - The maximum percentage (between 0.0 and 1.0) of index documents which contain a term, after which the word is considered to be a stop word.

  Returns:

  The number of stop words identified.

  Throws:

  java.io.IOException

• addStopWords

  public int addStopWords(IndexReader reader,
                          java.lang.String fieldName,
                          float maxPercentDocs)
                   throws java.io.IOException

  Automatically adds stop words for the given field with terms exceeding the maxPercentDocs

  Parameters:

  reader - The IndexReader which will be consulted to identify potential stop words that exceed the required document frequency

  fieldName - The field for which stopwords will be added

  maxPercentDocs - The maximum percentage (between 0.0 and 1.0) of index documents which contain a term, after which the word is considered to be a stop word.

  Returns:

  The number of stop words identified.

  Throws:

  java.io.IOException

• addStopWords

  public int addStopWords(IndexReader reader,
                          java.lang.String fieldName,
                          int maxDocFreq)
                   throws java.io.IOException

  Automatically adds stop words for the given field with terms exceeding the maxPercentDocs

  Parameters:

  reader - The IndexReader which will be consulted to identify potential stop words that exceed the required document frequency

  fieldName - The field for which stopwords will be added

  maxDocFreq - The maximum number of index documents which can contain a term, after which the term is considered to be a stop word.

  Returns:

  The number of stop words identified.

  Throws:

  java.io.IOException

• tokenStream

  public TokenStream tokenStream(java.lang.String fieldName,
                                 java.io.Reader reader)

  Description copied from class: Analyzer
  Creates a TokenStream which tokenizes all the text in the provided Reader. Must be able to handle null field name for backward compatibility.

  Specified by:

  tokenStream in class Analyzer

• reusableTokenStream

  public TokenStream reusableTokenStream(java.lang.String fieldName,
                                         java.io.Reader reader)
                                  throws java.io.IOException

  Description copied from class: Analyzer
  Creates a TokenStream that is allowed to be re-used from the previous time that the same thread called this method. Callers that do not need to use more than one TokenStream at the same time from this analyzer should use this method for better performance.

  Overrides:

  reusableTokenStream in class Analyzer

  Throws:

  java.io.IOException

• getStopWords

  public java.lang.String[] getStopWords(java.lang.String fieldName)

  Provides information on which stop words have been identified for a field

  Parameters:

  fieldName - The field for which stop words identified in "addStopWords" method calls will be returned

  Returns:

  the stop words identified for a field

• getStopWords

  public Term[] getStopWords()

  Provides information on which stop words have been identified for all fields

  Returns:

  the stop words (as terms)