Home       Trees       Indices       Help   
pylogsparser
Package logsparser :: Module lognormalizer :: Class LogNormalizer
[frames] | no frames]

Class LogNormalizer

source code

Basic normalization flow manager.
Normalizers definitions are loaded from a path and checked against the DTD.
If the definitions are syntactically correct, the normalizers are
instantiated and populate the manager's cache.
Normalization priormority is established as follows:

* Maximum priority assigned to normalizers where the "appliedTo" tag is set
  to "raw". They MUST be mutually exclusive.
* Medium priority assigned to normalizers where the "appliedTo" tag is set
  to "body".
* Lowest priority assigned to any remaining normalizers.

Some extra treatment is also done prior and after the log normalization:

* Assignment of a unique ID, under the tag "uuid"
* Conversion of date tags to UTC, if the "_timezone" was set prior to
  the normalization process.
Instance Methods
 
__init__(self, normalizers_paths, active_normalizers={})
Instantiates a flow manager.		 source code
 
reload(self)
Refreshes this instance's normalizers pool.		 source code
 
iter_normalizer(self)
Iterates through normalizers and returns the normalizers' paths.		 source code
 
__len__(self)
Returns the amount of available normalizers.		 source code
 
update_normalizer(self, raw_xml_contents, name=None, dir_path=None)
used to add or update a normalizer.		 source code
 
get_normalizer_by_uuid(self, uuid)
Returns normalizer by uuid.		 source code
 
get_normalizer_source(self, uuid)
Returns the raw XML source of normalizer uuid.		 source code
 
get_normalizer_path(self, uuid)
Returns the filesystem path of a normalizer.		 source code
 
activate_normalizers(self)
Activates normalizers according to what was set by calling set_active_normalizers.		 source code
 
get_active_normalizers(self)
Returns a dictionary of normalizers; keys are normalizers' uuid and values are True|False according to the normalizer's activation state.		 source code
 
set_active_normalizers(self, norms={})
Sets the active/inactive normalizers.		 source code
 
lognormalize(self, data)
This method is the entry point to normalize data (a log).		 source code
 
uuidify(self, log)
Adds a unique UID to the normalized log.		 source code
 
normalize(self, log)
plain normalization.		 source code
Method Details

__init__(self, normalizers_paths, active_normalizers={})
(Constructor)

source code 

Instantiates a flow manager. The default behavior is to activate every available normalizer.

Parameters:
  • normalizers_paths - a list of absolute paths to the normalizer XML definitions to use or a just a single path as str.
  • active_normalizers - a dictionary of active normalizers in the form {name: [True|False]}.

iter_normalizer(self)

source code 

Iterates through normalizers and returns the normalizers' paths.

Returns:
a generator of absolute paths.

update_normalizer(self, raw_xml_contents, name=None, dir_path=None)

source code 

used to add or update a normalizer.

Parameters:
  • raw_xml_contents - XML description of normalizer as flat XML. It must comply to the DTD.
  • name - if set, the XML description will be saved as name.xml. If left blank, name will be fetched from the XML description.
  • dir_path - the path to the directory where to copy the given normalizer.

activate_normalizers(self)

source code 

Activates normalizers according to what was set by calling set_active_normalizers. If no call to the latter function has been made so far, this method activates every normalizer.

set_active_normalizers(self, norms={})

source code 

Sets the active/inactive normalizers. Default behavior is to deactivate every normalizer.

Parameters:
  • norms - a dictionary, similar to the one returned by get_active_normalizers.

lognormalize(self, data)

source code  
This method is the entry point to normalize data (a log).

data is passed through every activated normalizer
and extra tagging occurs accordingly.

data receives also an extra uuid tag.

@param data: must be a dictionary with at least a key 'raw' or 'body'
             with BaseString values (preferably Unicode).

Here an example :
>>> from logsparser import lognormalizer
>>> from pprint import pprint
>>> ln = lognormalizer.LogNormalizer('/usr/local/share/normalizers/')
>>> mylog = {'raw' : 'Jul 18 15:35:01 zoo /USR/SBIN/CRON[14338]: (root) CMD (/srv/git/redmine-changesets.sh)'}
>>> ln.lognormalize(mylog)
>>> pprint mylog
{'body': '(root) CMD (/srv/git/redmine-changesets.sh)',
'date': datetime.datetime(2011, 7, 18, 15, 35, 1),
'pid': '14338',
'program': '/USR/SBIN/CRON',
'raw': 'Jul 18 15:35:01 zoo /USR/SBIN/CRON[14338]: (root) CMD (/srv/git/redmine-changesets.sh)',
'source': 'zoo',
'uuid': 70851882840934161193887647073096992594L}

   Home       Trees       Indices       Help   
pylogsparser
Generated by Epydoc 3.0.1 on Wed May 2 00:04:11 2012 http://epydoc.sourceforge.net