Main module

LogParser

class log2seq.LogParser(header_parsers, statement_parser, ignore_failure=False)

Log parser object.

LogParser in log2seq consists of two different parsers: HeaderParser and StatementParser.

Parsed results are returned in one dict object. It consists of following parsed information.

  • Header information (value_name as key)
  • Statement part in string format (“message” as key)
  • Segmented words in statement part (“words” as key)
  • Separator symbols in ststement part (“symbols” as key)

Example

>>> mes = "Jan  1 12:34:56 host-device1 system[12345]: host 2001:0db8:1234::1 (interface:eth0) disconnected"
>>> import log2seq
>>> parser = log2seq.init_parser()  # get default LogParser
>>> parsed_line = parser.process_line(mes)
>>> parsed_line["timestamp"]  # timestamp parsed by HeaderParser
datetime.datetime(2020, 1, 1, 12, 34, 56)
>>> parsed_line["host"]  # Hostname item in HeaderParser
'host-device1'
>>> parsed_line["message"]  # Statement part parsed by HeaderParser
'system[12345]: host 2001:0db8:1234::1 (interface:eth0) disconnected'
>>> parsed_line["words"]  # Segmented words parsed by StatementParser
['system', '12345', 'host', '2001:0db8:1234::1', 'interface', 'eth0', 'disconnected']
>>> parsed_line["symbols"]  # Separator symbols parsed by StatementParser
['', '[', ']: ', ' ', ' (', ':', ') ', '']

You can specify multiple HeaderParser as input. If so, LogParser try to parse a log message with them in order, and the first matched rule is used for the message.

Parameters:
  • header_parsers (HeaderParser or list of it) – one or multiple HeaderParser instance to use.
  • statement_parser (StatementParser) – one StatementParser instance to use.
  • ignore_failure (bool, optional) – If true, ignore non-matching lines with given header parsers.
process_header(line, verbose=False)

Parse header part in a log message.

This function uses all given HeaderParser rules. If all HeaderParser fails to match the input log message, it raises a LogParseFailure exception.

Parameters:
  • line (str) – A log message.
  • verbose (bool, optional) – Show intermediate progress of applying header rules.
Returns:

parsed header data.
Return type:

dict
process_line(line, verbose=False)

Parse a log message (i.e., a line).

If all HeaderParser fails to match the input log message, it raises a LogParserFailure exception.

Parameters:
  • line (str) – A log message. Line feed code will be removed.
  • verbose (bool, optional) – Show intermediate progress of applying rules.
Returns:

parsed data.
Return type:

dict
process_statement(statement, verbose=False)

Parse a log statement.

Parameters:
  • statement (str) – Statement part in a log message.
  • verbose (bool, optional) – Show intermediate progress of applying rules.
Returns:

List of two components: words and symbols. See statement.StatementParser.process_line().
Return type:

tuple
log2seq.init_parser(header_parsers=None, statement_parser=None)

Generate LogParser object.

If no arguments are given, this function generates LogParser with default configurations.

Parameters:

Exceptions

class log2seq.ParserDefinitionError

ParserDefinitionError is raised when the given rules are inappropriate (e.g., having syntax errors).

class log2seq.LogParseFailure

LogParserFailure is raised when the input log message not matched with all given HeaderParser rules.

If you want to pass such mismatching log messages, use try-except with this exception.