Skip to main content

parse_log

Parses common log formats into structured data. This is easier and often much faster than grok.

# Common config fields, showing default values
label: ""
parse_log:
  format: syslog_rfc5424
  codec: json

Fields​

format​

A common log format to parse.

Type: string
Default: "syslog_rfc5424"
Options: syslog_rfc5424, syslog_rfc3164.

codec​

Specifies the structured format to parse a log into.

Type: string
Default: "json"
Options: json.

best_effort​

Still returns partially parsed messages even if an error occurs.

Type: bool
Default: true

allow_rfc3339​

Also accept timestamps in rfc3339 format while parsing. Applicable to format syslog_rfc3164.

Type: bool
Default: true

default_year​

Sets the strategy used to set the year for rfc3164 timestamps. Applicable to format syslog_rfc3164. When set to current the current year will be set, when set to an integer that value will be used. Leave this field empty to not set a default year at all.

Type: string
Default: "current"

default_timezone​

Sets the strategy to decide the timezone for rfc3164 timestamps. Applicable to format syslog_rfc3164. This value should follow the time.LoadLocation format.

Type: string
Default: "UTC"

parts​

An optional array of message indexes of a batch that the processor should apply to. If left empty all messages are processed. This field is only applicable when batching messages at the input level.

Indexes can be negative, and if so the part will be selected from the end counting backwards starting from -1.

Type: array
Default: []

Codecs​

Currently the only supported structured data codec is json.

Formats​

syslog_rfc5424​

Attempts to parse a log following the Syslog rfc5424 spec. The resulting structured document may contain any of the following fields:

  • message (string)
  • timestamp (string, RFC3339)
  • facility (int)
  • severity (int)
  • priority (int)
  • version (int)
  • hostname (string)
  • procid (string)
  • appname (string)
  • msgid (string)
  • structureddata (object)

syslog_rfc3164​

Attempts to parse a log following the Syslog rfc3164 spec. The resulting structured document may contain any of the following fields:

  • message (string)
  • timestamp (string, RFC3339)
  • facility (int)
  • severity (int)
  • priority (int)
  • hostname (string)
  • procid (string)
  • appname (string)
  • msgid (string)