Skip to main content
Sumo Logic

Lab 3 - Parsing Options

Parsing your logs allow you to provide structure to your messages, identifying the fields that are meaningful to you.
Parsing your logs allow you to provide structure to your messages, identifying the fields that are meaningful to you.

 

  1. The nodrop option for the parse operator allow users to include messages in your results that do not meet the pattern criteria or a parse statement. In this example, messages from the first parse statement will be dropped, so they can be parsed by the second parse statement.

_sourceCategory=Labs/Apache/Error

| parse "[client *]" as client_ip nodrop

| parse "mod_log_sql: *" as message

//| where isBlank(client_ip)

  1. The parse field option allows you to do further parsing on an already extracted field. In this example, we want to identify the top 5 Sumo Logic committers in GitHub. We start by searching for committers in the last 30 days, and parse their email address.  We then use the parse field option to further parse the email address into user and domain, select only those users we care for, and lastly, count by user and identify the top 5 committers.

_sourceCategory=Labs/Github and "committer"

| parse "\"email\":\"*\"" as email

| parse field=email "*@*" as users, domain

| where domain="sumologic.com"

| count by users

| top 5 users by _count

  1. The parse multi option allows you to extract multiple occurrences of  the same pattern within one message. By default, parse only extracts the first occurrence. Notice how each message is repeated for each occurrence of an ip address, allowing you to do accurate counts.

_sourceCategory=labs/snort

| parse regex "(?<ip_address>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})" multi

Pre-parsing your Messages with Field Extraction Rules
  1. Field Extraction Rules (FERs) extract fields at the time the log messages are ingested. You can see all FERs available (and their details) under : Manage Data → Settings → Field Extraction Rules.Image of FER menu

To see the benefit of an Apache Access FER rule already in place, run a search to identify the count of 404s by source ip.

_sourceCategory=Labs/Apache/Access and status_code=404

| count by src_ip

Notice how the src_ip field didn't need to be parsed using query syntax?

QUIZ: True or False

  1. Parsing operators include csv, json, split, and keyvalue.

  2. Once a field has been parsed, it cannot be parsed any further.

  3. Fields parsed by the Field Extraction Rules are available in the Field Browser.