Field Extractions

Field extractions allow you to parse fields from your log messages at the time the messages are ingested, which eliminates the need to parse fields at the query level. With Field Extraction Rules (FERs) in place, users can use the pre-parsed fields for ad-hoc searches, scheduled searches, real-time alerts, and dashboards. In addition, field extraction rules help standardize field names and searches, simplify the search syntax and scope definition, and improve search performance. 

Fields are extracted from the time you create your FER moving forward. Therefore, set your FERs early on to take advantage of this automatic parsing mechanism.

For best practices on naming your fields, see Field Naming Convention. 

info

You need the Manage field extraction rules role capability to create a field extraction rule. 

The Manage Data > Logs > Field Extraction Rules page displays the following information: 

When hovering over a row in the table there are icons that appear on the far right for editing, disabling and deleting the rule.

• Status shows a checkmark in a green circle  to indicate if the Rule is actively being applied or an exclamation mark in a red circle  to indicate if the Rule is disabled.
• Rule Name
• Applied At indicates when the field extraction process occurs, either at Ingest or Run time.
• Scope 
• Created date and time by user
• Last Modified date and time by user
• Fields Capacity (bottom of table) shows how many fields your account is using, out of the total available for use.

You can view the fields created in your account and what features are referencing them on the Fields page.

On the Manage Data > Logs > Field Extraction Rules page you can:

• Click + Add to create a Field Extraction Rule.
• Search Field Extraction Rules by name and scope.
• Edit a Field Extraction Rule.
• Disable a Field Extraction Rule.
• Delete a Field Extraction Rule.

Limitations

Ingest Time FERs have the following limitations:

• There is a limit of 50 Ingest Time rules and 200 fields. Fields created as log metadata and from Ingest Time rules share the same quota of 200 fields. You can manage your fields on the Fields page.
• Ingest Time rule expressions are limited to a maximum of 16k (16,384) characters.
• Ingest Time rules can extract up to a maximum of 16k (16,384) characters for each field.
• The cumulative size of all fields extracted by a rule for a message/event is limited to 64kb.
• Ingest Time rules only apply to data moving forward. If you want to parse data ingested before the creation of your Ingest Time FER, you can either parse your data in your query, or create Scheduled Views to extract fields for your historical data.

Micro Lesson: Field Extraction Rules Basics

Edit a Field Extraction Rule

Changes to Field Extraction Rules are implemented immediately.

1. In Sumo Logic, go to Manage Data > Logs > Field Extraction Rules.
2. Find the rule in the table and click it. A window appears on the right of the table, click the Edit button.
3. Make changes as needed and click Save when done.

Delete a Field Extraction Rule

Deleting a Field Extraction Rule doesn't delete the fields it was parsing. You can delete any unwanted fields on the Fields page.

1. In Sumo Logic, go to Manage Data > Logs > Field Extraction Rules.
2. Find the rule to delete in the table and click it. A window appears on the right of the table, click the More Actions button, and select Delete.

Guide contents

In this section, we'll introduce the following concepts:

Field Naming Convention

Learn about the recommended naming conventions for standard fields in Sumo Logic.

Create a Field Extraction Rule

Learn how to instruct Sumo Logic to parse out fields automatically.

Edit Field Extraction Rules

Learn how to change Field Extraction Rules.

FER Templates

Learn how to use FER Templates to parse common fields for various applications.

Parse AWS ELB Logs

Learn how to parse the common fields in AWS ELB logs.

Sample Safend Field Extraction

Learn how to create Field Extraction Rules for Safend.