Skip to main content
Sumo Logic

Data Enrichment

Data Enrichment
Add more context to your data.

Data enrichment is the process of adding meaningful information to your data so you have more control and an easier time referencing data in searches. It's simply where you add more context to your data.

Sumo Logic supports data enrichment using metadata and lookups.

Using Metadata

Metadata is typically from your system or environment, and adds context about what or where the data came from and any associated services or apps. Logs and metrics use metadata that can be customized to anything you need.

  • Log metadata - In addition to having more data to reference in query operations, this allows you to define a more specific scope of data in search expressions, improving search performance, and allows more specific search filters in Roles and routing expressions in Partitions.

    • Log metadata is configured in Sumo as fields consisting of key-value pairs that are tagged to logs during collection.

      • You can define fields with Field Extraction Rules by parsing fields when log messages are ingested.

      • You can define fields on data sent to Sumo by manually defining them on Sources and Collectors.

      • You can provide custom fields through HTTP headers.

      • Our AWS Metadata Source allows you to collect tags from EC2 instances running on AWS.
         

  • Metric metadata - Sumo Logic provides a number of features you can use to enrich the metrics you collect with metadata. Metric metadata provides considerable benefits when you query your metrics: you can scope your metric queries to return only the metrics of interest. Metric metadata can also give you insight that can't be gleaned from unadorned metrics, especially in highly containerized and orchestrated environments. 

    • Metric metadata is referenced in Sumo with selectors consisting of key-value pairs that are tagged to metrics during collection.

      • You can attach custom metadata to metrics you send to an HTTP source.

      • You can use the AWS Metadata (Tag) Source for Metrics to apply tags from your EC2 instances to the host metrics, Graphite metrics, and Carbon 2.0 metrics you collect.

      • You can use metric rules editor to tag metrics with data derived from the metric identifier, and then use those tags in metric queries.
         

Using Lookups

Lookups allow you to enrich data with external context that is not part of the original data.

  • Log lookups - allow you to enrich your log data with information from external sources like customer data in salesforce or user data from active directory. This data enrichment lets you perform more rich and powerful analytics.  

    • There are two types of lookup tables available in Sumo for context enrichment. 

      • File based lookup tables maintain a copy of the original data within Sumo. You can create these types of lookup table using the save operator.

      • URL based lookups allow you to host your data outside of Sumo in CSV format and reference it for correlating your logs.  

    • Once you have a lookup table, you can use the lookup operator to enrich your log data with contextual information from lookup table.