Skip to main content
Sumo Logic

Metrics Data Volume Index

Sumo Logic populates the Metrics Data Volume Index with a set of JSON-formatted messages every five minutes. The messages contain the volume of metric data points your account is ingesting. 

You can query the index to:

  • Get the total metric data volume (data points) ingested by collector, source, source name, source category, or source host, including metrics generated by a logs-to-metrics rule.  
  • Get the total metric data volume for metrics generated by a logs-to-metrics rule.

Message format

Each JSON message contains the parent object for each source data point, and child objects that detail the data points for each parent.

For example, a single message for collector volume data may look similar to the following, where collector_N is the name of a collector. The data points values are the aggregated volume for a five minute time period.

{
    "collector_a":{"dataPoints":733296},
    "collector_b":{"dataPoints":4380031},
    "collector_c":{"dataPoints":386255},
    "collector_d":{"dataPoints":10823082},
    .
    .
}

Querying the Metrics Data Volume index

When you query the index, the query scope must include the following:

_index=sumologic_volume _sourceCategory=<index_source_category>

Where index_source_category is one of the categories listed in the table below.

Index Log Type Index Source Category Description
Collector collector_metrics_volume Use this source category to query metric volume by collector.

Results will include all ingested metrics, including those generated by logs-to-metrics rules. 
Source source_metrics_volume Use this source category to query metric volume by source.

Results will include all ingested metrics, including those generated by logs-to-metrics rules. 
SourceName sourcename_metrics_volume Use this source category to query metric volume by source name. 

Results will include all ingested metrics, including those generated by logs-to-metrics rules. 
SourceCategory sourcecategory_metrics_volume Use this source category to query metric volume by source category.

Results will include all ingested metrics, including those generated by logs-to-metrics rules. 
SourceHost sourcehost_metrics_volume Use this source category to query metric volume by source host. 

Results will include all ingested metrics, including those generated by logs-to-metrics rules. 
Logs-to-Metrics  logstometricsrulename_metrics_volume Use this source category to query metric volume by logs-to-metrics rule. 

Results will include only those ingested metrics that were generated by logs-to-metrics rules.

Metric volume query examples

Metric volume by source category

This query returns the metric volume by source category.

_index=sumologic_volume _sourceCategory=sourcecategory_metrics_volume
| parse regex "\"(?<sourcecategory>(?:[^\"]+)|(?:\"\"))\"\:\{\"dataPoints\"\:(?<dp>\d+)\}" multi
| sum(dp) as dp by sourcecategory

It returns results like these:

dp-by-category.png

Metric volume by collector

This query returns the metric volume by collector.

_index=sumologic_volume _sourceCategory=collector_metrics_volume
| parse regex "\"(?<collector>(?:[^\"]+)|(?:\"\"))\"\:\{\"dataPoints\"\:(?<dp>\d+)\}" multi
| sum(dp) as dp by collector

It returns results like these:

dp-by-collector.png

Metric volume for a specific collector

This query returns the metric volume for a specific Collector. 

_index=sumologic_volume _sourceCategory=collector_metrics_volume
| json "your-collect-name" as collector_json |
json field=collector_json "dataPoints" as dp
| sum(dp) as dp
| fields dp

Substitute the name of your collector for your-collect-name.

Data points per minute (DPM) by logs-to-metrics rule

This query returns the DPM resulting from each of your logs-to-metrics rules.

_index=sumologic_volume datapoints _sourceCategory="logstometricsrulename_metrics_volume"
| parse regex "\"(?<logstometricsrulename>[^\"]+)\"\:\{\"dataPoints\"\:(?<datapoints>\d+)\}" multi
| sum(datapoints) as datapoints by logstometricsrulename
| ((queryEndTime() - queryStartTime())/(1000*60)) as duration_in_min
| datapoints / duration_in_min as %"DPM" 
| fields logstometricsrulename,DPM
| sort by DPM

It returns results like these:

dpm-by-l2m-rule.png

Query for metric throttling events

This query searches the Audit Index for messages that indicate that metric ingestion has been throttled. Metric throttling occurs when you exceed your DPM burst limit. For more information, see Metric Throttling.  

_index=sumologic_audit _sourceCategory=account_management _sourceName=VOLUME_QUOTA "Resource type: MetricIngest"

Suggested search time range: Last one hour (-1h)

Suggested frequency for scheduling: Every one hour

Query for metric ingestion outliers 

This query runs against the metrics volume index and uses the outlier operator to find timeslices in which your metric ingestion in DPM was greater than the running average by a statistically significant amount. 

_index=sumologic_volume _sourceCategory=sourcecategory_metrics_volume
| parse regex "\"(?<sourcecategory>(?:[^\"]+)|(?:\"\"))\"\:\{\"dataPoints\"\:(?<dp>\d+)\}" multi
| timeslice 15m
| sum(dp) as dp by _timeslice
| outlier dp window=5,threshold=3,consecutive=1,direction=+
| where dp_violation > 1

Suggested search time range: Last 3.5 hours (-210m)

Suggested frequency for scheduling: Every one hour

Sustained DPM above plan limit query

This query runs against the metrics volume index and checks for 15 minute periods during which your metric ingestion in DPM was continuously over your account DPM limit.

_index=sumologic_volume _sourceCategory=sourcecategory_metrics_volume
| parse regex "\"(?<sourcecategory>(?:[^\"]+)|(?:\"\"))\"\:\{\"dataPoints\"\:(?<dp>\d+)\}" multi
| timeslice 15m
| sum(dp) as dp by _timeslice
| compare with timeshift 15m 8 min
//dpm limit = 3000000 * 15 minutes (size of timeslice bucket)
| 300000 * 15 as dp_limit
| where (dp > dp_limit and dp_120m_min > dp_limit)

Suggested search timeframe: Last 15 minutes (-15m)

Suggested frequency for scheduling: Every 15 minutes

Predict DPM exceeding account limit

This query runs against the metrics volume index and uses the predict operator to predict when in the future your metric ingestion in DPM is likely to exceed the current DPM limit for your account.   

_index=sumologic_volume _sourceCategory=sourcecategory_metrics_volume
| parse regex "\"(?<sourcecategory>(?:[^\"]+)|(?:\"\"))\"\:\{\"dataPoints\"\:(?<dp>\d+)\}" multi
| timeslice 15m
| sum(dp) as dp by _timeslice
| predict dp by 15m model=ar, ar.window=1
//dpm limit = 3000000 * 15 minutes (size of timeslice bucket)
| 300000 * 15 as dp_limit
| where dp_predicted > dp_limit

Suggested search time range: Last 24 hours (-24h)

Suggested frequency for scheduling: Every one hour

Source categories not collecting metrics

This query returns a list of metric source categories for which no metrics were ingested in the last 60 minutes.

_index=sumologic_volume _sourceCategory=sourcecategory_metrics_volume
| parse regex "\"(?<sourcecategory>(?:[^\"]+)|(?:\"\"))\"\:\{\"dataPoints\"\:(?<dp>\d+)\}" multi
| first(_messagetime) as MostRecent, sum(dp) as TotalDataPoints by sourcecategory
| formatDate(fromMillis(MostRecent),"yyyy/MM/dd HH:mm:ss") as MostRecentTime 
| toMillis(now()) as currentTime
| formatDate(fromMillis(currentTime),"yyyy/MM/dd HH:mm:ss") as SearchTime
| (currentTime-MostRecent) / 1000 / 60 as mins_since_last_datapoint
| where mins_since_last_datapoint >= 60
| fields -mostrecent, currenttime 
| format ("%s Has not collected data in the past 60 minutes", sourcecategory) as message

For example:

no-dpmpng.png

Suggested search time range: Last one hour (-1h)

Suggested frequency for scheduling: Every one hour