A Sumo Logic CloudWatch Source allows you to gather metrics data from an Amazon resource.
Supported AWS metrics
A Sumo CloudWatch Source only supports CloudWatch metrics that are emitted at a regular interval. It cannot ingest metrics that are emitted with significant latency, such as AWS S3 Daily Storage Metrics or AWS Billing metrics, or at sporadic intervals, such as AWS DynamoDB throttled events.
Sumo does support S3 Request Metrics. Since S3 does not publish the request metrics by default, you must enable them if you want to collect them. For more information, see Monitoring Metrics with Amazon CloudWatch in AWS help.
Set up an Amazon CloudWatch Source
- Before you begin, grant permission for Sumo Logic to list available metrics and get metric data points. See Grant Access to an AWS Product for details.
- In Sumo Logic select Manage Data > Collection > Collection.
- Click Add Source next to a Hosted Collector.
- Select AWS CloudWatch.
- Configure the following:
- Name. Enter a name to display for the new source.
- Description. Optional description.
- Regions. Select one or more Amazon regions.
- Namespaces. Select one or more Amazon namespaces.
- EC2 Filters. This setting is visible only if you select an EC2 namespace.
To restrict the CloudWatch source to particular EC2 instances, enter AWS tags for the instances in key=value format.
- Use semicolons if you want to include multiple values for an individual key (OR match).
- You can specify multiple key/value pairs. Additional entry fields are added as needed, up to the maximum allowed number of pairs.
- Custom Namespaces. Enter a comma-separated list of any custom namespaces from which you want to collect custom metrics. For more information about custom metrics, see http://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html.
- Source Category. Enter any string to tag the output collected from this Source. (Category metadata is stored in a searchable field called _sourceCategory.)
- For AWS Access you have two Access Method options. Select Role-based access or Key access based on the AWS authentication you are providing. Role-based access is preferred, this was completed in step 1, Grant Sumo Logic access to an AWS Product.
- Scan Interval. Use the default of 5 minutes, or change this value to indicate how frequently Sumo Logic should poll the CloudWatch API. To learn more about polling interval considerations, see AWS CloudWatch Scan Interval below.
- Total Metrics. This field displays the total number of metrics (unique metric time series) that will be collected if the Source is created with the current configuration. If all of your CloudWatch metrics are published at a 1 minute interval, then "Total Metrics" will also be the total number of 'data points per minute' that are generated by this source. However, if your CloudWatch metrics are published every 5 minutes, then you would divide this number by 5 to get the number of 'data points per minute' that would be generated by this source. The field automatically refreshes the count when there are changes to the following fields: Regions, Namespaces, Access Key ID, and Secret Access Key.
- Click Save.
AWS CloudWatch Scan Interval
The scan interval defines how long Sumo Logic waits between calls to the CloudWatch API. This does not affect the number of metric data points collected. If metrics are published to CloudWatch every minute, and you scan every 5 minutes, then each API response would return 5 data points. Decreasing the interval will reduce the number of API calls, which may help with your AWS bill. However, it will also add latency to your CloudWatch Metrics collection.
AWS reports CloudWatch metrics at different granularities (1-minute, 3-minute, and 5-minute intervals), so setting a scan interval that's too short could lead to excessive querying. Setting an interval that's too long can delay the update frequency of new metrics appearing in Sumo Logic.
Querying the AWS CloudWatch Metrics API can incur data transfer charges that may appear on your AWS bill.
Throttling of CloudWatch data
AWS automatically throttles CloudWatch data if the limits that Amazon sets for the associated APIs are exceeded. If you have a high volume of metrics data points in your account, it is likely that Amazon will throttle your CloudWatch data.
If no adjustments are made on the Sumo Logic side, throttling on the Amazon side can cause metrics data to be dropped. To prevent this from occurring, Sumo Logic automatically doubles the CloudWatch scan interval if more than one throttling message is received in a single interval. However, the change in scan interval isn't reflected in the Sumo Logic UI. The original configured interval is still shown.
If the scan interval is automatically changed, a message similar to the following is added to the audit log. No action is required by the Sumo Logic user.
CloudWatch source ui-cw-oldPrimary received throttling exception from AWS while querying for metrics. Increasing scan interval to 20 minutes.
CloudWatch data point aggregation
AWS pre-aggregates CloudWatch data points using these aggregators:
You can query CloudWatch data points on the Metrics page using a
Statistic=<aggregator> query. See Create a Metrics Visualization for details.
For details on Amazon CloudWatch collected metrics, refer to: