Skip to main content
Sumo Logic

Collect Logs and Metrics for the Amazon ElastiCache Redis ULM App

This page provides instructions for collecting logs and metrics for the Sumo Logic App for Amazon ElastiCache Redis ULM, as well as providing an example log message and sample queries..

The Amazon ElastiCache Redis ULM  App is used for monitoring CloudTrail event logs and CloudWatch Metrics. Metrics allow you to monitor in-memory database resource utilization and throughput performance. CloudTrail events help you monitor use of Amazon ElastiCache Redis services and operations by users.

This page provides instructions for collecting logs and metrics for the Amazon ElastiCache Redis ULM App. Click a link to jump to a topic.

Step 1. Plan Source Categories

Before you configure the log and metric sources for the Sumo Logic App for Amazon ElastiCache Redis ULM, define the source category that will be assigned to each. We recommend a hierarchical category structure that allows for the use of wildcards when you perform searches, such as the following examples:

  • For AWS CloudTrail source for CloudTrail Events, use the source category: AWS/CloudTrail
  • For AWS CloudWatch Metric source to collect cloudwatch metrics, use the source category: AWS/ElastiCache/Metric
  • For AWS ElastiCache Events through SNS, use the source category: AWS/ElastiCache/Events/Notifications

Step 2. Collect AWS CloudTrail events using AWS CloudTrail Source

This section explains how to configure a Hosted Collector and add a AWS CloudTrail Source to collect CloudTrail events.

To configure a Hosted Collector and add a AWS CloudTrail Source, do the following:
  1. Configure a Hosted Collector.

  2. To your Hosted Collector, add an AWS CloudTrail Source, specifying the following:

    • Name—Enter a name to display for the new Source.

    • Description—Enter an optional description.

    • S3 Region—Select the Amazon Region for your CloudTrail Amazon ElastiCache S3 bucket.

    • Bucket Name—Enter the exact name of your CloudTrail Amazon ElastiCache S3 bucket.

    • Path Expression—Enter the string that matches the S3 objects you'd like to collect. You can use a wildcard (*) in this string. (DO NOT use a leading forward slash. See Amazon Path Expressions..)The S3 bucket name is not part of the path. Don’t include the bucket name when you are setting the Path Expression.

    • Source Category—Enter a source category. For example, AWS/Cloudtrail.

    • Access Key ID and Secret Access Key—Enter your Amazon Access Key ID and Secret Access Key.

    • Scan Interval—Use the default of 5 minutes. Alternately, enter the frequency Sumo Logic will scan your S3 bucket for new data.

    • Enable Timestamp Parsing—Select the checkbox.

    • Time Zone—Select Ignore time zone from log file and instead use, and select UTC.

    • Timestamp Format—Select Automatically detect the format.

    • Enable Multiline Processing—Select the checkbox, and select Infer Boundaries.

  3. Click Save.

Step 3. Collect Amazon ElastiCache metrics using AWS CloudWatch Metric Source

This section explains how to configure a Hosted Collector and add an Amazon CloudWatch Metric Source to collect CloudWatch metrics.

To configure a Hosted Collector and add an Amazon CloudWatch Metric Source, do the following:
  1. Configure a Hosted Collector.

  2. Configure an Amazon CloudWatch Metrics Sourcespecifying the following:

    • Name—Enter a name to display for the new Source.

    • Description—Enter an optional description.

    • Regions—Select your Amazon Regions for Amazon RDS.

    • Namespaces—Select AWS/ElastiCache

    • Source Category—Enter a source category. For example, AWS/ElastiCache/Metric.

    • Access Key ID and Secret Access Key—Enter your Amazon Access Key ID and Secret Access Key.

    • Scan Interval—Use the default of 5 minutes, or enter the frequency Sumo Logic will scan your CloudWatch Sources for new data.

  3. Click Save.

Step 4: Collect Amazon ElastiCache events with AWS SNS

This section shows you how to configure ElastiCache to send notifications for important cluster events using Amazon Simple Notification Service (Amazon SNS) to Sumo.

To collect Amazon ElastiCache events using AWS SNS, do the following:
  1. Configure a Hosted Collector.

  2. Configure an  HTTP source

    • Name—Enter a name to display for the new Source.

    • Description—Enter an optional description.

    • Source Category—Enter a source category, such as: AWS/ElastiCache/Events/Notifications

    • Timestamp Parsing Settings:

      • Enable Timestamp Parsing—True

      • Timezone—Logs are sent in UTC by default and can be auto detected

      • Timestamp Format—Auto Detect

    • Deselect checkbox—Enable Multiline Processing

    • Enable—One Message Per Request

  3. Click Save.
     Note the source EndPoint URL, as you will receivie SNS notifications on it.
  4. Create SNS Topic, if not already created in SNS Service at AWS account from which you want to forward events to Sumo. Note the SNS Topic ARN.

  5. Create a Subscription to the SNS Topic you just created, specifying the following:

    • Topic ARN—from the Step 4.

    • Protocol—HTTPS

    • EndPoint—Sumo source Endpoint URL you noted in Step 3.

  6. Request confirmation for the subscription, and when the message is sent to the Sumo source EndPoint UR, do the following:

    • Go to the Sumo search box and execute the following: 
      _sourcecategory=AWS/ElastiCache/Events/Notifications SubscribeURL

    • Look in Sumo for the source category, and get the SubscribeURL field value you will need to confirm the subscription by entering it in the Subscription confirmation URL field.

    • In your AWS Account > SNS service, select the subscription you just created, choose Confirm Subscription, paste the SubscribeURL, and click Confirm Subscription.

  7. Select the subscription, then under Other Subscription actions or Actions, select Edit Subscription Attributes, check the check box for Raw message delivery, and click Set Subscription attributes. This delivers a message to Sumo source with only one field as raw.

  8. After completing a successful subscription, go to AWS ElastiCache service, select the ElastiCache cluster to which you want to forward event notifications using SNS, and configure Topic for SNS Notification* with the topic name from Step 4.

Sample log message

The following is a sample log file is for Amazon CloudTrail Event for Elasticache Redis.

{"eventVersion":"1.05","userIdentity":{"type":"IAMUser","principalId":"A12345678904QEWUABG5Q","arn":
"arn:aws:iam::123456789038:user/Nitin","accountId":"123456789038","accessKeyId":"A1234567890FHCUQYQRM","userName":"Nitin",
"sessionContext":{"attributes":{"mfaAuthenticated":"true","creationDate":"2018-10-29T07:08:50Z"}},"invokedBy":
"signin.amazonaws.com"},"eventTime":"2018-10-29T08:38:13Z","eventSource":"elasticache.amazonaws.com","eventName":
"CreateCacheSubnetGroup","awsRegion":"us-west-1","sourceIPAddress":"49.48.90.17","userAgent":"signin.amazonaws.com",
"requestParameters":{"cacheSubnetGroupName":"nitin-redis-subnet-grp1","subnetIds":["subnet-b33fc55e"]},"responseElements":
{"cacheSubnetGroupDescription":" ","vpcId":"vpc-b12fc345","subnets":[{"subnetAvailabilityZone":{"name":"us-west-1a"},
"subnetIdentifier":"subnet-b33fc55e"}],"cacheSubnetGroupName":"nitin-redis-subnet-grp1"},"requestID":
"c6a79737-1234-5678-bb74-9f27f56e6306","eventID":"70c2c865-1234-4567-893c-9800b91e2502","eventType":"AwsApiCall",
"recipientAccountId":"123456789038"}

The following sample log file is for Amazon ElastiCache Event as SNS Notifications.

{"ElastiCache:ReplicationGroupScalingInStarted":"nitin-redis-cluster1"}

Query samples

The following query is from the Event Status Trend panel of the Amazon ElastiCache Redis ULM - CloudTrail Event - Overview Dashboard.


(_sourceCategory=*cloudtrail* or _sourceCategory=*AWS_EAGLE*) "\"eventSource\":\"elasticache.amazonaws.com\""
| json "userIdentity", "eventSource", "eventName", "awsRegion", "sourceIPAddress", "userAgent", "eventType", "recipientAccountId", "requestParameters", "responseElements", "requestID", "errorCode", "errorMessage" nodrop
| json field=userIdentity "type", "principalId", "arn", "userName", "accountId" nodrop
| json field=userIdentity "sessionContext.attributes.mfaAuthenticated" as mfaAuthenticated nodrop
| json field=requestParameters "replicationGroupId", "engine", "engineVersion" as req_replicationGroupId, req_engine, req_engineVersion nodrop
| json field=responseElements "replicationGroupId", "engine", "engineVersion", "status" as res_replicationGroupId, res_engine, res_engineVersion, res_status nodrop
| parse field=arn ":assumed-role/*" as user nodrop  
| parse field=arn "arn:aws:iam::*:*" as accountId, user nodrop
| if (isEmpty(errorCode), "Success", "Failure") as eventStatus
| if (isEmpty(userName), user, userName) as user
| if (isEmpty(req_replicationGroupId), res_replicationGroupId, req_replicationGroupId) as replicationGroupId
| if (isEmpty(req_engine), res_engine, req_engine) as engine
| if (isEmpty(req_engineVersion), res_engineVersion, req_engineVersion) as engineVersion
| where eventSource = "elasticache.amazonaws.com"
| timeslice 6h
| count by _timeslice, eventStatus, eventName
| transpose row _timeslice column eventStatus, eventName

The following  query sample is from the Cache Hit Rate panel of the Amazon ElastiCache Redis ULM - Metric - Overview Dashboard.


_sourceCategory=AWS/ElastiCache/Metric Namespace=AWS/ElastiCache metric=CacheHits Statistic=Average !CacheClusterId=* !CacheNodeId=* | avg
_sourceCategory=AWS/ElastiCache/Metric Namespace=AWS/ElastiCache metric=CacheMisses Statistic=Average !CacheClusterId=* !CacheNodeId=* | avg
(#A / (#A + #B) * 100)

The following query is from the Recent Events panel of the Amazon ElastiCache Redis ULM - Notifications Dashboard.

_sourceCategory=AWS/ElastiCache/SNSNotifications* Elasticache
| json "Message" as msg nodrop
| if (!isEmpty(msg), msg, _raw) as msg
| parse field=msg "\"ElastiCache:*\":\"*\"" as event, clusterId nodrop
| parse field=msg "\"Elasticache:*\":\"*\"" as event, clusterId nodrop
| json field=msg "Total Slots Moved" as TotalSlotsMoved nodrop
| json field=msg "Amount Of Node Groups That Will Be Removed" as AmountOfNodeGroupsThatWillBeRemoved nodrop
| if (event matches "*Failed", "Failure", "Success") as status
| timeslice 1s
| count by _timeslice, event, clusterId, status
| sort by _timeslice