Skip to main content
Sumo Logic

Amazon CloudWatch Logs

Learn how to collect Amazon CloudWatch Logs.

The preferred method for collecting Amazon CloudWatch Logs is to use the AWS Lambda function provided by Sumo Logic to subscribe to your CloudWatch Log Group. Our AWS Lambda function converts the CloudWatch log format into a format that is compatible with our platform, then POSTs the data directly to a Sumo Logic HTTP Source.  This is the preferred method for the following types of data that are delivered through Amazon CloudWatch Logs:

  1. Custom CloudWatch log data. The AWS Lambda function should handle any log data.  However, you should make sure to test this with your actual data, to ensure that unusually formatted logs are parsed correctly.
  2. Amazon VPC Flow Logs. The AWS Lambda function is compatible with our "Sumo Logic Amazon VPC Flow Logs App." 
  3. AWS Lambda logs. The AWS Lambda function is built for logs generated by your AWS Lambda functions, and is compatible with our "Sumo Logic AWS Lambda App."

The instructions below tell you how to download and configure Sumo Logic's AWS Lambda function for Amazon CloudWatch Logs.

Add a Hosted Collector and HTTP Source

  1. In Sumo Logic, configure a Hosted Collector.
  2. In Sumo Logic, configure an HTTP Source.

Create a Lambda Function

Sumo Logic has created a lambda function for your use with Amazon Web Services (AWS).

https://github.com/SumoLogic/sumologic-aws-lambda/tree/master/cloudwatchlogs/cloudwatchlogs_lambda.js

This file contains a function to collect AWS Lambda logs via CloudWatch Logs. The function extracts and adds a "RequestId" field to each log line to make correlation easier. Download the script file and save it locally. You will use it in these instructions.

To add an Amazon Lambda function:

  1. Make sure you are signed in to the AWS Management Console.
  2. Go to Lambda management.
  3. Create a new function OR click Get Started.
  4. Select Blank Function.
  5. Click Next on the Configure trigger screen
  6. Configure  the new Lambda function:
    Name: sumo-vpc (or something similiar)
    Runtime: Node.js 4.3
    Lambda Function Code: https://github.com/SumoLogic/sumologic-aws-lambda/blob/master/cloudwatchlogs/cloudwatchlogs_lambda.js
    Environment variables (located right underneath the Lambda code):
    key. SUMO_ENDPOINT
    value. URL of HTTP Collector
    Handler. index.handler (default)
    Role. Choose an existing role
    Existing Role. lambda_basic_execution
    No advanced settings
  7. Click Next.
  8. Click Create Function.
  9. Test.
    1. Select CloudWatch Logs from the Sample event template
    2. Click Save and test.

Alternate Collection Methods 

Sumo Logic makes alternate methods of collecting logs form AWS CloudWatch available in case the AWS Lambda method is either not available to you, or you have additional requirements. 

  1. Using Amazon Kinesis : If AWS Lambda is not available to you, or you need increased delivery reliability, review how to add Amazon Kinesis to the integration. See Collecting Amazon CloudWatch Logs using Amazon Kinesis.
  2. Using the Sumo Logic Collector and a Script.  If you have a relatively small amount of CloudWatch logs to collect, and you do not want to set up any additional AWS infrastructure, you may install the Sumo Logic Collector agent locally, and run a script that we have developed for CloudWatch logs, with a special focus on Amazon VPC Flow Logs.  See Collect Amazon CloudWatch Logs Using a Collector Script.