Skip to main content
Sumo Logic

Amazon CloudWatch Logs

Learn how to collect Amazon CloudWatch Logs.

We recommend collecting Amazon CloudWatch Logs through our AWS Lambda function to subscribe to your CloudWatch Log Group. Our AWS Lambda function converts the CloudWatch log format into a format that is compatible with our platform, then POSTs the data directly to a Sumo Logic HTTP Source.  This is the preferred method for the following types of data that are delivered through Amazon CloudWatch Logs:

  • Custom CloudWatch log data. The AWS Lambda function should handle any log data.  However, you should make sure to test this with your actual data, to ensure that unusually formatted logs are parsed correctly.
  • Amazon VPC Flow Logs. The AWS Lambda function is compatible with our Sumo Logic Amazon VPC Flow Logs App.
  • AWS Lambda logs. The AWS Lambda function is built for logs generated by your AWS Lambda functions, and is compatible with our Sumo Logic AWS Lambda App.

The following instructions below tell you to how download and configure an AWS Lambda function for Amazon CloudWatch Logs and send to Sumo.

Add a Hosted Collector and HTTP Source

  1. In Sumo Logic, configure a Hosted Collector.
  2. In Sumo Logic, configure an HTTP Source.

Create a Lambda Function

Sumo Logic has created a lambda function for your use with Amazon Web Services (AWS).

https://github.com/SumoLogic/sumologic-aws-lambda/tree/master/cloudwatchlogs/cloudwatchlogs_lambda.js

This file contains a function to collect AWS Lambda logs using CloudWatch Logs. The function extracts and adds a RequestId field to each log line to make correlation easier. Download the script file and save it locally. You will use it in these instructions.

To add an Amazon Lambda function:

  1. Make sure you are signed in to the AWS Management Console.
  2. Go to Lambda management.
  3. Create a new function OR click Get Started.
  4. Select Blank Function.
  5. Click Next on the Configure trigger screen
  6. Configure the new Lambda function:

    Name. sumo-vpc (or something similar)
    Runtime. Node.js 4.3
    Lambda Function Code. https://github.com/SumoLogic/sumologic-aws-lambda/blob/master/cloudwatchlogs/cloudwatchlogs_lambda.js
    Environment variables (located right underneath the Lambda code):
    key. SUMO_ENDPOINT
    value. URL of HTTP Collector
    Handler. index.handler (default)
    Role. Choose an existing role
    Existing Role. lambda_basic_execution
    No advanced settings.
     
  7. Click Next.
  8. Click Create Function.
  9. Test.
    1. Select CloudWatch Logs from the Sample event template
    2. Click Save and test.

Create a CloudWatch Log Group

You will need at least one CloudWatch Log Group to assign to your Lambda function. For details on how to create a CloudWatch Log Group, see create a CloudWatch Log Group.

Assign CloudWatch Log Groups to Your Lambda Function

  1. Go to the Triggers tab of your Lambda function.
  2. Select Add Trigger.
  3. In the Add Trigger prompt, click the box as instructed and select CloudWatch Logs from the drop-down menu.
  4. Select a CloudWatch Log Group to add to your function. You need at least one CloudWatch Log Group to see this option. For details on creating a log group, see create a CloudWatch Log Group.
  5. Add a Filter Name to your trigger.
  6. (Optional) you can add a Filter Pattern to your trigger.
  7. Click Enable Trigger.
  8. Click Submit to add the trigger to your Lambda function.

Alternate Collection Methods 

If you can't use AWS Lambda to collect logs from CloudWatch, choose one of the following methods:

  1. Using Amazon Kinesis. If AWS Lambda is not available to you, or you need increased delivery reliability, review how to add Amazon Kinesis to the integration. See Collecting Amazon CloudWatch Logs using Amazon Kinesis.
  2. Using the Sumo Logic Collector and a Script.  If you have a relatively small amount of CloudWatch logs to collect, and you do not want to set up any additional AWS infrastructure, you may install the Sumo Logic Collector agent locally, and run a script that we have developed for CloudWatch logs, with a special focus on Amazon VPC Flow Logs.  See Collect Amazon CloudWatch Logs Using a Collector Script.