Skip to main content
Sumo Logic

Amazon CloudWatch Logs

Learn how to collect Amazon CloudWatch Logs.

We recommend collecting Amazon CloudWatch Logs through our AWS Lambda function to subscribe to your CloudWatch Log Group. Our AWS Lambda function converts the CloudWatch log format into a format that is compatible with our platform, then POSTs the data directly to a Sumo Logic HTTP Source.  This is the preferred method for the following types of data that are delivered through Amazon CloudWatch Logs:

  • Custom CloudWatch log data. The AWS Lambda function should handle any log data.  However, you should make sure to test this with your actual data, to ensure that unusually formatted logs are parsed correctly.
  • Amazon VPC Flow Logs. The AWS Lambda function is compatible with our Sumo Logic Amazon VPC Flow Logs App.
  • AWS Lambda logs. The AWS Lambda function is built for logs generated by your AWS Lambda functions, and is compatible with our Sumo Logic AWS Lambda App.

The following instructions below tell you to how download and configure an AWS Lambda function for Amazon CloudWatch Logs and send to Sumo.

Add a Hosted Collector and HTTP Source

  1. In Sumo Logic, configure a Hosted Collector.
  2. In Sumo Logic, configure an HTTP Source.

Sumo provides a Lambda function for use with Amazon Web Services (AWS). It collects AWS Lambda logs using CloudWatch Logs and it extracts and adds a RequestId field to each log line to make correlation easier. 

To add an Amazon Lambda function:

  1. Sign into the AWS Management Console.
  2. Click Lambda in the Compute section.
  3. On the AWS Lambda page, click Create a Function
  4. On the Blueprints page, enter sumologic in the search field, and click the search icon.
  5. Select sumologic-process-logs.
    The Create Function page appears.
  6. In the Basic information section:

    lambda4.png
    1. Name—Enter a name for the function.
    2. Role—Choose one of the following options:
      • Choose an existing role. If you have any appropriate roles, you can select one.
      • Create new role from template(s). If you select this option, you can continue without choosing any policy templates—it will create a role with basic Lambda execution privileges by default.
    3. Role Name—Enter a name for the role.
    4. Policy templates—If you selected Create new role from template(s) above, you can leave this blank. 
  7. In the cloudwatch-logs section, you can create a trigger now, or click Remove if you prefer to create it later. To create the trigger:
    trigger.png
    1. Log Group—Select the log group that serves as the event source. Events sent to the log source will trigger your Lambda function. 
    2. Filter Name—Enter a filter name.
    3. Filter Pattern—May be left blank. For information about AWS filter patterns, see Filter and Pattern Syntax in AWS help.
    4. Enable trigger—Check the box to enable the trigger immediately. 
    5. Click Create Function.
  8. On the Environment Variables page, create a environment variable named SUMO_ENDPOINT. Set the value of the variable to the URL of the HTTP source to which your logs will be sent.

    In addition, you can set any of the following optional variables:lambda6.png

     
    • ENCODING (Optional)—Encoding to use when decoding CloudWatch log events. Default is utf-8.
    • SOURCE_CATEGORY_OVERRIDE (Optional)—Override _sourceCategory value configured for the HTTP source.
    • SOURCE_HOST_OVERRIDE (Optional)—Override _sourceHost value configured for the HTTP source.
    • SOURCE_NAME_OVERRIDE (Optional)—Override _sourceName value configured for the HTTP source.

Create a CloudWatch Log Group

You will need at least one CloudWatch Log Group to assign to your Lambda function. For details on how to create a CloudWatch Log Group, see create a CloudWatch Log Group.

Assign CloudWatch Log Groups to Your Lambda Function

  1. Go to the Triggers tab of your Lambda function.
  2. Select Add Trigger.
  3. In the Add Trigger prompt, click the box as instructed and select CloudWatch Logs from the drop-down menu.
  4. Select a CloudWatch Log Group to add to your function. You need at least one CloudWatch Log Group to see this option. For details on creating a log group, see create a CloudWatch Log Group.
  5. Add a Filter Name to your trigger.
  6. (Optional) you can add a Filter Pattern to your trigger. For information about AWS filter patterns, see Filter and Pattern Syntax in AWS documentation 
  7. Click Enable Trigger.
  8. Click Submit to add the trigger to your Lambda function.

Alternate Collection Methods 

If you can't use AWS Lambda to collect logs from CloudWatch, choose one of the following methods:

  1. Using Amazon Kinesis. If AWS Lambda is not available to you, or you need increased delivery reliability, review how to add Amazon Kinesis to the integration. See Collecting Amazon CloudWatch Logs using Amazon Kinesis.
  2. Using the Sumo Logic Collector and a Script.  If you have a relatively small amount of CloudWatch logs to collect, and you do not want to set up any additional AWS infrastructure, you may install the Sumo Logic Collector agent locally, and run a script that we have developed for CloudWatch logs, with a special focus on Amazon VPC Flow Logs.  See Collect Amazon CloudWatch Logs Using a Collector Script.