Collect Logs and Metrics for the AWS Lambda ULM App

AWS CloudWatch Logs

AWS Lambda monitors Lambda functions, and reports metrics through Amazon CloudWatch. Lambda then logs all requests handled by your function and stores logs through AWS CloudWatch Logs.

The AWS Lambda ULM App uses the Lambda logs via CloudWatch and visualizes operational and performance trends about all the Lambda functions in your account, providing insight into executions such as memory and duration usage, broken down by function versions or aliases.

Step 1. Collect Amazon CloudWatch Logs

Sumo supports several methods for collecting Lambda logs from Amazon CloudWatch. 

• You can configure collection of Amazon CloudWatch Logs using our AWS Lambda function using a Sumo-provided CloudFormation template, as described in Amazon CloudWatch Logs.
• To configure collection without using CloudFormation, see Collect Amazon CloudWatch Logs using a Lambda Function. 

Step 2. Collect CloudTrail Lambda Data Events

To configure a CloudTrail Source, perform these steps:

1. Grant Sumo Logic access to an Amazon S3 bucket.
2. Configure DataEvents with CloudTrail in your AWS account.
3. Confirm that logs are being delivered to the Amazon S3 bucket.
4. Add an AWS CloudTrail Source to Sumo Logic.

Step 3. Collect Amazon CloudWatch Metrics

To collect Amazon CloudWatch Metrics, see Amazon CloudWatch Source For Metrics.

Continue with the process of enabling Provisioned Concurrency configurations for Lambda functions, as necessary.

Amazon CloudWatch Log

{"id":"32563142671071560797760688825700039436306340248688066573","timestamp":1511808906799,"message":"REPORT RequestId: cf75cfa3-fe16-11e5-9b16-e3e4c70845f2    Duration: 50.23 ms    Billed Duration: 100 ms     Memory Size: 128 MB    Max Memory Used: 24 MB    
","requestID":null,"logStream":"2017/11/27/[Prod]1108153ced144f8cbb161aef096218d1","logGroup":"/aws/lambda/AWSlambda1"}

CloudTrail Lambda Data Events

{
   "eventVersion":"1.06",
   "userIdentity":{
      "type":"IAMUser",
      "principalId":"AIDAJ45Q7YFFAREXAMPLE",
      "arn":"arn:aws:iam::111111111111:user/duc",
      "accountId":"111111111111",
      "accessKeyId":"AKIAIOSFODNN7EXAMPLE",
      "userName":"duc"
   },
   "eventTime":"2017-11-27T19:05:20.524Z",
   "eventSource":"lambda.amazonaws.com",
   "eventName":"Invoke",
   "awsRegion":"us-west-1",
   "sourceIPAddress":"155.14.186.236",
   "userAgent":"aws-cli/1.11.129 Python/2.7.8 botocore/1.5.92",
   "requestParameters":{
      "invocationType":"RequestResponse",
      "functionName":"arn:aws:lambda:us-west-1:111111111111:function:function237",
      "clientContext":"ew0KICAiB99udGV6lGtleSIgOiAiY29udGV4dHZhbEXAMPLE=="
   },
   "responseElements":null,
   "additionalEventData":{
      "functionVersion":"arn:aws:lambda:us-west-1:111111111111:function:function238:$LATEST"
   },
   "requestID":"e38fb262-8f45-11e7-9845-e5f2f205b110",
   "eventID":"277a6881-66f4-4f3e-ade5-ba76255b7d93",
   "readOnly":false,
   "resources":[
      {
         "accountId":"111111111111",
         "type":"AWS::Lambda::Function",
         "ARN":"arn:aws:lambda:us-west-1:111111111111:function:function239"
      }
   ],
   "eventType":"AwsApiCall",
   "managementEvent":false,
   "recipientAccountId":"111111111111"
}

Query Sample

Count of IAM users invoking CloudTrail Lambda function

_sourceCategory=cloudtrail/lambda "lambda.amazonaws.com" Invoke
| json field=_raw "eventName" as event_name
| json field=_raw "sourceIPAddress" as src_ip
| json field=_raw "requestParameters.functionName" as func_name nodrop
| json field=_raw "additionalEventData.functionVersion" as func_version nodrop
| parse regex field=func_name "\w+:\w+:\S+:[\w-]+:\S+:\S+:(?<function_name>[\S]+)$"
| parse regex field=func_version "\w+:\w+:\S+:[\w-]+:\S+:\S+:(?<function_version>[\S]+:[\S ]+)$" | json field=_raw "userAgent" as user_agent
| json field=_raw "userIdentity.type" as caller_type
| json field=_raw "userIdentity.invokedBy"as invoked_by nodrop
| json field=_raw "userIdentity.userName"as user_name nodrop
| if (isNull(user_name), invoked_by, user_name ) as caller
| if (isNull(invoked_by), user_name, invoked_by ) as caller
| where caller_type = "IAMUser"
| count by caller
| sort by _count

Maximum memory used in MB

_sourceCategory=aws_lambda/lambda*| json "message","logStream","logGroup"
| parse field=message "REPORT RequestId: *Duration: * ms\tBilled Duration: * ms \tMemory Size: * MB\tMax Memory Used: * MB" as RequestId, Duration,BilledDuration,MemorySize,MaxMemoryUsed 
| parse field=logstream "*/[*]*" as logstreamDate,version,logstreamID 
| parse field=loggroup "/aws/lambda/*" as function 
| timeslice 1h
| sum(MaxMemoryUsed) as MaxMemoryUsed by function, _timeslice
| sort by _timeslice