AWS Classic Load Balancer

AWS Elastic Load Balancer Classic distributes the incoming application traffic across multiple EC2 instances in multiple Availability Zones.

The Sumo Logic app for AWS Elastic Load Balancer Classic is a unified logs and metrics App that helps you monitor the classic load balancer. The preconfigured dashboards provide information on the latency, HTTP backend codes, requests, and host status, that help you investigate the issues in the load balancer.

Log and metric types

ELB logs are stored as *.log files in the buckets you specify when you enable logging. The process to enable collection for these logs is described in AWS ELB Enable Access Logs.

The logs themselves contain these fields in this order:

datetime, ELB_Server, clientIP, port, backend, backend_port, requestProc, ba_Response, cli_Response, ELB_StatusCode, be_StatusCode, rcvd, send, method, protocol, domain, server_port, path

The log format is described in AWS ELB Access Log Collection.

For details on AWS Classic Load Balancer metrics, see here.

Sample access log message

2017-11-06T23:20:38 stag-www-lb 250.38.201.246:56658 10.168.203.134:23662 0.007731 0.214433 0.000261 404 200 3194 123279 \
"GET https://stag-www.sumologic.net:443/json/v2/searchquery/3E7959EC4BA8AAC5/messages/raw?offset=29&length=15&highlight=true&_=1405591692470 HTTP/1.1" \
"Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:23.0) Gecko/20131011 Firefox/23.0" \
ECDHE-RSA-CAMELLIA256-SHA384 TLSv1.2

Sample queries

Response Codes Distribution by Domain and URI (Access Log Based)

account={{account}} region={{region}} namespace={{namespace}}
| parse "* * * * * * * * * * * \"*\" \"*\" * *" as datetime, loadbalancername, client, backend, request_processing_time, backend_processing_time, response_processing_time, elb_status_code, backend_status_code, received_bytes, sent_bytes, request, user_agent, ssl_cipher, ssl_protocol
| where tolowercase(loadbalancername) matches tolowercase("{{loadbalancername}}")
| parse field=request "* *://*:*/* HTTP" as Method, Protocol, Domain, ServerPort, URI nodrop
| parse field=client "*:*" as clientIP, port nodrop
| parse field=backend "*:*" as backendIP, backend_port nodrop
| fields - request, client, backend
| if (backend_status_code matches "5*",1,0) as Backend_5XX
| if (backend_status_code matches "4*",1,0) as Backend_4XX
| if (backend_status_code matches "3*",1,0) as Backend_3XX
| if (backend_status_code matches "2*",1,0) as Backend_2XX
| sum(Backend_5XX) as Backend_5XX, sum(Backend_4XX) as Backend_4XX, sum(Backend_3XX) as Backend_3XX, sum(Backend_2XX) as Backend_2XX by loadbalancername, Domain, URI
| limit 20
| sort by Backend_5XX, Backend_4XX, Backend_3XX, Backend_2XX

4XX by Load Balancer (Metrics-based)

account={{account}} region={{region}} Namespace={{namespace}} \
loadbalancername={{loadbalancername}} metric=HTTPCode_ELB_4XX \
Statistic=Sum | sum by account, region, namespace, loadbalancername

Collecting logs and metrics for the AWS Classic Load Balancer

When you create an AWS Source, you'll need to identify the Hosted Collector you want to use or create a new Hosted Collector. Once you create an AWS Source, associate it with a Hosted Collector. For instructions, see Configure a Hosted Collector and Source.

Collect metrics

1. Sumo Logic supports collecting metrics using two source types:
   • Configure an AWS Kinesis Firehose for Metrics Source (recommended); or
   • Configure an Amazon CloudWatch Source for Metrics
2. Metadata. Click the +Add Field link to add custom log metadata fields. Define the fields you want to associate, each field needs a name (key) and value.
   1. Add an account field and assign it a value which is a friendly name / alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
      
   2. Keep in mind:
      • A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema.
      • An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled, in the Fields table schema. In this case, an option to automatically add or enable the nonexistent fields to the Fields table schema is provided. If a field is sent to Sumo Logic that does not exist in the Fields schema or is disabled it is ignored, known as dropped.

Collect access logs

Prerequisites

Before you can begin to use the AWS Classic Load Balancing (ELB) App, complete the following steps:

1. Grant Sumo Logic access to an Amazon S3 bucket.
2. Enable Application Load Balancer logging in AWS.
3. Confirm that logs are being delivered to the Amazon S3 bucket.

Collecting access logs for AWS Classic Load Balancer

1. Configure a Classic Load Balancing (CLB) Access Logs Source.
2. Metadata. Click the +Add Field link to add custom log metadata fields. Define the fields you want to associate, each field needs a name (key) and value. The following Fields are to be added in the source:
   1. Add an account field and assign it a value which is a friendly name / alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
   2. Add a region field and assign it the value of respective AWS region where the Load Balancer exists.
   3. Add an accountId field and assign it the value of the respective AWS account id which is being used.
   4. Keep in mind:
      • A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema.
      • An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled, in the Fields table schema. In this case, an option to automatically add or enable the nonexistent fields to the Fields table schema is provided. If a field is sent to Sumo Logic that does not exist in the Fields schema or is disabled it is ignored, known as dropped.

Collect Cloudtrail logs

1. Configure a Classic Load Balancing (CLB) Cloudtrail Logs Source.
2. Metadata. Click the +Add Field link to add custom log metadata fields. Define the fields you want to associate, each field needs a name (key) and value.
   1. Add an account field and assign it a value which is a friendly name / alias to your AWS account from which you are collecting logs. Logs can be queried via the “account field”.
   2. Keep in mind:
      • A green circle with a check mark is shown when the field exists and is enabled in the Fields table schema.
      • An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled, in the Fields table schema. In this case, an option to automatically add or enable the nonexistent fields to the Fields table schema is provided. If a field is sent to Sumo Logic that does not exist in the Fields schema or is disabled it is ignored, known as dropped.

note

Namespace for AWS Classic Load Balancer Service is AWS/ELB.

Field in field schema

1. Classic UI. In the main Sumo Logic menu, select Manage Data > Logs > Fields.
   New UI. In the top menu select Configuration, and then under Logs select Fields. You can also click the Go To... menu at the top of the screen and select Fields.
2. Search for the loadbalancername field.
3. If not present, create it. Learn how to create and manage fields here.

Field Extraction Rule(s)

Create a Field Extraction Rule for AWS Classic Load Balancer access logs and Cloudtrail logs. Learn how to create Field Extraction Rules here.

AWS Classic Load Balancer access logs

Rule Name: AwsObservabilityElbAccessLogsFER
Applied at: Ingest Time
Scope (Specific Data): account=* region=* _sourceCategory=aws/observability/clb/logs

Parse Expression

| parse "* * * * * * * * * * * \"*\" \"*\" * *" as datetime, loadbalancername, client, backend, request_processing_time, backend_processing_time, response_processing_time, elb_status_code, backend_status_code, received_bytes, sent_bytes, request, user_agent, ssl_cipher, ssl_protocol
| parse regex field=datetime "(?<datetimevalue>\d{0,4}-\d{0,2}-\d{0,2}T\d{0,2}:\d{0,2}:\d{0,2}\.\d+Z)"
| where !isBlank(loadbalancername) and !isBlank(datetimevalue)
| "aws/elb" as namespace
| tolowercase(loadbalancername) as loadbalancername
| fields loadbalancername, namespace

AWS Classic Load Balancer CloudTrail Logs

Rule Name: AwsObservabilityCLBCloudTrailLogsFER
Applied at: Ingest Time
Scope (Specific Data): account=* eventSource eventName "elasticloadbalancing.amazonaws.com" "2012-06-01"

Parse Expression

json "eventSource", "awsRegion", "recipientAccountId", "requestParameters.loadBalancerName" as event_source, region, accountid, loadbalancername nodrop 
| where event_source = "elasticloadbalancing.amazonaws.com"
| toLowerCase(loadbalancername) as loadbalancername 
| "aws/elb" as namespace 
| fields region, namespace, loadbalancername, accountid

Install the AWS Classic Load Balancer app

Now that you have set up a collection for AWS Classic Load Balancer, install the Sumo Logic app to use the pre-configured searches and dashboards that provide visibility into your environment for real-time analysis of overall usage.

To install the app:

1. Select App Catalog.
2. In the 🔎 Search Apps field, run a search for your desired app, then select it.
3. Click Install App.
   note

   Sometimes this button says Add Integration.

4. On the next configuration page, under Select Data Source for your App, complete the following fields:
   • Data Source. Select one of the following options:
     • Choose Source Category and select a source category from the list; or
     • Choose Enter a Custom Data Filter, and enter a custom source category beginning with an underscore. For example, _sourceCategory=MyCategory.
   • Folder Name. You can retain the existing name or enter a custom name of your choice for the app.
   • All Folders (optional). The default location is the Personal folder in your Library. If desired, you can choose a different location and/or click New Folder to add it to a new folder.
5. Click Next.
6. Look for the dialog confirming that your app was installed successfully.
   

Post-installation

Once your app is installed, it will appear in your Personal folder or the folder that you specified. From here, you can share it with other users in your organization. Dashboard panels will automatically start to fill with data matching the time range query received since you created the panel. Results won't be available immediately, but within about 20 minutes, you'll see completed graphs and maps.

Viewing the AWS Classic Load Balancer dashboards

Overview

The AWS Classic Load Balancer - Overview dashboard provides visibility into the health of your Classic Load Balancer, with at-a-glance views of latency, request and host status, requests from malicious sources, and HTTP backend codes.

Use this dashboard to:

• Monitor requests to each load balancer to ensure the load is being distributed as desired.
• Monitor trends for load balancers errors, 4xx and 5xx errors, as well as healthy and unhealthy hosts.
• Monitor the current state across all load balancers via active connections, new connections, backend connection errors, and rejected connections.

Response Analysis

The AWS Classic Load Balancer - Response Analysis dashboard provides insights into how your load balancers respond to clients.

Use this dashboard to:

• Monitor incoming client locations for all 5XX, 4XX, and 3XX error responses.
• Quickly correlate error responses using load balancer access logs and AWS CloudWatch metrics to determine the possible cause for failures and decide corrective actions.

Backend Response Analysis

The AWS Classic Load Balancer - Backend Response Analysis dashboard provides insights into how various backend servers respond to client requests.

Use this dashboard to:

• Monitor trends of all response codes for your backend servers by LoadBalancer and availability zones.
• Correlate response code trends across load balancer access logs and CloudWatch metrics to determine the root cause for failures.

Latency Overview

The The AWS Classic Load Balancer - Latency Overview dashboard provides insights into load balancers response times and availability zones, including backend log response times.

Use this dashboard to:

• Monitor response times by load balancer, and availability zone.
• Monitor client latency and processing times for backend servers.

Latency Details

The The AWS Classic Load Balancer - Latency Details dashboard provides insights into client latency by domain and ELB server and processing times by ELB server throughout your infrastructure.

Use this dashboard to troubleshoot load balancer performance via detailed views across client, request processing, and response time latencies.

Connection and Host Status

The AWS Classic Load Balancer - Connection and Host Status dashboard provides insights into active and rejected connections, backend connection errors, and healthy and unhealthy hosts.

Use this dashboard to:

• Monitor active connections, new connections, rejected connections, and connection errors for load balancers.
• Monitor healthy and unhealthy host counts by the load balancer and availability zone across your infrastructure.

Requests and Processed Bytes

The AWS Classic Load Balancer - Requests and Processed Bytes dashboard provides insights into client requests, network traffic, and processed data.

Use this dashboard to:

• Monitor client request load, network traffic, and processed bytes to determine how to configure load balancers for optimal performance best.
• Determine how to allocate best backend resources based on load.

Threat Intel

The AWS Classic Load Balancer - Threat Intel dashboard provides insights into incoming requests from malicious sources determined via Sumo Logic threat intelligence. Dashboard panels show detailed information on malicious IPs and the malicious confidence of each threat.

Use this dashboard to:

• Identify known malicious IPs that are accessing your load-balancers and use firewall access control lists to prevent them from sending you traffic going forward.
• Monitor malicious confidence level for all incoming malicious IP addresses posing the threats.

CloudTrail Audit

The AWS Classic Load Balancer - CloudTrail Audit dashboard provides a comprehensive overview of activities through CloudTrail audit logs. It visualizes successful and failed events globally, event trends, error details, and user activities, offering insights into load balancer performance, security, and usage patterns.

Use this dashboard to:

• Monitor the geographical distribution of successful and failed load balancer events, allowing for quick identification of regions with high activity or potential issues.
• Track the overall success rate of load balancer events and analyze trends over time, helping to identify sudden changes or patterns in the performance.
• Investigate specific error events, including their details, frequency, and associated users, enabling faster troubleshooting and resolution of issues.
• Identify the most common error types and the users experiencing highest failure rates, facilitating targeted improvements and user support.