Skip to main content
Sumo Logic

AWS Lambda - Python function instrumentation with Sumo Logic tracing

This document covers how to install and configure OpenTelemetry distributed tracing for AWS Lambda functions written in Python and send data to Sumo Logic. To obtain telemetry data from AWS Lambda functions developed in Python language you can use the Sumo Logic AWS OTel Python Lambda.

Sumo Logic AWS OTel Lambda supports:

  • Python 3.8 and Python 3.9 runtimes
  • x86_64 and arm64 architectures

Sumo Logic AWS OTel Lambda layer

Sumo Logic AWS OTel Lambda Layer provides packed OpenTelemetry Python libraries that automatically instrument Lambda functions. The biggest advantage of installing Sumo Logic OTel AWS as a layer is disabling/enabling instrumentation of the Lambda function without changing the code.

Requirements

It is very simple to instrument your AWS Python Lambda function using the Sumo Logic AWS Distro Lambda layer. You'll need the following:

  • Python 3.8 or Python 3.9
  • Lambda layers add permissions
  • HTTP Traces Source endpoint URL
  1. Navigate to functions in the AWS Lambda Console and open the function you want to instrument.

  2. Navigate to the Layers section and click Add a layer.

  3. In the Choose a layer menu, select Specify an ARN and paste the ARN ID for your Lambda function AWS Region. Reference the table at the end of this document for the ARN ID.
    clipboard_e33cc151156446f9a4e1c6612546766e3.png

  4. Ensure the AWS Distro layer is present in the Layers section:

    clipboard_ea062eafe23be89cb167264b5d79a61fe.png

  5. Navigate to the Configuration > Environment variables section and set up the following environment variables:

    • AWS_LAMBDA_EXEC_WRAPPER = /opt/otel-instrument enables auto-instrumentation.

    • OTEL_TRACES_SAMPLER = always_on - enables traces sampling.

    • OTEL_SERVICE_NAME = YOUR_SERVICE_NAME - Ensure you define it as a string value that represents the function name and its business logic such as "Check SQS Lambda". This will appear as the tracing service name in Sumo Logic.

    • Tracing application and cloud.account.id are set with the OTEL_RESOURCE_ATTRIBUTES environment variable.

      • application=YOUR_APPLICATION_NAME - the string value, if the function is a part of complex system/application then set it for all other functions/applications.

      • cloud.account.id=YOUR_CLOUD_ACCOUNT_ID - set an additional tag that will contain your AWS Lambda Account ID. This will help to provide more relevant data. 

        All of the attributes above are comma separated key/value pairs (this is also a way to add additional information to the spans, just after comma add additional key=value pair) such as, OTEL_RESOURCE_ATTRIBUTES=application=YOUR_APPLICATION_NAME,cloud.account.id=123456789012.

    • SUMOLOGIC_HTTP_TRACES_ENDPOINT_URL has to be set to send all gathered telemetry data to Sumo Logic. The URL comes from an HTTP Traces Endpoint URL. You can use an existing Source or create a new one if needed.

      clipboard_e1644794a1020f81d821dfdad62ffd80f.png

  6. Make sure you have X-Ray Tracing disabled in Lambda API Stage. Navigate to AWS API Gateway console, find your API and go to Stages. In the Logs/Tracing tab unchek "Enable X-Ray Tracing" option.
    clipboard_e7eb7ebab8b79b7c4cc3f3e1b2ac33663.png
    Note: If for whatever reason you can't disable this, configure X-Ray context propagation by setting OTEL_PROPAGATORS=xray environment variable on your client side.

  7. Your function should be successfully instrumented. Invoke the function and find your traces in the Sumo Logic Tracing screen.

Sumo Logic AWS Distro Lambda layers for AWS Region - amd64 (x86_64) architecture

Click to go back to Step 3.

AWS Region ARN
US East (N.Virginia) us-east-1 arn:aws:lambda:us-east-1:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
US East (Ohio) us-east-2 arn:aws:lambda:us-east-2:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
US West (N.Carolina) us-west-1 arn:aws:lambda:us-west-1:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:2
US West (Oregon) us-west-2 arn:aws:lambda:us-west-2:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
Africa (Cape Town) af-south-1 arn:aws:lambda:af-south-1:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
Asia Pacific (Hong Kong) ap-east-1 arn:aws:lambda:ap-east-1:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
Asia Pacific (Mumbai) ap-south-1 arn:aws:lambda:ap-south-1:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
Asia Pacific (Osaka) ap-northeast-3 arn:aws:lambda:ap-northeast-3:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
Asia Pacific (Seoul) ap-northeast-2 arn:aws:lambda:ap-northeast-2:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
Asia Pacific (Singapore) ap-southeast-1 arn:aws:lambda:ap-southeast-1:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
Asia Pacific (Sydney) ap-southeast-2 arn:aws:lambda:ap-southeast-2:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
Asia Pacific (Tokyo) ap-northeast-1 arn:aws:lambda:ap-northeast-1:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
Canada (Central) ca-central-1 arn:aws:lambda:ca-central-1:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
Europe (Frankfurt) eu-central-1 arn:aws:lambda:eu-central-1:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:4
Europe (Ireland) eu-west-1 arn:aws:lambda:eu-west-1:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
Europe (London) eu-west-2 arn:aws:lambda:eu-west-2:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
Europe (Milan) eu-south-1 arn:aws:lambda:eu-south-1:663229565520:layer:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
Europe (Paris) eu-west-3 arn:aws:lambda:eu-west-3:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
Europe (Stockholm) eu-north-1 arn:aws:lambda:eu-north-1:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:4
Middle East (Bahrain) me-south-1 arn:aws:lambda:me-south-1:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:4
South America (Sao Paulo) sa-east-1 arn:aws:lambda:sa-east-1:663229565520:layer:sumologic-otel-python-x86_64-ver-1-11-1:3
Sumo Logic AWS Distro Lambda layers for AWS Region - arm64 (arm) architecture

Click to go back to Step 3.

AWS Region ARN
US East (N.Virginia) us-east-1 arn:aws:lambda:us-east-1:663229565520:layer:sumologic-otel-python-arm64-ver-1-11-1:3
US East (Ohio) us-east-2 arn:aws:lambda:us-east-2:663229565520:layer:sumologic-otel-python-arm64-ver-1-11-1:3
US West (Oregon) us-west-2 arn:aws:lambda:us-west-2:663229565520:layer:sumologic-otel-python-arm64-ver-1-11-1:3
Asia Pacific (Mumbai) ap-south-1 arn:aws:lambda:ap-south-1:663229565520:layer:sumologic-otel-python-arm64-ver-1-11-1:4
Asia Pacific (Osaka) ap-northeast-3 arn:aws:lambda:ap-northeast-3:663229565520:layer:sumologic-otel-python-arm64-ver-1-11-1:4
Asia Pacific (Singapore) ap-southeast-1 arn:aws:lambda:ap-southeast-1:663229565520:layer:sumologic-otel-python-arm64-ver-1-11-1:4
Asia Pacific (Sydney) ap-southeast-2 arn:aws:lambda:ap-southeast-2:663229565520:layer:sumologic-otel-python-arm64-ver-1-11-1:3
Asia Pacific (Tokyo) ap-northeast-1 arn:aws:lambda:ap-northeast-1:663229565520:layer:sumologic-otel-python-arm64-ver-1-11-1:3
Europe (Frankfurt) eu-central-1 arn:aws:lambda:eu-central-1:663229565520:layer:sumologic-otel-python-arm64-ver-1-11-1:4
Europe (Ireland) eu-west-1 arn:aws:lambda:eu-west-1:663229565520:layer:sumologic-otel-python-arm64-ver-1-11-1:3
Europe (London) eu-west-2 arn:aws:lambda:eu-west-2:663229565520:layer:sumologic-otel-python-arm64-ver-1-11-1:4

Sumo Logic AWS OTel Lambda container instrumentation

Sumo Logic AWS OTel Lambda also provides packed OpenTelemetry Python libraries for container based Lambda functions. 

Requirements

Instrumentation of container based AWS Lambda function requires some changes in the Dockerfile and image rebuild. You'll need the following:

  • Docker
  • Python 3.8 or Python 3.9
  • HTTP Traces Source endpoint URL
  1. Download and extract Sumo Logic AWS OTel Lambda archive with instrumentation packages specific for your architecture - amd64 (x86_64) or arm64.

  2. Extracted instrumentation libraries have to be added to the image in /opt directory. Please see Dockerfile example:

    FROM public.ecr.aws/lambda/python:3.8-arm64

    # Lambda Function Code
    COPY lambda_function.py ${LAMBDA_TASK_ROOT}
    COPY requirements.txt  .
    RUN  pip3 install -r requirements.txt --target "${LAMBDA_TASK_ROOT}"

    # Copy OT Instrumentation
    COPY collector-config/ /opt/collector-config/
    COPY extensions/ /opt/extensions/
    COPY python/ /opt/python/
    COPY otel-instrument /opt/

    CMD [ "lambda_function.lambda_handler" ]

  3. Rebuild docker image.
  1. Navigate to functions in the AWS Lambda Console and open the function you want to instrument.

  2. Deploy new function image.

  3. Navigate to the Configuration > Environment variables section and set up the following environment variables:

    • AWS_LAMBDA_EXEC_WRAPPER = /opt/otel-instrument enables auto-instrumentation.

    • OTEL_TRACES_SAMPLER = always_on - enables traces sampling.

    • OTEL_SERVICE_NAME = YOUR_SERVICE_NAME - Ensure you define it as a string value that represents the function name and its business logic such as "Check SQS Lambda". This will appear as the tracing service name in Sumo Logic.

    • Tracing application and cloud.account.id are set with the OTEL_RESOURCE_ATTRIBUTES environment variable.

      • application=YOUR_APPLICATION_NAME - the string value, if the function is a part of complex system/application then set it for all other functions/applications.

      • cloud.account.id=YOUR_CLOUD_ACCOUNT_ID - set an additional tag that will contain your AWS Lambda Account ID. This will help to provide more relevant data. 

        All of the attributes above are comma separated key/value pairs (this is also a way to add additional information to the spans, just after comma add additional key=value pair) such as, OTEL_RESOURCE_ATTRIBUTES=application=YOUR_APPLICATION_NAME,cloud.account.id=123456789012.

    • SUMOLOGIC_HTTP_TRACES_ENDPOINT_URL has to be set to send all gathered telemetry data to Sumo Logic. The URL comes from an HTTP Traces Endpoint URL. You can use an existing Source or create a new one if needed.

      clipboard_e1644794a1020f81d821dfdad62ffd80f.png

  4. Make sure you have X-Ray Tracing disabled in Lambda API Stage. Navigate to AWS API Gateway console, find your API and go to Stages. In the Logs/Tracing tab unchek "Enable X-Ray Tracing" option.
    clipboard_e7eb7ebab8b79b7c4cc3f3e1b2ac33663.png
    Note: If for whatever reason you can't disable this, configure X-Ray context propagation by setting OTEL_PROPAGATORS=xray environment variable on your client side.

  5. Your function should be successfully instrumented. Invoke the function and find your traces in the Sumo Logic Tracing screen.