Add a New AWS Service to the AWS Observability Explore Hierarchy

1. Enrich Metrics Data

The following are a list of steps for adding metrics-based dashboards to the hierarchy:

1. Add the account field as a metadata tag to the AWS CloudWatch metrics source. 
2. Validate the namespace and region metadata tags.
3. Identify the AWS Resource Name field.
4. Update the existing hierarchy.
5. Validate the updated hierarchy.

2. Enrich Log Data

The following is a list of required steps for ensuring log data appears as expected in the hierarchy:

1.  Add the account field to log data.
2. Add namespace, region, and the AWS resource name as metadata tags to log data using field extraction rule.

 3. Modify dashboards 

The following is a list of requires steps to modify the dashboards:

1. Apply Stack Linking
2. Add template variables (Optional)
3. Improve queries to include variables in log queries (Optional)
4. Improve queries to include variables in metric queries (Optional)

Add the account field as a metadata tag to the AWS CloudWatch metrics source

The Account field is already a part of CloudWatch metrics sources that were configured as part of the AWS Observability Solution’s CloudFormation template. Therefore the Account metadata field needs to be added to the CloudWatch metrics source using Fields configuration.

This can be done by following the steps below:

1. Log in to your Go to Sumo Logic account
2. In the AWS Observability solution, identify the account alias for the AWS account you have configured that is running the service you want to monitor
3. Edit the CloudWatch Metrics source for the AWS service you wish to add to the AWS Observability solution
4. Add Account field as by adding a field as shown in the screenshot below:
   
5. To confirm if the account tag is indeed added as metadata, go to your Sumo Logic AWS Cloudwatch Metric source and check the metrics data.
   

Validate the namespace and region metadata tags 

The namespace and region tags are generally present in AWS CloudWatch metrics when collecting metrics using the Sumo Logic AWS CloudWatch metrics source. 

For the desired AWS Service, go to your Sumo Logic AWS Cloudwatch Metric source and check the metric data.

1. Go to Sumo Logic account
2. Open a metrics tab and run a query to get metrics data for the AWS Service you wish to add to AWS Observability
3. Check the data in the Legend tab and ensure namespace and region metadata tags are present. If these are not present, you will not be able to add this service to the AWS Observability solution.

Identify the AWS Resource Name field

An AWS Resource Name metadata field represents an instance of the AWS service like table name, load balancer name, database instanceId, database clusterId, API name, function name etc,. For AWS service you wish to monitor, identify the field in the metrics data that can uniquely represent the AWS service.

For Example, If we wish to monitor AWS/SQS service, then QueueName can be used as the AWS Resource name.

Update the existing hierarchy

Once all the tags are checked and identified in the metrics, we can update the existing hierarchy to show the new AWS Service. Follow the steps below to update the existing hierarchy :

1. Run the below curl command to get the existing AWS Observability hierarchy.

curl -s -H "Content-Type: application/json" --user 
"<ACCESS_ID>:<ACCESS_KEY>" -X GET 
https://<SUMOLOGIC_URL>/api/v1/entities/hierarchies | json_pp 
-json_opt pretty,canonical | grep -B 80 "\"AWS Observability\"" | 
grep "id" | head -1 | awk -F":" '{ print $2}' | tr -cd '[:digit:]' | 
xargs -I {} curl -s -H "Content-Type: application/json" --user 
"<ACCESS_ID>:<ACCESS_KEY>" -X GET 
https://<SUMOLOGIC_URL>/api/v1/entities/hierarchies/{} | json_pp -json_opt pretty

The output of the command will look something like below (it is trimmed output, the actual output can vary as per your hierarchy)

{
  "filter": null,
  "id": "0000000000000278",
  "level": {
    "nextLevel": {
      "nextLevel": {
        "nextLevelsWithConditions": [
          {
            "condition": "AWS/ApplicationElb",
            "level": {
              "entityType": "loadbalancer",
              "nextLevelsWithConditions": [],
              "nextLevel": null
            }
          },
          {
            "condition": "AWS/ApiGateway",
            "level": {
              "nextLevel": null,
              "entityType": "apiname",
              "nextLevelsWithConditions": []
            }
          }
        ],
        "entityType": "namespace",
        "nextLevel": null
      },
      "nextLevelsWithConditions": [],
      "entityType": "region"
    },
    "nextLevelsWithConditions": [],
    "entityType": "account"
  },
  "name": "AWS Observability"
}

2. If the AWS Service namespace is not present in the output JSON, update the JSON as below. We are taking AWS/SQS as an example here.

{
  "filter": null,
  "id": "0000000000000278",
  "level": {
    "nextLevel": {
      "nextLevel": {
        "nextLevelsWithConditions": [
          {
            "condition": "AWS/ApplicationElb",
            "level": {
              "entityType": "loadbalancer",
              "nextLevelsWithConditions": [],
              "nextLevel": null
            }
          },
          {
            "condition": "AWS/ApiGateway",
            "level": {
              "nextLevel": null,
              "entityType": "apiname",
              "nextLevelsWithConditions": []
            }
          },
          {
            "condition": "AWS/SQS", -> Namespace
            "level": {
              "nextLevel": null,
              "entityType": "queuename", -> AWS Resource Name identified in Step 1.3
              "nextLevelsWithConditions": []
            }
          }
        ],
        "entityType": "namespace",
        "nextLevel": null
      },
      "nextLevelsWithConditions": [],
      "entityType": "region"
    },
    "nextLevelsWithConditions": [],
    "entityType": "account"
  },
  "name": "AWS Observability"
}

3. Update the hierarchy using the below command.

curl -s -H "Content-Type: application/json" --user 
"<ACCESS_ID>:<ACCESS_KEY>" -X PUT 
https://<SUMOLOGIC_URL>/api/v1/entities/hierarchies/<ID> -d 
'<JSON_CONTENT_AFTER_UPDATE>'

Validate the new Hierarchy

Once you are done with the above steps, the AWS service will be added to the AWS Observability view hierarchy. To validate this:

1. Go To Explorer in your Sumo Logic account.
2. Select AWS Observability from the drop-down.
3. You should be able to see the new service in the hierarchy represented by namespace/entity.

Add account field to log data

Logs from AWS services are collected into Sumo Logic via Amazon S3, AWS Elastic Load Balancing, Amazon Cloudfront, AWS Cloudtrail, Amazon S3 Audit, or HTTP Log source (Cloudwatch logs). You can add metadata fields to sources using Fields configuration. 

Add account field by adding fields to your log source as shown below:

Add the namespace and region metadata tags to log data using Field Extraction Rule

To enrich the logs data with namespace, region, and aws resource name, we will create a Field Extraction Rule that will add metadata to the logs.

We will take AWS/SQS as an example. For SQS, we selected QueueName as our resource name in metrics data. We will create below FER to extract region, namespace, and queuename from cloudtrail logs.

Name: AwsObservabilitySqsFieldExtractionRule

Add Stack Linking to Dashboards

To add any dashboard to the hierarchy, perform the below steps:

1. Go to your dashboard in the Sumo Logic account. 
2. Select Create Stack Linking as per the below screenshot.
   
3. In the pop-up, add the fields shown below to make it part of AWS Observability in the hierarchy.

   Key	Value
   account	*
   region	*
   namespace	<namespace of aws service>  Example for SQS Service provide value as aws/sqs
   AWS Resource name	* <For example, if you were to use SQS, the Key here will be “queuename”>

Consider the example of an SQS dashboard:  

1. Add dashboard at the namespace level.

   • Add account, region, namespace in stack linking.
     

   • Go to AWS Observability view to look at the dashboard on namespace level.
     
2. Add dashboard at queuename level.

   • Add account, region, namespace, and queuename in stack linking.
     

   • Go to AWS Observability view to look at the dashboard on queuename level
     

Add Template Variables (Optional)

You can add template variables to dashboards to better filter your data in the AWS Observability view.

Follow the steps to add variables to the dashboards :

1. Go to the dashboard.
2. Click + button near the ”Create a template variable” text.
   
3. Add a template variable as shown below:
   

Refer to this document for further details on how to use filters with template variables.

Improve Queries to include variables in log queries (Optional)

Write log queries to include variables so data can be filtered in the Explore view. See the Sumo Logic help doc for details on how to use template variables in log queries.

For example:

account={{account}} namespace={{namespace}} region={{region}} queuename={{queuename}} “sqs.amazonaws.com” 
| json "eventName", "awsRegion", "requestParameters.queueName", "sourceIPAddress", "userIdentity.userName" as event_name, Region, queuename, src_ip, user
| count by event_name

Improve queries to include variables in metric queries (Optional)

You can add the created template variable to queries to better filter the data. Please see this document help doc for details

You can add a variable to your queries by using

<key name> = {{variable name}}

For example:

namespace=aws/sqs metric=NumberOfMessagesSent 
account={{account}}  region={{region}} queuename={{queuename}} | avg by QueueName, account, region, namespace