Skip to main content
Sumo Logic

Install the Kubernetes App, Alerts, and view the Dashboards

This page provides instructions for installing the Kubernetes App and Alerts, as well as descriptions and examples for each of the dashboards. These instructions assume you have already set up the collection as described in the Collect Logs and Metrics for the Kubernetes App page.

Pre-Packaged Alerts

Sumo Logic has provided out of the box alerts available through Sumo Logic monitors to help you quickly determine if the Kubernetes cluster is available and performing as expected. These alerts are built based on metrics datasets and have preset thresholds based on industry best practices and recommendations.

For details on the individual alerts,  please see this page.

Installing Alerts

  • To install these alerts, you need to have the Manage Monitors role capability.
  • Alerts can be installed by either importing them a JSON or a Terraform script.
Method 1: Install the alerts by importing a JSON file:
  1. Download the JSON file describing all the monitors. 
  2. The alerts should be restricted to specific clusters and/or namespaces to prevent the monitors hitting the cardinality limits. To limit the alerts, update the JSON file by replacing the text ‘$$kubernetes_data_source’ with ‘<Your Custom Filter>’, for example: ‘cluster=k8s-prod.01’

  3. Go to Manage Data > Alerts > Monitors.

  4. Click Add:

Add monitors page.png

  1. Click Import to import monitors from the JSON above.

Method 2: Install the alerts using a Terraform script

Step 1: Generate a Sumo Logic access key and ID

Generate an access key and access ID for a user that has the Manage Monitors role capability in Sumo Logic using these instructions. Please identify which deployment your Sumo Logic account is in, using this  link.

Step 2: Download and install Terraform 0.13 or later 

Step 3: Download the Sumo Logic Terraform package for Kubernetes alerts

The alerts package is available in the Sumo Logic github repository. You can either download it through the “git clone” command or as a zip file. 

Step 4: Alert Configuration 

After the package has been extracted, navigate to the package directory terraform-sumologic-sumo-logic-monitor/monitor_packages/kubernetes/

Edit the kubernetes.auto.tfvars file and add the Sumo Logic Access Key, Access Id and Deployment from Step 1 .

access_id   = "<SUMOLOGIC ACCESS ID>"

access_key  = "<SUMOLOGIC ACCESS KEY>"

environment = "<SUMOLOGIC DEPLOYMENT>"

The alerts should be restricted to specific clusters and/or namespaces to prevent the monitors hitting the cardinality limits. To limit the alerts, update the variable ‘kubernetes_data_source’ with your Custom filter, for example: ‘cluster=k8s.prod.01’

All monitors are disabled by default on installation, if you would like to enable all the monitors, set the parameter monitors_disabled to false in this file.

By default, the monitors are configured in a monitor folder called “Kubernetes”, if you would like to change the name of the folder, update the monitor folder name in this file.

If you would like the alerts to send email or connection notifications, configure these in the file kubernetes_notifications.auto.tfvars. For configuration examples, refer to the next section.

Step 5: Email and Connection Notification Configuration Examples

Modify the file kubernetes_notifications.auto.tfvars and populate connection_notifications_critical, connection_notifications_warnings, connection_notifications_missingdata  and email_notifications_critical, email_notifications_warnings, email_notifications_missingdata as per below examples.

Pagerduty Connection Example:
connection_notifications_critical = [
    {
      connection_type       = "PagerDuty",
      connection_id         = "<CONNECTION_ID>",
      payload_override      = "{\"service_key\": \"your_pagerduty_api_integration_key\",\"event_type\": \"trigger\",\"description\": \"Alert: Triggered {{TriggerType}} for Monitor {{Name}}\",\"client\": \"Sumo Logic\",\"client_url\": \"{{QueryUrl}}\"}",
      run_for_trigger_types = ["Critical", "ResolvedCritical"]
    },
    {
      connection_type       = "Webhook",
      connection_id         = "<CONNECTION_ID>",
      payload_override      = "",
      run_for_trigger_types = ["Critical", "ResolvedCritical"]
    }
  ]

Replace <CONNECTION_ID> with the connection id of the webhook connection. The webhook connection id can be retrieved by calling the Monitors API.

For overriding payload for different connection types, refer to this document.

Email Notifications Example:
email_notifications_critiical = [
    {
      connection_type       = "Email",
      recipients            = ["abc@example.com"],
      subject               = "Monitor Alert: {{TriggerType}} on {{Name}}",
      time_zone             = "PST",
      message_body          = "Triggered {{TriggerType}} Alert on {{Name}}: {{QueryURL}}",
      run_for_trigger_types = ["Critical", "ResolvedCritical"]
    }
  ]

Step 6: Install the Alerts

  1. Navigate to the package directory terraform-sumologic-sumo-logic-monitor/monitor_packages/kubernetes/ and run terraform init. This will initialize Terraform and will download the required components.
  2. Run terraform plan to view the monitors which will be created/modified by Terraform.
  3. Run terraform apply.

Step 7: Post Installation

If you haven’t enabled alerts and/or configured notifications through the Terraform procedure outlined above, we highly recommend enabling alerts of interest and configuring each enabled alert to send notifications to other people or services. This is detailed in Step 4 of this document.


Note: There are limits to how many alerts can be enabled - please see the Alerts FAQ.

Install the App

Now that you have set up the collection for Kubernetes App, install the Sumo Logic App for Kubernetes to use the pre-configured Kubernetes dashboards that provide visibility into your Kubernetes environment.

To install the app, do the following:

  1. Locate and install the app from the App Catalog. If you want to see a preview of the dashboards included with the app before installing, click Preview Dashboards.
  2. From the App Catalog, search for Kubernetes and select the app.

  3. To install the app, click Add to Library.

  4. Complete the following fields:

    1. App Name. You can retain the existing name, or enter a name of your choice for the app.


    2. Data Source.  For each the sources listed, enter a Custom Data Filter or Source Category, as follows:

    • For Falco Log Source leave Source Category selected, and enter the following source category: *falco* or one that matches the source categories in your environment. 

    • For Events Log Source  leave Source Category selected, and enter the following source category: *events* or one that matches the source categories in your environment.

    1. Advanced. Select the location in the Library (the default is the Personal folder in the Library), or click New Folder to add a new folder.

  5. Click Add to Library.

Filter with template variables   

Template variables provide dynamic dashboards that can rescope data on the fly. As you apply variables to troubleshoot through your dashboard, you view dynamic changes to the data for a quicker resolution to the root cause. For more information, see the Filter with template variables help page.

Kubernetes - Cluster Explorer Dashboard

The Kubernetes - Cluster Explorer dashboard provides a high-level view of the health of the cluster services, along with details on the utilized resources by service.

Use this dashboard to:

  • Navigate the cluster topology
  • Review the memory and CPU usage by cluster and service components.

K8s_Cluster_Explorer.png

Kubernetes – Cluster Dashboard 

The Kubernetes - Cluster dashboard provides detailed status of the cluster health, along with details on all the components, resources and related entities. 

Use this dashboard to: 

  • Monitor overall cluster health.
  • Get insight into the state and resource usage of cluster components and use this information to fine-tune your Kubernetes cluster. 
  • Get quick insights into the state of the related entities.

Kubernetes - Cluster Overview Dashboard

The Kubernetes - Cluster Overview dashboard provides a high-level view of the cluster health.. 

Use this dashboard to: 

  • Get quick insights into the health of the cluster.
  • View top resource intensive components and use this information to fine tune your cluster.

 K8s_Cluster_Overview.png

Kubernetes - Node Dashboard

The Kubernetes - Node dashboard provides detailed information on the health and performance of nodes in a Kubernetes cluster. 

Use this dashboard to:

  • Monitor node health.
  • Get insight  into how resources are being used across nodes and fine-tune node configurations accordingly.
  • Investigate potential issues with nodes.

K8s_Nodes.png

Kubernetes - Node Overview Dashboard 

The Kubernetes - Node Overview dashboard provides a high-level view of a node, along with details on all the related components and resources. 

Use this dashboard to: 

  • Get quick insights into the health of the node. 
  • View top resource intensive components and use this information to fine tune your node.

Kubernetes - Namespace Dashboard

The Kubernetes - Namespace dashboard provides insights into the health and resource utilization of a namespace. 

Use this dashboard to: 

  • Monitor namespace health. 
  • Get insight into the components of a namespace and how resources are being used across namespaces and fine-tune configurations accordingly. 
  • Investigate potential issues.

K8s_Namespace_Overview.png

Kubernetes - Pod Dashboard

The Kubernetes - Pod dashboard provides insights into the health of and resource utilization of a Kubernetes pod. 

Use this dashboard to: 

  • Monitor pod health. 
  • Get insight into the components of a pod and how resources are being used across namespaces and fine-tune configurations accordingly. 
  • Investigate potential issues.

K8s_Pods.png

Kubernetes - Container Dashboard

The Kubernetes - Container dashboard provides insights into the health and resource utilization of a Kubernetes container. 

Use this dashboard to: 

  • Monitor container health. 
  • Get insight into container resource utilization and fine-tune configurations accordingly. 
  • Determine if containers are stuck in CrashLoopBackOff, Terminated or Waiting states and make necessary adjustments. 
  • Investigate containers that are over-utilizing resources.

K8s_Containers.png

Kubernetes - Daemonsets Overview Dashboard

The Kubernetes - Daemonsets Overview dashboard provides insights into the health of and resource utilization of Kubernetes Daemonsets. 

Use this dashboard to: 

  • Monitor the health of Daemonsets.  
  • Identify whether the required replica level is achieved or not. 
  • View logs and errors and investigate potential issues.

Kubernetes - StatefulSets Overview Dashboard 

The Kubernetes - StatefulSets Overview dashboard provides insights into the health of and resource utilization of Kubernetes StatefulSets. 

Use this dashboard to: 

  • Monitor the health of StatefulSets.  
  • Identify whether the required replica level is achieved or not.
  • View logs and errors and investigate potential issues.

K8s_Daemonsets_StatefulSets.png

Kubernetes - Deployment Overview Dashboard

The Kubernetes - Deployment Overview dashboard provides insights into the health and performance of your Kubernetes deployments. 

Use this dashboard to: 

  • Monitor the health of deployments in your Kubernetes environment.  
  • Identify whether the required replica level has been achieved or not. 
  • View logs and errors and investigate potential issues.

K8s_Deployment_Overview.png

Kubernetes - Health Check

The Kubernetes - Health Check dashboard displays the collection status from all the components in the Kubernetes cluster.

Use this dashboard to: 

  • Monitor the health of FluentD and FluentBit pods in your Kubernetes environment.
  • Gain insights into Prometheus metric collection endpoint status.
  • Get insight into resource utilization and fine-tune configurations accordingly.
  • View logs and errors and investigate potential issues.

clipboard_e3bfbf91847de6daa52b18fc73edc105e.png

Kubernetes - Deployment Dashboard 

The Kubernetes - Deployment dashboard provides insights into the health and performance of your Kubernetes deployments. 

Use this dashboard to: 

  • Monitor the health of deployments in your Kubernetes environment.  
  • Identify whether the required replica level has been achieved or not. 
  • View logs and errors and investigate potential issues.

Kubernetes - Security Overview Dashboard

The Kubernetes - Security Overview dashboard provides high level details around anomalous container, application, host, and network activity detected by Falco. 

Use this dashboard to: 

  • Identify and investigate anomalous activity.

 

Kubernetes - Security Rules Triggered Dashboard

The Kubernetes - Security Rules Triggered dashboard provides detailed information around anomalous activity detected by Falco. It also shows information around the OOB Falco rules triggered by anomalous activity in your Kubernetes environments.

Use this dashboard to:

  • Reviewed detailed information of anomalous activity.
  • Review if the OB Falco security events are triggered and identify the root cause.

K8s_Sec_Rules_Triggered.png

Kubernetes - Service  Dashboard

The Kubernetes - Service dashboard provides a high-level view of the health of the cluster services, along with details on utilized resources by service. 

Use this dashboard to: 

  • Reviewed detailed information of services. 
  • Identify components by Services. 
  • Determine any errors and warnings by Services.

K8s_Service_Overview.png

Kubernetes - Hygiene Check Dashboard

The Kubernetes - Hygiene Check dashboard provides visibility into the configuration hygiene of your Kubernetes cluster. This dashboard displays color-coded performance checks for nodes, along with resource utilization, pod capacity, pod errors, and pod states. 

Use this dashboard to: 

  • Assess bad configurations and determine the trouble areas for proactive adjustment.  
  • Monitor resource allocation across your cluster to maintain optimum performance.

K8s_Health_Check.png

Kubernetes - CoreDNS

CoreDNS is a DNS server and can be used as a replacement for kube-dns in a kubernetes cluster.

The Kubernetes - CoreDNS dashboard provides visibility into the health and performance of CoreDNS.  

Use this dashboard to: 

  • Track the total number of requests. 
  • Review Cache statistics. 
  • Monitor CoreDNSs resource usage and spikes.

K8s_DPM.png

Kubernetes – HPA Dashboard 

The Horizontal Pod Autoscaler automatically scales the number of Pods in a replication controller, deployment, replica set or stateful set based on observed CPU utilization.

The Kubernetes - HPA dashboard provides visibility into the health and performance of HPA.  

Use this dashboard to: 

  • Identify whether the required replica level has been achieved or not. 
  • View logs and errors and investigate potential issues.