Skip to main content
Sumo Logic

Lab 2 - Using Explore to Manage Kubernetes Data

Use the Explore view which contains pre-built dashboards that allow you traverse the hierarchy and do cross signal correlation as you monitor, analyze and troubleshoot the Kubernetes environment.

Getting Started

Sumo Logic unifies metadata across the Kubernetes system, providing consistent metadata between logs, messages, events, and security. Metadata enrichment, such as pods, containers, namespaces, deployments, and services, allows users to do cross signal correlation in monitoring, analysis, and troubleshooting - users can move from an event, to a log, to a metric to get the full picture

In this lab, you will learn to use the Explore view which contains pre-built dashboards that allow you traverse the hierarchy and do cross signal correlation as you monitor, analyze and troubleshoot the Kubernetes environment.

Unified and consistent Kubernetes metadata between logs, messages, events and security 

One of the really powerful features is that we can unify the metadata across the Kubernetes system and provide consistent metadata between logs, messages, events, and security. Sumo Logic pumps all collected data through fluentd for metadata enrichment. Kubernetes is rich with metadata, to name a few examples - pods, containers, namespaces, deployments, and services. This allows you to do cross signal correlation both in monitoring, analysis, and troubleshooting. Because of this, you can move from an event, to a log, to a metric because they are all being emitted from the same place, see the following diagram.

Screen Shot 2019-09-04 at 11.10.47 PM.png

In this lab, you take a look at a Kubernetes dashboards viewed through our Explore tab. You will learn how to navigate and observe your consistent metadata from the different views. 

  1. Open an Explore tab. On the Home page, click +New.

     Screen Shot 2020-01-30 at 2.38.14 PM.png

  2. Click Explore (you can also use Alt+g).

  3. In Explore By, make sure Kubernetes Service View is selected. If needed, select Kubernetes Service View.

    Screen Shot 2019-09-05 at 5.31.29 PM.png

  4. What displays is our Kubernetes Service View, with the Kubernetes - API Server dashboard.  The API is the gateway into the master node of the Kubernetes system. This displays information on the API server logs, which is the control plane component that exposes the Kubernetes API. The panels show details on the API server errors, warnings, and activities.

    Using this view, we can monitor the health and performance of the API server, or review any of the following: server request rates, server success/failure request rates, client activity, and server errors for troubleshooting insights.

  5. To drill down into the hierarchy of the service view, click prod01.travellogic.info.
    Screen Shot 2019-09-04 at 11.13.00 PM.png

    Before we continue to navigate down into our hierarchy, you may want to know how to go up in the hierarchy.

  6. To go back up in the hierarchy, click prod01.travellogic.info. Our user interface toggles between expanding down into the hierarchy or going back up with a single click.
    Screen Shot 2019-09-04 at 11.16.54 PM.png

  7. We are at the cluster level, and we want to continue to go down into our hierarchy of the service view, click prod01.travellogic.info, again.

    You will see the next level in the hierarchy called namespace.

    Screen Shot 2019-09-04 at 10.33.29 PM.png
    A namespace provides a scope for names. Names of resources need to be unique within a namespace, but not across namespaces. It is useful for when you need to divide cluster resources between multiple users.

  8. To navigate further into the service view hierarchy, under prod01.travellogic.info cluster, click sumologic.

  9. Since we are in the Kubernetes Service View, all the services under the prod01.travellogic.info cluster, and sumologic namespace are shown.

    Screen Shot 2019-09-05 at 5.42.34 PM.png

  10. Look at the panels as you traverse the hierarchy and if you happen to see No Data To Display it means that there is no data based on the filter conditions. This maybe because there are no errors within that time range. Screen Shot 2019-09-04 at 10.39.02 PM.png

  11. Let’s go and examine this more carefully. Click sumologic and scroll to the bottom to the Events in Namespace panelTo activate the panel, click in the middle of the Events in Namespace panel.

  12. At the top right of the panel, you will see the time range. It is -15m, which defines the range to be the last 15 minutes of data. Let’s widen the range to the last week to see if we had any Events in Namespace. Choose -7d.

  13. From the Relative time options, select Last 7 Days.

    Screen Shot 2019-09-04 at 10.41.19 PM.png


    You will now see a list of tracked events, displaying Events in Namespace. It shows you what events happened in the last 7 days in the sumologic namespace . What you just learned is that although no errors were reported in the last 15 minutes, there were errors in the last week. This is easily shown by changing the panel time range filter. Now, if we had received the No Data To Display again, then that tells us there were no errors in the new time range we defined.

    Screen Shot 2019-09-04 at 10.43.04 PM.png

  14. Now that we traversed the hierarchy from the service view perspective, let's look at the other Explore views and learn more ways to use Explore.

  15. Under Explore By, click Kubernetes Service View, (optionally you can click the drop down arrow). Four different views popup that you can select.
    Screen Shot 2019-09-04 at 10.55.17 PM.png

  16. Click Kubernetes Node View. To the right, in the Dashboard, select from the drop down, Kubernetes - Cluster Overview.

  17. The following dashboard should display. Explore is broken into two aspects. In the left pane of the Explore tab you see the hierarchy of the clusters. Every piece of infrastructure has some type of topography that explains how the components are interconnected. 


Screen Shot 2020-01-30 at 2.52.29 PM.png

  1. On the right side of the Explore tab you can see all the Kubernetes cluster high level statistics monitoring capability, since we are in the Kubernetes - Cluster Overview dashboard.  Also notice that you are able to have panels of metrics, logs, and security all in one view. 
  2. Scroll down to the Pods Running panel. Notice that they are grouped by namespace metadata. The prod-loggen pods are not running, as indicated in red. 

    Screen Shot 2019-09-05 at 5.55.21 PM.png

  3. If you hover over the prod-loggen pods and you will receive more detailed information in a pop-up. 

Screen Shot 2020-01-30 at 3.02.12 PM.png

  1. The pods that are not running are coming from prod-loggen and begin with carbonblack and pagerduty. In a Kubernetes environment the the pods are ephemeral by design for elastic ability to scale and contract. As you hover over the pods notice that in the details the pod name is given if you want to investigate it further, which we will do in another lab.
  2. Also, if we hover over a pod and left mouse click Error Logs, we can analyze its logs from the drop-down list. Optionally, you can also view Related Logs, Recommended Dashboards and Related Explores for the selected pod.

Screen Shot 2020-01-30 at 3.50.20 PM.png

 

Quiz 

  1. The expander in the Explore By pane, changes direction to indicate if I’ve traversed up or down in the hierarchy. 

  2. There is consistent metadata tagging across logs, metrics and events which gives us the ability to look at our Kubernetes system from four views which are namespace, node, service and deployment.

  3. You can both hover over a pod and right mouse click on a pod. 

Summary

Congratulations! You’ve completed these tasks in Part 2 of the Kubernetes Hands-on Labs:

  1. Applied centralized data collection to examine Kubernetes.

  2. Learned how to traverse up and down the hierarchy of a Kubernetes system using the Explore tab.

  3. Learned how to select from our four hierarchical views in the Explore tab.