Skip to main content
Sumo Logic

Lab 1 - Modifying a Kubernetes Collector

Getting Started

When analyzing Kubernetes data, Sumo Logic allows users to label their logs with their own custom metadata using key-value pairs, making it easier for users to search through their logs. In this lab, you will learn how to configure and use key-value pairs with Kubernetes data. We will also cover how you can access/display any of your Kubernetes labels using our field browser these fields from your query results.

Adding key value pairs to collectors, and examining metadata

  1. In the left navigation pane, Click Manage Data.

    Right below that, Click Collection, which opens up all data collectors available in the system.

    Screen Shot 2019-09-04 at 3.00.18 PM.png

    Look at the 4th column labeled Source Category, and notice that typically the metadata naming convention has a hierarchical structure, such as sec/aws/elb and sec/aws/vpc/flows. This has been our recommended best practice, to start off general and become more specific.  It requires that your familiarity with how your environment has been set up. 

  2. Now, with our Kubernetes app, you have the ability to add fields to define key value pairs at the source level. You can search for them, use them for partition definition, or use them in RBAC queries.

  3. Let’s go look at how to do this. First we bring up our lab Kubernetes source. At the top of the Collection tab you will see a magnifying glass to search. Go to the right of the magnifying glass,  type Labs - Kubernetes.

  4. Click on Labs - Kubernetes. A popup window displays.
    Screen Shot 2019-09-04 at 3.53.29 PM.png

  5. To the right of Fields, click + Add Field 
  6. In the first field box, enter user###In the second field box, enter username### (where ### is the numbers from your assigned username). 
    EditCollector.png
  7. Because this is a training environment and there are so many of us, we won't actually add them. Custom metadata is expected to be set up by your administrator, usually once and done. Please cancel out of this window, click Cancel.

Key value pairs on a Kubernetes collector can be used in searches, partitions, and Role Based Access Control (RBAC) queries. This can be very helpful. Now any log data that comes in to this collector will automatically inherit the key value pairs. You can label anything that is coming in and you are not limited to _sourceCategory.  As a Kubernetes user, you have a mental model of what’s going on and you have naming conventions associated with your architectures.  We enable you with a whole new way of approaching your container based data.

Working with Metadata

Kubernetes is rich with metadata. Now you can search by the parts of Kubernetes you want to investigate. You can search by containers, pods, you can investigate what namespaces you are looking at. Metadata set with key value pairs, makes it very easy to find your log data. We will also cover how you can access/display any of your Kubernetes labels using our field browser these fields from your query results.

To look at these Kubernetes components using metadata, do the following:

  1. To open a query, on the Home page, click +New.

    NewExplore.png
  2. Select Log Search. 
  1. Indicate the metadata namespace, and enter namespace=sumologic.

  2. To run the query, click Start.
    image11.png

  3.  From the Messages tab under Hidden Fields, enable namespace. With our Kubernetes app, the metadata rich environment is populated in Hidden fields can be easily displayed in the Display Fields. 

  4. You can also easily access your Kubernetes labels to obtain any key value pair. If you want to look at a Prometheus container, type container=prometheus.

  5. We don't know exactly when the problem occurred, but we suspect that it was in the last hour. Let's change from the last 15 min to the last 60 min. Click -15m and select Last 60 Minutes
    Screen Shot 2019-09-05 at 5.01.31 PM.png

  6. Using metadata, is an easy way of quickly accessing what you need and finding particular Kubernetes log file data. You can further investigate this container by grouping common messages with logreduce, click Logreduce. This will group common log message into signature groupings. 

  7. Often times when troubleshooting a problem, our lesser quantity contains the root cause. Now, let’s examine the details of the smaller set of signatures that appear under Select Count. Under Select Count, click 1.

    Screen Shot 2019-09-04 at 10.04.56 PM.pngNotice we have a warning indicating Endpoints ended with: too old resource version. This may be something to possibly investigate or just the indication of an ongoing upgrade.

  8. We are currently observing only the Time and Message fields. To check things out further, you can display the Kubernetes components by enabling them by selecting the box, select the box to the left of the following: namespace, cluster, container, pod, service and Source Host.

Quiz

  1. I can add key value pairs to our Kubernetes cluster that can be used in searches, partitions, and RBAC queries. 

  2. I can display a hidden field key value pair, by clicking on my personal folder. 

  3. I can look at a Prometheus container by typing container=prometheus in a query and click Start.

Summary

Congratulations! You’ve completed these tasks in Part 1 of the Kubernetes Hands-on Labs:

  1. Signed into Sumo Logic.

  2. Added a key value pair to a Kubernetes collector

  3. Used the key value pair in a query.

  4. Learned how to enable viewing metadata in the Field Browser from Hidden Fields.