Skip to main content
Sumo Logic

Lab 1 - Using metadata effectively in Kubernetes

Getting Started

When analyzing Kubernetes data, Sumo Logic allows users to label their logs with their own custom metadata using key-value pairs, making it easier for users to search through their logs. In this lab, you will learn how to configure and use key-value pairs with Kubernetes data. We will also cover how you can access/display any of your Kubernetes labels using our field browser these fields from your query results.

Adding key value pairs to collectors, and examining metadata

  1. In the left navigation pane, Click Manage Data.

    Right below that, Click Collection, which opens up all data collectors available in the system.

    Screen Shot 2019-09-04 at 3.00.18 PM.png

    Look at the 4th column labeled Source Category, and notice that typically the metadata naming convention has a hierarchical structure, such as sec/aws/elb and sec/aws/vpc/flows. This has been our recommended best practice, to start off general and become more specific.  It requires that your familiarity with how your environment has been set up. 

  2. Now, with our Kubernetes app, you have the ability to add fields to define key value pairs at the source level. You can search for them, use them for partition definition, or use them in RBAC queries.

  3. Let’s go look at how to do this. First we bring up our lab Kubernetes source. At the top of the Collection tab you will see a magnifying glass to search. Go to the right of the magnifying glass,  type Labs - Kubernetes.

  4. Click on Labs - Kubernetes. A popup window displays.
    Screen Shot 2019-09-04 at 3.53.29 PM.png

  5. To the right of Fields, click + Add Field 
  6. In the first field box, enter user###In the second field box, enter username### (where ### is the numbers from your assigned username). 
  7. Because this is a training environment and there are so many of us, we won't actually add them. Custom metadata is expected to be set up by your administrator, usually once and done. Please cancel out of this window, click Cancel. We will use previously assigned key value pairs in the next lab so you can see the effect.

Key value pairs on a Kubernetes collector can be used in searches, partitions, and Role Based Access Control (RBAC) queries. This can be very helpful. Now any log data that comes in to this collector will automatically inherit the key value pairs. You can label anything that is coming in and you are not limited to _sourceCategory.  As a Kubernetes user, you have a mental model of what’s going on and you have naming conventions associated with your architectures.  We enable you with a whole new way of approaching your container based data.

Working with Metadata

Kubernetes is rich with metadata. Now you can search by the parts of Kubernetes you want to investigate. You can search by containers, pods, you can investigate what namespaces you are looking at. Metadata set with key value pairs, makes it very easy to find your log data. We will also cover how you can access/display any of your Kubernetes labels using our field browser these fields from your query results.

To look at these Kubernetes components using metadata, do the following:

  1. To open a query, on the Home page, click +New.

  2. Select Log Search. 
  1. Indicate the metadata namespace, and enter namespace=sumologic. This is Kubernetes key value pair that is very similar to the collector key value pairs in the previous lab. We will use them to filter on ingested log messages that contain namespace=sumologic in them.

  2. To run the query, click Start.

  3.  From the Messages tab under Hidden Fields, enable namespace. With our Kubernetes app, the metadata rich environment is populated in Hidden fields can be easily displayed in the Display Fields. 

  4. You can also easily access your Kubernetes labels to obtain any key value pair. If you want to look at a Prometheus container, type container=prometheus.

  5. We are currently observing only the Time and Message fields. To check things out further, you can display the Kubernetes components by enabling them in the Display Fields, by selecting the box to the left of the following: namespace, cluster, container, pod, service and Source Host.Screen Shot 2020-04-15 at 4.29.15 PM.png

  6. Take a look at the container column and you will see that all the log results use the container=prometheus key value pair. This is because we are using the key value pair as a filter on any incoming logs. So we are only seeing the logs that have container=prometheus.

  7. Key value pairs are an effective way of receiving just those logs you're interested in without having to know the exact data source pointer, such as the Source Category as defined in the 4th column of Managed Data> Collection>Collection.

Screen Shot 2020-04-15 at 4.52.16 PM.png

Quiz (True or False?)

  1. I can add key value pairs to our Kubernetes cluster that can be used in searches, partitions, and RBAC queries. 

  2. I can display a hidden field key value pair, by clicking on my personal folder. 

  3. I can look at a Prometheus container by typing container=prometheus in a query and click Start.


Congratulations! You’ve completed these tasks in Part 1 of the Kubernetes Hands-on Labs:

  1. Signed into Sumo Logic.

  2. Added a key value pair to a Kubernetes collector

  3. Used the key value pair in a query.

  4. Learned how to enable viewing metadata in the Field Browser from Hidden Fields.