Skip to main content
Sumo Logic

Lab 1 - Monitoring user activity

Learn basic operators to parse and group your search results.

Screen Shot 2020-08-05 at 4.41.24 PM.pngMonitoring user activity helps you understand what they are doing in your AWS cloud and how they are using the environment.  By monitoring user activity you may be able to isolate potential malicious activity.

In this lab, you will create a query that will show the Top 10 AWS Activity with their user information. You will also learn basic parsing with json and how to make a visual chart. The visual chart will become one of the panels in your starter Security Operations Center (SOC) dashboard with dashboard filtering. By the end of Lab 6 you will have created 5 panels for this SOC dashboard.


Lab Activity

  1. A CloudTrail log is a record in JSON format. The log contains information about requests for resources in your account, such as who made the request, the services used, the actions performed, and parameters for the action. First you search AWS CloudTrail logs to extract the user, event, IP addresses as metadata using a parse json, so you can use the extracted values later to monitor user activity. For time, use Last 24 Hours, then click Start or press Return or Enter. Notice we are renaming the fields using the as clause, such as userIdentity.userName as actor.

_sourceCategory=Labs/AWS/CloudTrail {{event_type}}
| json field=_raw "userIdentity.userName" as actor
| json field=_raw "eventType" as event_type
| json field=_raw "sourceIPAddress" as src_ip

  1. If you want you can use our UI to interactively parse the eventName in json. Position your cursor at the end of the last line in the query. In the field window below, you will highlight eventName, then right mouse click and select Parse selected key. It will automatically populate the parse as the last line in your query. Assign it to event_name metadata tag at the end of the json parse line, by typing as event_name. and then you can click Start to run the query.

| json field=_raw "eventName" as event_name

Screen Shot 2020-04-29 at 6.56.01 PM.png

Note: You can also take advantage of json logs structured format using json auto, which will extract all metadata tags.
 

  1. Congratulations! The first 5 lines of the query are complete, and you have learned how easy it is to parse using JSON. Now you can add the additional lines to complete the query. First create a space for the code to be inserted. Click at the end of the bottom line of code. Then do a soft return by simultaneously holding down shift + enter or shift + return on your keyboard.


Screen Shot 2020-08-12 at 2.19.11 PM.png
 

  1. Since we want to filter by event_type in our panel,  so we can pivot this panel, and eventually the dashboard, enter the following code and let's create it interactively using the UI.

    | where event_type matches event_type
     
  2. Highlight event_type and on the bottom right of the query builder click Create a parameter.

    Screen Shot 2020-10-20 at 7.32.12 PM.png
     
  3. A pop up window will appear called Manage Parameter Settings. At Parameter Name enter event_type and click Save.

    Screen Shot 2020-10-20 at 7.34.53 PM.png
     
  4. Now you will see a parameter window appear to the right of the query builder with event_type. Also you will see curly brackets around the highlighted event_type in the where statement. You have created a parameter for event_type which allows you to enter different values for this metadata key event_type to get the results you desire.

    Screen Shot 2020-10-20 at 7.35.16 PM.png
     
  5. Now you will create a second filter directly by coding. Using a where statement we can create the parameter actor. The curly brackets {{ }} define variables that create a parameter, just like when we created a event_type. On the second line, use count to aggregate the results by actor and event_type using count. The third line returns the top 10 actors. The last line is the transpose operator used for formatting the actor to appear on the rows and the event_type to appear in the column. 

| where actor matches {{actor}}
| count by actor,event_type,event_name
| top 10 actor by event_type,_count
| transpose row actor column event_type

  1. In the Parameters window, you will need to place a wildcard, using an asterisk *, for both the event_type and the actor. The wildcard will act as a holding spot for all values. With defining these two parameters, we are building the filtering capability for eventually being able to filter on metadata in the dashboard view.

    Screen Shot 2020-08-12 at 1.30.46 PM.png
    Note: The query will not run unless you have a value entered for each parameter.
     

     

  2. Go ahead and run the entire query for the Last 24 hours.  The results will appear in the Aggregate tab. 

_sourceCategory=Labs/AWS/CloudTrail
| json field=_raw "userIdentity.userName" as actor
| json field=_raw "eventType" as event_type
| json field=_raw "sourceIPAddress" as src_ip
| json field=_raw "eventName" as event_name
| where event_type matches {{event_type}}
| where actor matches {{actor}}

| count by actor,event_type,event_name
| top 10 actor by event_type,_count
| transpose row actor column event_type

Screen Shot 2020-06-10 at 2.51.39 PM.png
 

Note: Quite often we find our customers want to track by regions. Below is a similar query that provides results by awsRegion.

_sourceCategory=Labs/AWS/CloudTrail 
| json field=_raw "userIdentity.userName" as actor
| json field=_raw "eventType" as event_type
| json field=_raw "sourceIPAddress" as src_ip
| json field=_raw "eventName" as event_name
| json field=_raw "awsRegion" as region
| where event_type matches {{event_type}}
| where actor matches {{actor}}
| count by actor, region, event_type,event_name
| top 10 actor by region, _count
| transpose row actor column region

Creating a chart from the table

  1. You can chart your results by choosing any of the available charting options. Click bar graph iconScreen Shot 2020-04-08 at 3.55.44 PM.png below to obtain the bar graph. Screen Shot 2020-06-10 at 3.00.10 PM.png
  2. You may want to change the bar color to indicate a threshold count. To do this select the settings icon Screen Shot 2020-04-29 at 7.27.15 PM.png. First  select Change Properties and select the color red. Then click Save.

Screen Shot 2020-06-10 at 3.02.49 PM.png

  1. The bars will appear red as we want this to appear as potential exceptions.Screen Shot 2020-07-28 at 3.31.31 PM.png

Customizing the parameter with a lookup table

  1. At the top of your query, notice you have a Parameter window with two variables event_type and actor. Both were created in the query using the double curly brackets on the first and sixth lines for {{event_type}} and {{actor}}. Parameters are very useful for security investigation. They allow you to filter for investigation in your dashboard. To manage the parameter event_type click the details icon.Screen Shot 2020-07-28 at 4.14.50 PM.png
  2. Then select Manage Parameter Settings.

Screen Shot 2020-07-28 at 3.55.51 PM.png

  1. The popup window below will appear. Customize the Manage Parameter Settings window as below. Using the dropdown in the Data Type field, select Keyword. Expand into the Set Values for Parameter. Using the dropdown in the Select a format, select Lookup. In the Lookup file *, enter /shared/event_types. This file was already created in our training environment for your use. We will show you how you can create a file like this in a future lab. Using the dropdown in the Select a Field for Values, select event_type. Then click Save.

Screen Shot 2020-07-28 at 4.14.28 PM.png

 

  1. For your changes to take effect, run the query, click Start. Your chart will also update to reflect the lookup options both in the display and the legend.

    Screen Shot 2020-08-12 at 3.14.11 PM.png
     

Create a panel in your SOC Dashboard

  1. Now that you have successfully created a lookup parameter, go and look at it in a dashboard, so you can observe how filtering helps. In the chart, next to the top right settings icon, click Add to Dashboard.

Screen Shot 2020-07-28 at 3.27.33 PM.png

  1. A popup called Add Panel to Dashboard will appear. For the Panel Title type Top 10 User Activity. For the Dashboard name, type SOC_<your initials and 3 random numbers>. An example would be SOC_jas232. Using your SOC_<your initials and 3 random numbers>, click SOC_aaa### and then click Add

Screen Shot 2020-06-10 at 3.07.31 PM.png

  1. This will save the new dashboard to your personal folder Screen Shot 2020-04-29 at 7.53.05 PM.png and also open the dashboard. 

Screen Shot 2020-06-10 at 3.07.48 PM.png

  1. In the top right of the panel, notice the filter is enabled with the blue color indication. You can click on the filter and enter various values for the event_type and actor to filter the panel.
  2. Since we added a lookup to the event_type metadata tag, once you delete the asterisk * the options from the lookup will be offered for you to select. 
  3. You can compare back to actor metadata tag by deleting the asterisk * and here it expects you know what to type in.
  4. Once we add more panels, you will be able to filter all panels by using the dashboard filter at the top left of the dashboard. This gives you the option to filter at either the panel or dashboard level.

    Screen Shot 2020-08-12 at 3.25.19 PM.png
     

 

Quiz (True or False?)

  1. json field=_raw "sourceIPAddress" as src_ip  Extracts sourceIPAddress and all it's subsequent values as appearing in the incoming log messages.

  2. Double curly brackets enclosing a variable label are used to create parameters. 
  3. I used the settings icon in the Message tab to change the color of the bar graph.

 

Summary

Congratulations! You’ve completed the following tasks:

  1. Created a query to monitor user activity.
  2. Created a bar chart.
  3. Created both a lookup and string parameter.
  4. Created a panel in your SOC Dashboard.
  5. Created a filter for a dashboard panel.