Skip to main content
Sumo Logic

Collect logs for the Slack App

This pages shows you how to configure log collection for the Slack App, as well as providing a sample log message and query example.

This page explains how to collect logs from Slack and ingest them into Sumo Logic for use with the Slack App predefined dashboards and searches.

Log types

Slack logs are in JSON format. The Slack App utilizes the following log types.

The availability of all types of logs is determined by the slack plans

Log Type

Free plan

Standard plan

 Plus plan

Enterprise plan

User logs

Public Channel logs

Public Message logs

Access logs

 

Audit logs

     

Collection overview

Sumo Logic enables you to collect logs from Slack via the Slack API. You can then configure various log types to collect. The logs are then forwarded to a Sumo Logic HTTP Source. By default the collection starts from the current date and time, but this can be configurable. Please see the Advanced Configuration section for more details. Configuring log collection for the Slack App includes the following tasks:

  • Step 1: Create a Slack API token for log collection
  • Step 2: Add a Hosted Collector and HTTP Source
  • Step 3: Configure collection for Slack

Step 1: Create a Slack API token for log collection

This section demonstrates how to generate a Slack API token for all types of Slack plans, and is organized based on the type of Slack plan and log type. Identify your Slack plan and generate the Slack API token, as described in the following steps.

Users, channels, and access logs

You must have admin privileges to perform this task. The token generated in the following steps can be used by all Slack plans to collect the mentioned log types.

To generate a Slack API token for users, channels and access logs, do the following:

  1. Go to Apps page.
  2. Click Create New App.

Slack_Create_New_App_dialog.png

  1.  Enter the App Name and select the Development Slack Workspace for which you need to generate a token and collect logs.

Slack_App_Name_dialog.png

  1. Click Create App.
  2. In the Basic Information section for the app created above, click Permissions.

Slack_Basic_Information_dialog.png

  1. In the Scopes section, add the following permissions to collect logs, and then click Save. Logs will be collected based on these permissions:

Permission

Log Collected

Slack Plan

admin

Access Logs

All

channels:history

Public Message Logs

All

channels:read

Public Channel Logs

All

Users:read

users:read.email

Users Logs

All

team:read

Team name in all logs.

All

Slack_Scopes_dialog.png

  1. Go to Install App and click Install App to Workspace.

Slack_Install_App_dialog.png

The app prompts you for permission to install based on your selected permission.

  1. Click Allow to install the app to workspace.

Slack_Confirm_Permission.png

  1. Copy the generated token. You will need to use this token when configuring the Slack collector.

Slack_App_Token_dialog.png

  1. Verify that the generated token is valid with the following commands. If the token is valid, the output will have "ok":true in the response. Replace the <API_TOKEN> variable with the generated token you copied in the previous step.
curl -X GET -H "Content-Type: application/json" https://slack.com/api/team.info?token=<API_TOKEN>&pretty=1
curl -X GET -H "Content-Type: application/json" https://slack.com/api/users.list?token=<API_TOKEN>&limit=5&pretty=1
curl -X GET -H "Content-Type: application/json" https://slack.com/api/channels.list?token=<API_TOKEN>&limit=2&pretty=

Audit logs

This generated token can only be used by the Enterprise Slack plan to collect audit logs.

To generate a Slack API token for audit logs, do the following:

  1. For the Sumo Slack app you created in Users, channels and access logs, Go To OAuth and Permission.
  2. Go to Redirect URLs and add a Redirect URL as http://localhost, then click Save URLs.

Slack_OAuth&Permissions_dialog.png

  1. Go To Manage Distribution > Share Your App with Other Workspaces
  2. Open the ​Remove Hard Coded Information ​section on the same page and check the I’ve reviewed and removed any hard-coded information ​checkbox.

Slack_Remove-Hardcoded-Info.png

  1. Click the Activate Public Distribution.
  2. Copy the Sharable URL and append auditlogs:read at the end. Such as in the following example:
https://slack.com/oauth/authorize?client_id=12345686.853580033397&scope=admin,channels:history,channels:read,team:read,users:read,users:read.email,auditlogs:read

Slack_Manage-Distribution-dialog.png

  1. Open a new tab in your browser, paste the modified URL and press Enter.
  2. Select the drop-down menu in the upper right corner and choose the correct organization.

Slack_Select-Org-menu.png

  1. Click Allow.
  2. Ignore the error message and copy the Code in the URL field, as shown in the following example.

Slack_Copy-URL.png

  1. Get the client ID and client secret from the Basic information of your Slack app. Replace the <CODE>, <CLIENT_ID> and <CLIENT_SECRET> variables in the following URL.
https://slack.com/api/oauth.access?code=<CODE>&client_id=<CLIENT_ID>&client_secret=<CLIENT_SECRET>
  1. Open a new browser tab and paste the URL from the previous step into the URL field, then press Enter.
  2. From the response, copy the token value from the field access_token.
{
  "ok": true,
  "access_token": "xoxp-1236544616-Example-Access-Token5bf71298dad60d941f2a44b371",
  "scope": "admin,identify,channels:history,groups:history,im:history,channels:read,team:read,users:read,users:read.email,auditlogs:read",
  "user_id": "WA7PQK3U5",
  "team_id": "EFSFVS",
  "enterprise_id": "EASFEF",
  "team_name": "Test Slack App"
}
  1. Verify that the generated token is valid with the following commands. If the token is valid, the output will have "ok":true in the response. Replace the <API_TOKEN> variable with the generated token you copied in the previous step.
curl -X GET -H "Content-Type: application/json" https://slack.com/api/team.info?token=<ACCESS_TOKEN>&pretty=1
curl -X GET -H "Content-Type: application/json" -H "Authorization: Bearer <ACCESS_TOKEN>" https://api.slack.com/audit/v1/logs?limit=5&pretty=1

Step 2: Add a Hosted Collector and HTTP Source

This section demonstrates how to add a hosted Sumo Logic collector and HTTP Logs source, to collect logs for Slack.

Identify an existing Sumo Logic Hosted Collector you want to use, or create a new Hosted Collector as described in the following task

To add a hosted collector and HTTP source, do the following:

  1. Create a new Sumo Logic Hosted Collector by performing the steps in Configure a Hosted Collector.

  2. Create a new HTTP Log Source in the hosted collector created above by following these instructions.

Step 3: Configure collection for Slack

This section covers the various ways in which to collect logs from Slack and send them to Sumo Logic. The logs are then shown in dashboards as part of the Slack App. You can configure a Sumo Logic collector for Slack in Amazon Web Services (AWS) using AWS Lambda service, or use a script on a Linux machine with a cron job. Choose the method that is best suited for your environment:

  • AWS Lambda based collection via a via a Serverless Application Model (SAM) application
  • Script based collection

Sumo Logic Slack SAM application

In this collection method, you deploy the SAM application, which creates the necessary  resources in your AWS account.

To deploy the Sumo Logic Slack SAM application, do the following:

  1. Go to https://serverlessrepo.aws.amazon.com/applications.

  2. Search for sumologic-slack and make sure  the checkbox next to the text Show apps that create custom IAM roles or resource policies is selected, then click the app link when it appears.

Slack_Serverless_App_Repo_dialog.png

  1. When the page for the Sumo app appears, click Deploy.

Slack_Deploy_SAM_app.png

  1. In the AWS Lambda > Functions > Application Settings panel, enter the following parameters in the corresponding text fields:

  • HTTPLogsEndpoint: Copy and paste the URL for the HTTP log source from Step 2.
  • Token: Copy and paste the Authorization token from Step 1.
  • BackfillDays: Number of days before the event collection will start. If the value is 1, then events are fetched from yesterday to today.

Slack_Application_Settings_dialog.png

  1. Click Deploy.

Configuring collection for multiple Slack Workspaces

This section shows you how to configure collection for multiple projects assuming you are already collecting Slack data for one project.

To configure collection for multiple projects, do the following: 

  1. Deploy the SAM application with configuration for new project.

  2. After the deployment is complete, change the database name by adding environment variable (DBNAME) in AWS Lambda.

Slack_DBNAME_dialog.png

Sumo Logic Slack Script based collection

This section provides instructions for deploying script based collection for the Sumo Logic Slack App.

Prerequisites

  • You must have successfully added a Hosted Collector and HTTP source and copied configuration parameter (token) from Slack, as described in Step 1 and Step 2.
  • You must be logged in to the user account with which you will install the collector. If you are not, use this command to switch to that account: 
    sudo su <user_name>

Configure the script on a Linux machine

This task shows you how to install the script on a Linux machine.

To deploy the script, do the following:

  1. If pip is not already installed, follow the instructions in the pip documentation to download and install pip

  2. Log in to a Linux machine (compatible with either Python 3.7 or Python 2.7. 

  3. Do one of the following:

For Python 2, run the following command: pip install sumologic-slack

For Python 3, run the following command: pip3 install sumologic-slack 

  1. Create a configuration file slackcollector.yaml in the home directory as shown in the following example and specify the parameters where indicated.

Slack:
 TOKEN: <Paste the Slack authorization token>
 
Collection:
 BACKFILL_DAYS: <Enter the number of days before the event collection will start>

SumoLogic:
 HTTP_LOGS_ENDPOINT: "<Paste the URL for HTTP log sourc from Sumo Logic collector>"
  1. Create a cron job  to run the collector every 5 minutes, (use the crontab -e option). Do one of the following:

For Python 2, add the following line to your crontab: 
*/5 * * * *  /usr/bin/python -m sumoslack.main > /dev/null 2>&1

For Python 3, add the following line to your crontab: 
*/5 * * * *  /usr/bin/python3 -m sumoslack.main > /dev/null 2>&1

Configuring collection for multiple projects 

This section shows you how to configure collection for multiple projects assuming you are already collecting Slack data for one project.

To configure collection for multiple projects, do the following:

  1. After configuring the script on a Linux machine, go to your configuration file.
  2. Change the DB_NAME in the slackcollector.yaml file, as indicated in the following example.
Slack:
 TOKEN: <Paste the Slack authorization token>

Collection:
 BACKFILL_DAYS: <Enter the number of days before the event collection will start>
 DBNAME: "<Database Name>"

SumoLogic:
 HTTP_LOGS_ENDPOINT: "<Paste the URL for HTTP log sourc from Sumo Logic collector>"

Advanced Configuration

This section is common for both AWS Lambda based collection and script based collection

The following table provides a list of variables for Slack that you can optionally define in the configuration file.

Variable Usage

LOG_TYPES in Slack Section

Remove logs based on the type of token used.

LOG_TYPES:

 - USER_LOGS

 - ACCESS_LOGS

 - CHANNELS_LOGS

 - CHANNELS_MESSAGES_LOGS

 - AUDIT_LOGS

# The following audit logs that can be excluded. Use the exact action name from Slack.

ExcludeAuditLogs:

- Exclude_action_name1

- Exclude_action_name2

BACKFILL_DAYS in Collection Section Number of days before the event collection will start. If the value is 1, then events are fetched from yesterday to today.
LOG_FORMAT in Logging Section Log format used by the python logging module to write logs in a file.
ENABLE_LOGFILE in Logging Section Set to TRUE to write all logs and errors to a log file.
ENABLE_CONSOLE_LOG in Logging Section Enables printing logs in a console.
LOG_FILEPATH in Logging Section Path of the log file used when ENABLE_LOGFILE is set to TRUE.
NUM_WORKERS in Collection Section Number of threads to spawn for API calls.
MAX_RETRY in Collection Section Number of retries to attempt in case of request failure.
BACKOFF_FACTOR in Collection Section A backoff factor to apply between attempts after the second try. If the backoff_factor is 0.1, then sleep() will sleep for [0.0s, 0.2s, 0.4s, ...] between retries.
TIMEOUT in Collection Section Request time out used by the requests library.
HTTP_LOGS_ENDPOINT in SumoLogic section HTTP source endpoint url created in Sumo Logic for ingesting Logs.

Troubleshooting

This section shows you how to run the function manually and then verify that log messages are being sent from Slack.

To run the function manually, do the following:

  1. Enter  one of the following commands:
  • For Python 2, use this command: python -m sumoslack.main
  • For Python 3, use this command: python3 -m sumoslack.main
  1. Check the automatically generated logs in  /tmp/sumoapiclient.log to verify whether the function is getting triggered or not.
  2. If you get an OAuth Error: team_not_authorized error when you try to add scopes to your slack app, remove auditlogs:read scope from the app.

Sample log messages

The following table provides sample log messages for the different log types.

Log Type Sample log message
User logs

{

  "id": "UM27LNGHK",

  "name": "test",

  "deleted": false,

  "real_name": "test",

  "tz": "Asia/Kolkata",

  "tz_label": "India Standard Time",

  "is_admin": false,

  "is_owner": false,

  "is_primary_owner": false,

  "is_restricted": false,

  "is_ultra_restricted": false,

  "is_bot": false,

  "is_app_user": false,

  "updated": 1565005724,

  "has_2fa": false,

  "teamName": "TestSlack",

  "email": "test@test.com",

  "billable": true,

  "logType": "UserLog"

}

Public Channel logs

{

  "channel_id": "CKN1D8010",

  "channel_name": "testchannel",

  "members": 2,

  "logType": "ChannelDetail",

  "teamName": "TestSlack"

}

Public Message logs

{

  "type": "message",

  "text": "Test",

  "files": [

    {

      "name": "Test",

      "fileType": "epub",

      "fileSize": 1258,

      "urlPrivate": "https://files.slack.com/files-pri/TJ...htyhomsdconmps",

      "urlPrivateDownload": "https://files.slack.com/files-pri/TJ...htyhomsdconmps",

      "permalink": "https://testslack-xj11408.slack.com/...htyhomsdconmps"

    }

  ],

  "attachments": [

    {

      "id": 16,

      "text": "Test",

      "author_name": "",

      "author_link": "",

      "pretext": "",

      "fallback": "Messages Sent"

    }

  ],

  "upload": true,

  "user": "e65b0bd8",

  "display_as_bot": false,

  "ts": "1566215592",

  "client_msg_id": "23849274-580c-4644-9478-8328e5716b89",

  "userName": "roy",

  "channelId": "e65b0d0e",

  "channelName": "app-for-slack",

  "teamName": "TestSlack",

  "logType": "ConversationLog"

}

Access logs

{

  "user_id": "e65b0476",

  "username": "dave",

  "date_first": 1566215532,

  "date_last": 1566215532,

  "count": 2,

  "ip": "213.14.129.105",

  "user_agent": "Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1467.0 Safari/537.36",

  "isp": "Inetbroadband",

  "country": "PA",

  "region": "EU",

  "teamName": "TestSlack",

  "logType": "AccessLog"

}

Audit logs

{

  "logType": "UserAuditLog",

  "id": "bdcb13e3-28a3-41f0-9ace-a20952def3a0",

  "date_create": 1566215192,

  "action": "user_created",

  "actor": {

    "type": "user",

    "user": {

      "id": "e65b0f5c",

      "name": "roy",

      "email": "aaron@demo.com"

    }

  },

  "entity": {

    "id": "e65b107e",

    "privacy": "public",

    "name": "BigCo ISP",

    "is_shared": false,

    "is_org_shared": false,

    "filetype": "text/csv",

    "title": "john",

    "is_distributed": false,

    "is_directory_approved": false,

    "scopes": [

      "identify",

      "bot",

      "incoming-webhook",

      "channels:read",

      "groups:read",

      "im:read",

      "users:read",

      "chat:write:bot",

      "users:read.email",

      "groups:write",

      "channels:write",

      "team:read",

      "chat:write:user"

    ]

  },

  "context": {

    "location": {

      "type": "workspace",

      "id": "e65b11aa",

      "name": "Docker",

      "domain": "Docker"

    },

    "ua": "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:23.0) Gecko/20131011 Firefox/23.0",

    "ip_address": "120.188.0.246"

  },

  "details": {

    "id": "USLACKUSER",

    "name": "himanshu",

    "email": "kumar@demo.com"

  }

}

Query example

The sample query is from Channel Summary panel of Slack - Public Channels dashboard.

_sourceCategory=Labs/slack
| join ("logType":"channelDetail"
| json "channel_name", "channel_id", "teamName", "members" as Channel, ChannelId, Workspace, Members
| withtime Members
| most_recent(Members_withtime) as Members by Channel, ChannelId, Workspace) as T1,("logType":"ConversationLog" | json "user", "userName", "type", "subtype", "ts", "text", "channelId", "channelName", "teamName" as ID, User, Type, SubType, Time, Text, ChannelId, Channel, Workspace nodrop
| count_distinct(Time) as Messages by ID, ChannelId, Workspace) as T2 on T1.ChannelId = T2.ChannelId and T1.Workspace=T2.Workspace
| T2_Workspace as Workspace | T2_ID as User| T1_Channel as Channel
| where Workspace matches {{Workspace}} and Channel matches {{Channel}}
| T1_Members as %"Team Members"
| fields Workspace, Channel, User, %"Team Members" ,T2_Messages
| where [subquery:"logType":"UserLog"
| json "id", "name", "deleted", "is_bot", "teamName" as User, Name, Deleted, Bot, Workspace nodrop
| where Bot matches "false" and !(Name matches "slackbot") and Deleted matches "false"
| withtime Name
| most_recent(Name_withtime) as Name by User, Workspace
| compose User, Workspace]
| sum(T2_Messages) as %"Total Messages", count_distinct(User) as %"Members Posted Messages" by Workspace, Channel, %"Team Members"
| fields Workspace, Channel, %"Team Members", %"Total Messages", %"Members Posted Messages"
| sort by %"Total Messages"
| limit 20