Skip to main content
Sumo Logic

Set up the Software Development Optimization Solution

This page provides instruction for setting up the Software Development Optimization Solution using the Terraform script and manual configuration.

Prerequisites

Before setting up the solution, you need to perform the following tasks:

  • API Keys/Creds for Jira, Github, Jenkins, Bitbucket, Opsgenie with Webhook creation capabilities. 
  • Access to Jenkins (Manage Plugins permissions)
  • Make sure you have access to the Sumo logic console and as a user that is associated with a Sumo Logic role that has the following permissions:
    • Manage field extraction rules
    • View Fields
    • View field extraction rules
    • Manage Collectors
    • View Collectors
    • Manage Fields
    • Manage connections
    • Manage Content
  • Create a Sumo Logic API key and ID  for the above user.

The Software Development Optimization solution can be set up either by:

  1. A Terraform script OR 
  2. By manually installing the apps, collection and FERs for each supported tool
  3. Once the SDO solution has been setup, you can perform the post-setup configuration

Manual Configuration

To set up the solution manually: 

  1. Manually configure collection and install apps for each tool that you wish to integrate with this solution by following the instructions in the app pages for:

  1. You will need to manually create new field extraction rules that map events from tools of your choice to the SDO event schema by using the scope and parse expression from this JSON file of field extraction rules for each tool. Please note that while copying the scope, replace the “$$<toolname>” variable with the source category you configured for the tool you configured in Step 1. For example for Bitbucket, you will replace $$bitbucket with _sourceCategory=bitbucket. 

  2. Install the Software Development Optimization app

Set up using the Terraform Script

Terraform Script Setup Considerations

The Terraform Script automates the setup of all the necessary resources for collecting Atlassian,Github and Pagerduty product logs and displaying the data in Sumo Logic pre-configured intuitive-analytic dashboards. 

Review the setup consideration before proceeding with the Terraform template:

  • With the Terraform script you can easily manage the integration of Sumo Logic with Atlassian products (Bitbucket, Jira, Jira Service Desk, Opsgenie), Github, Pagerduty. If you would like to bring your own toolset, follow this page to integrate your tool with the SDO solution.
  • The Terraform script allows you to quickly get started by installing a copy of the configured applications. After the initial setup, if you need additional copies of the Sumo Logic applications, you can install them from the Sumo Logic App catalog.
  • If you plan to integrate Jenkins with this solution, the Jenkins plugin is not configured by the Terraform script. To configure Jenkins collection, please follow the steps in the Jenkins documentation -  Install the Jenkins Plugin, Configure Jenkins Plugin, and Optional - Advanced Configuration to install and configure the Jenkins Sumo Logic plugin. In Configure Jenkins Plugin, a source category is configured which is utilized by the plugin.  Use this source category in the file sumologic.auto.tfvars file. The Jenkins source, app and FERs are installed via Terraform.
  • This script configures Jira Server WebHooks and creates resources in Sumo Logic. Jira Server Logs collection needs to be configured as explained in Step 1 here. Configure the log collection and update the variable jira_server_access_logs_sourcecategory in atlassian.auto.tfvars with the selected source category.
  • Sumo Logic to Jira and Sumo Logic to Opsgenie Webhooks are in Beta. To participate, contact your Sumo account executive.

The Terraform script performs the following actions:

  1. Creation of Sumo Logic collection resources. (Collector, Sources, Folder)
  2. Configures collection mechanisms in the supported tools. For example, Webhooks in Pagerduty and Github
  3. Sets up Sumo Logic Field Extraction Rules to enrich the data.
  4. Configures connections in Sumo Logic to send alerts from Sumo Logic to issue management and incident management platforms like Jira, PagerDuty and Opsgenie. 
  5. Installs Sumo Logic Apps in the configured folder.

The Terraform script requires a set of parameters to be configured as defined here.

Follow the below instructions to set up the SDO Solution using the Terraform script.

Step 1: Setup the Terraform environment

Before you run the Terraform script, please perform the following actions on a server machine of your choice:

  • Install Terraform version equal or greater than 0.12.20 and lower than 0.13.0. 
  • Install the latest version of curl
  • Install Python version 2.7 or later.
  • Install the following third Party Terraform providers
    • Rest API Terraform Provider version 1.12 and above
      • Download terraform-provider-restapi binary from Github
      • Unzip the zip file
      • Then move terraform-provider-jira binary to $HOME/.terraform.d/plugins directory
        mkdir -p $HOME/.terraform.d/plugins
        mv terraform-provider-restapi   
        $HOME/.terraform.d/plugins/terraform-provider-jira
    • If you plan to use Jira, then install Jira Terraform Provider version  0.1.11 and above.
      • Download terraform-provider-jira binary from Github
      • Unzip the zip file
      • Then move terraform-provider-jira binary to $HOME/.terraform.d/plugins directory
        mkdir -p $HOME/.terraform.d/plugins
        mv terraform-provider-jira
        $HOME/.terraform.d/plugins/terraform-provider-jira
Step 2: Configure the Terraform script
  1. Clone the repository:
    $ git clone https://github.com/SumoLogic/sumolog...-templates.git

  2. The SDO solution script is organized into following groups of files:

    • Configuration Files. 

      • sumologic.auto.tfvars

      • sumologic_fer.auto.tfvars

      • sumologic_webhooks.auto.tfvars

      • pagerduty.auto.tfvars

      • github.auto.tfvars

      • atlassian.auto.tfvars

    • Sumo Logic Resource Creation Files. 

      • sumologic_collection_content.tf

      • sumologic_fer.tf

      • sumologic_webhooks.tf

    • Other Systems Resource Creation Files.

      • atlassian.tf

      • pagerduty.tf

      • github.tf

    • System Files:

      • outputs.tf

      • providers.tf

      • Variables.tf

    • Test Files:

      • integration_test.go

      • fileutil.go

      • unit_tests.sh

  3. Initialize the Terraform working directory by navigating to the directory sumologic-solution-templates/software-development-optimization-terraform. and running terraform init.
    This will install the required Terraform providers i.e. Sumo Logic Terraform Provider, Template, Null, BitBucket Terraform Provider, Github Terraform Provider and Pagerduty Terraform Provider.

  4. Choose which Sumo Logic Applications to configure by updating the values of the following variables in the sumologic.auto.tfvars file.

  • install_jira_cloud,
  • install_jira_server
  • install_bitbucket_cloud
  • install_opsgenie
  • install_github
  • install_pagerduty
  • Install_jenkins
  • install_sdo 
  1. You can choose which Webhooks to configure by updating the variables in sumologic.auto.tfvars.

  • install_sumo_to_opsgenie_webhook
  • install_sumo_to_jiraserver_webhook,  
  • install_sumo_to_jiraservicedesk_webhook, 
  • install_sumo_to_jiracloud_webhook and 
  • install_sumo_to_pagerduty_webhook 
  1. Update the following placeholder values in the  sumologic.auto.tfvars file: atlassian.auto.tfvars, pagerduty.auto.tfvars, github.auto.tfvars, sumologic_fer.auto.tfvars and sumologic_webhooks.auto.tfvars so they correspond with your Sumo Logic, Atlassian, Github and Pagerduty environments. See the list of input parameters in the Configurable Parameters section below.
Step 3: Install  the Software Development Optimization (SDO) Solution

There are three different ways in which you can install the Software Development Optimization solution by using the Terraform script:

  1. Install the complete Software Development Optimization solution via the Sumo Logic Terraform script. You will choose this method if you have not set up any collection of data or apps in Sumo Logic already.

  2. Install one or more parts of the solution

    • Install only the Sumo Logic Apps via the Terraform script.  Choose this method if you are already collecting data and have set up Webhooks. Note: This will not configure collectors, sources, FERs, and connections in Sumo Logic and will not configure webhooks in other systems. )

    • Install only the FERs via Terraform Script. Choose this method if you have already configured and set up the collection and have installed the relevant apps. Note: This will not configure collectors, sources, App installation, and connections in Sumo Logic and will not configure Webhooks in other systems.

    • Install connections via Terraform script. Choose this method if you want to only configure Sumo Logic connections. Note: This will not configure collectors, sources, App installation, and FERs in Sumo Logic and will not configure Webhooks in other systems.

1. Install the complete SDO solution via the Sumo Logic Terraform Script.

This procedure will configure collection in other systems (For Example, Jira Cloud) and it will create sources and FERs/Fields and connections in Sumo Logic along with App installation:

  • Set install_(app) variables as ‘all’ for the required applications in sumologic.auto.tfvars, as defined in this section.
  • Setting install_(app) this variable as ‘all’ will result in Sumo Logic apps being installed,  collectors being configured and field extraction rules and fields being created in Sumo Logic.
  • Set install_(app) variable as ‘none’ for the tools that are not applicable to your environment.
  • To install the outbound connections in Sumo Logic configure the install_sumo_to_(app)_webhook variables as ’true’ as defined in this section.
  • Configure required variables in sumologic.auto.tfvars, atlassian.auto.tfvars, github.auto.tfvars, pagerduty.auto.tfvars.
  • Navigate to the directory 
    sumologic-solution-templates/software-development-optimization-terraform and execute the commands below:
    $ terraform plan
    $ terraform apply
2. Install one or more parts of the solution

Install FERs via Terraform Script
If you would like to use existing sources and apps, or configure these manually, follow the process below :

  • Configure sources and source categories in Sumo Logic.
  • Configure collection in respective systems for example Jira Cloud.
  • Populate source categories in sumologic.auto.tfvars, as defined in this section.
  • Set install_(app) variable as ‘fer’ for the required applications in sumologic.auto.tfvars, as defined in this section.
  • Setting install_(app) variable as ‘fer’ will result in FER configuration in Sumo Logic, it will not configure collection in other systems (for example Jira Cloud) and it will not create sources and Fields in Sumo Logic and will not install Apps.
    • Navigate to the directory 
      sumologic-solution-templates/software-development-optimization-terraform and execute the commands below:
      • $ terraform plan
      • $ terraform apply

Install only the apps via Terraform

If you would like to use existing sources, FERs/Fields or configure these manually and install only the applications, please follow the steps below: 

  • Other than the SDO app, configure collection in Sumo Logic if not already done Based on the app documentation
  • Populate source categories that you set up during the collection phase in sumologic.auto.tfvars, as defined in this section.
  • Set install_(app) variables as ‘app’ for the required applications in sumologic.auto.tfvars, as defined in this section.
  • Setting this flag as ‘app’ will result in app installation in Sumo Logic, it will not configure collection in other systems (for example Jira Cloud) and it will not create sources and FERs/Fields in Sumo Logic.
  • Set install_(app) variables as ‘none’ for the applications which should not be configured.
  • Set install_sumo_to_(app)_webhook variables as ‘false’ for the Sumo Logic connections if you do not wish to configure the outgoing connections in Sumo Logic.
  • Navigate to the directory sumologic-solution-templates/software-development-optimization-terraform and execute the commands:
    $ terraform plan
    $ terraform apply

Install the Sumo Logic outgoing connections via Terraform Script.

If you would like to configure the Sumo Logic connections only, please follow the steps below:

  • Set install_(app)_webhook variables as ‘none’ for all the applications except Opsgenie. Opsgenie connection cannot be installed without configuring Opsgenie collection as there is a cyclic dependency.
  • Set install_sumo_to_(app)_webhook variables as ‘true’ for the Sumo Logic connections which should be configured in Sumo Logic.
  • Configure required variables in sumologic_webhooks.auto.tfvars.
  • Navigate to the directory sumologic-solution-templates/software-development-optimization-terraform and execute the commands:
    $ terraform plan
    $ terraform apply

Post set-up Configuration

Post-set-up configuration involves instrumenting your DevOps pipeline to specially identify and send build and deploy events in your pipeline to Sumo Logic. These events are correlated with other events in the DevOps lifecycle.

  • If you are using Bitbucket for Build and Deploy
    • Go to Sumo Logic Platform, and navigate to Manage Data -> Collection page. Search for Software Development Optimization Collector. Under this Collector, click on Show URL for the source Bitbucket Cloud. Make a note of this URL and use this URL to  configure the Bitbucket CI/CD Pipeline to collect deploy events :
      Deploy. Follow the steps outlined in this document to configure the Bitbucket CI/CD Pipeline to collect deploy events.
  • If you are using Jenkins for Build and Deploy
    Install the latest Jenkins Plugin as described here
    • Go to Sumo Logic Platform, and navigate to Manage Data -> Collection page. Search for Software Development Optimization Collector. Under this Collector, click on Show URL for the source Jenkins. Make a note of this URL and Source Category, you will use these to configure the Jenkins Plugin :
      Build Pipeline Stages. Follow Configure Jenkins Plugin, and optionally Optional - Advance Configuration to configure the Jenkins Sumo Logic plugin.
    • Build. Follow this doc to modify your Jenkins plugin to explicitly identify, enrich, and send Build Events to Sumo Logic.
    • Deploy. Follow this doc to modify your Jenkins plugin to explicitly identify, enrich, and send Deploy Events to Sumo Logic.
  • If you are using tools other than Jenkins and Bitbucket pipelines for Build and Deploy phases:
    •  Build. Configure your tool to construct and send events using the build event schema in the section Build Event. 

    • Deploy. Configure your tool to construct events using the deploy event schema in the section Deploy Events.

Uninstalling the Solution

To uninstall the solution, navigate to the directory sumologic-solution-templates/software-development-optimization-terraform and execute the command:

$ terraform destroy

Configurable Parameters

Configure the following parameters in sumologic.auto.tfvars, atlassian.auto.tfvars, github.auto.tfvars, pagerduty.auto.tfvars, sumologic_fer.auto.tfvars and webhooks.auto.tfvars.

Sumo Logic

Configure these parameters in sumologic.auto.tfvars.

Parameter Description Default
sumo_access_id Sumo Logic Access ID  
sumo_access_key Sumo Logic Access Key  
deployment Sumo Logic Deployment us1
sumo_api_endpoint Sumo Logic API Endpoint. Make sure the trailing "/" is present. https://api.sumologic.com/api/v1/
app_installation_folder The Sumo Logic apps will be installed in a folder under your personal folder in Sumo Logic. Software Development Optimization
install_jira_cloud Install Sumo Logic Application and WebHooks for Jira Cloud. Options: app, collection, fer, all, and none. all
install_jira_server Install Sumo Logic Application and WebHooks for Jira Server. Options: app, collection, fer, all, and none. all
install_bitbucket_cloud Install Sumo Logic Application and WebHooks for BitBucket Cloud. Options: app, collection, fer, all, and none. all
install_opsgenie Install Sumo Logic Application and WebHooks for Opsgenie. Options: app, collection, fer, all, and none. all
install_github Install Sumo Logic Application and WebHooks for Github. Options: app, collection, fer, all, and none. all
install_pagerduty Install Sumo Logic Application and WebHooks for Pagerduty. Options: app, collection, fer, all, and none. all
install_jenkins Install Sumo Logic Application for Jenkins. Options: app, collection, fer, all, and none.
The Terraform script does not configure the Jenkins Sumo Logic plugin, choosing ‘collection’ will create http source in Sumo Logic for Jenkins and will configure the Jenkins FERs.
 
all
install_sdo Install Sumo Logic Application for Software Development Optimization. Options: app or none. app
install_sumo_to_opsgenie_webhook Install Sumo Logic to Opsgenie WebHook. install_opsgenie should be true for this option to be true. true
install_sumo_to_jiracloud_webhook Install Sumo Logic to Jira Cloud WebHook. true
install_sumo_to_jiraserver_webhook Install Sumo Logic to Jira Server WebHook true
install_sumo_to_jiraservicedesk_webhook Install Sumo Logic to Jira Service Desk WebHook true
install_sumo_to_pagerduty_webhook Install Sumo Logic to Pagerduty WebHook true
jira_cloud_sc Source Category for Jira Cloud SDO/Jira/Cloud
jira_server_sc Source Category for Jira Server SDO/Jira/Server/Events
bitbucket_sc Source Category for BitBucket Cloud SDO/Bitbucket
opsgenie_sc Source Category for Opsgenie SDO/Opsgenie
pagerduty_sc Source Category for Pagerduty SDO/Pagerduty
github_sc Source Category for Github SDO/Github
jenkins_sc Source Category for Jenkins SDO/Jenkins
Sumo Logic Field Extraction Rules

Configure these parameters in sumologic_fer.auto.tfvars. There is a set of FER's for the SDO Apps. Each FER needs scope and a parse expression. In most cases default values will suffice, if required you can edit the scope and parse expression as per your requirements.

Parameter Description
(app)_pull_request_fer_scope A keyword search expression that points to the subset of logs you'd like to parse. For more details see FER.
(app)_pull_request_fer_parse A valid parse expression with supported parse and search operators. For more details see FER.
Jira Cloud

Configure these parameters in atlassian.auto.tfvars.

Parameter Description
jira_cloud_url Jira Cloud URL
jira_cloud_user Jira Cloud Username
jira_cloud_password Jira Cloud Password or API Key
jira_cloud_jql Jira Cloud Query Language Example: "project = Sumo"
jira_cloud_events Jira Cloud Events
Sumo Logic to Jira Cloud Webhook

Configure these parameters in webhooks.auto.tfvars.

Parameter Description
jira_cloud_auth Jira Cloud URL
jira_cloud_user Jira Cloud Username
jira_cloud_password Jira Cloud Password or API Key
jira_cloud_jql Jira Cloud Query Language Example: "project = Sumo"
jira_cloud_events Jira Cloud Events
Sumo Logic to Jira Service Desk Webhook

Configure these parameters in webhooks.auto.tfvars.

Parameter Description
jira_servicedesk_url Jira Service Desk URL, can be same as Jira Cloud URL
jira_server_auth Basic Authorization Header
jira_server_projectkey Jira Server Project Key
jira_server_issuetype Jira Server Issue Type, for example 'Bug'
jira_server_priority Issue Priority. For Example, 3
Jira Server

Configure these parameters in atlassian.auto.tfvars.

Parameter Description
jira_server_access_logs_sourcecategory Jira Server Access Logs Source Category, default "SDO/Jira/Server*", refer to this link.
jira_server_url Jira Server URL
jira_server_user Jira Server Username
jira_server_password Needs to be the password. API Key is not supported on Jira Server yet.
jira_server_jql Jira Server Query Language Example: "project = Sumo"
jira_servicedesk_priority Jira Server Events
Sumo Logic to Jira Server Webhook

Configure these parameters in webhooks.auto.tfvars.

Parameter Description
jira_server_auth Basic Authorization Header
jira_server_projectkey Jira Server Project Key
jira_server_issuetype Jira Server Issue Type, for example 'Bug'
jira_server_priority Issue Priority, for example 3
Bitbucket

Configure these parameters in atlassian.auto.tfvars.

Parameter Description
bitbucket_cloud_user Bitbucket Username
bitbucket_cloud_password Bitbucket password or App Password
bitbucket_cloud_owner The owner of the repositories. Can be you or any team having write access.
bitbucket_cloud_repos Specify the repositories for which WebHooks should be created. Format: ["repo1","repo2"]
bitbucket_cloud_desc The name / description to show in the UI
bitbucket_cloud_events Bitbucket Events to track
Opsgenie

Configure these parameters in atlassian.auto.tfvars.

Parameter Description
opsgenie_api_url Opsgenie API URL. Do not add the trailing "/". If using the EU instance of Opsgenie, the URL needs to be https://api.eu.opsgenie.com for requests to be executed.
opsgenie_key Opsgenie API Key
Sumo Logic to Opsgenie Webhook

Configure these parameters in webhooks.auto.tfvars.

Parameter Description
opsgenie_priority Opsgenie Alert Priority
PagerDuty

Configure these parameters in pagerduty.auto.tfvars.

Parameter Description
pagerduty_api_key Pagerduty API Key.
pagerduty_services_pagerduty_webhooks List of Pagerduty Service IDs. Example, ["P1QWK8J","PK9FKW3"]. You can get these from the URL after opening a specific service in Pagerduty. These are used for Pagerduty to Sumo Logic webhooks.
Sumo Logic to PagerDuty Webhook

Configure these parameters in webhooks.auto.tfvars.

Parameter Description
pagerduty_services_sumo_webhooks Sumo Logic to Pagerduty Webhook. List of Pagerduty Service IDs. Example, ["P1QWK8J","PK9FKW3"]. You can get these from the URL after opening a specific service in Pagerduty. These are used for Sumo Logic to Pagerduty Webhooks.
GitHub

Configure these parameters in github.auto.tfvars.

Parameter Description
github_token Github Token
github_organization Organization Name.
github_repo_webhook_create Create webhooks at repo level. Default "true".
github_repository_names List of repository names for which webhooks need to be created. Example, ["repo1","repo2"]
github_org_webhook_create Create webhooks at org level. Default "false".
github_repo_events List of repository events which should be sent to Sumo Logic. Example, ["create","delete","fork"]
github_org_events List of organization level events which should be sent to Sumo Logic. Example, ["create","delete","fork"]
Jenkins

The Terraform script does not configure the Jenkins Sumo Logic plugin, it creates an HTTP source in Sumo Logic for Jenkins and configures the Jenkins FERs.

Parameter Description
jenkins_sc Jenkins Source Category.