Skip to main content
Sumo Logic

SDO Setup with Terraform Script

Learn how to set up the Software Development Optimization (SDO) Solution using a Terraform script.

These instructions help you prepare and set up the Software Development Optimization (SDO) Solution using a Terraform script. 

The Terraform script performs the following actions:

  • Creates Sumo Logic collection resources including Collector, Sources, and Folder.
  • Configures collection mechanisms in the supported tools, for example Webhooks in Pagerduty and GitHub.
  • Sets up Sumo Logic Field Extraction Rules (FERs) to enrich the data.
  • Configures connections in Sumo Logic to send alerts from Sumo Logic to issue management and incident management platforms including Jira, PagerDuty, and Opsgenie. 
  • Installs Sumo Logic Apps in the configured folder.

The Terraform script requires a set of parameters to be configured as defined here.

Before you begin

Before setting up the SDO solution, you need to complete the following tasks:

  • Create or collect API Keys and credentials for Jira, Github, GitLab, Jenkins, Bitbucket, CircleCI, and Opsgenie with Webhook creation capabilities. 

  • Get access to Jenkins with Manage Plugins permissions.

  • Make sure you have access to the Sumo logic console. The user account associated with a Sumo Logic role needs the following permissions:

    • Manage field extraction rules

    • View Fields

    • View field extraction rules

    • Manage Collectors

    • View Collectors

    • Manage Fields

    • Manage connections

    • Manage Content

  • Create a Sumo Logic API key and ID  for the above user.

Terraform Script Setup Considerations

The Terraform Script automates the setup of all the necessary resources for collecting Atlassian, GitHub, GitLab, and Pagerduty product logs and displaying the data in Sumo Logic pre-configured intuitive-analytic dashboards. 

Review the following considerations before proceeding with the Terraform template method:

  • With the Terraform script, you can easily manage the integration of Sumo Logic with Atlassian products (including Bitbucket, Jira, Jira Service Desk, and Opsgenie), GitHub, GitLab, CircleCI, and Pagerduty. If you would like to bring your own toolset, follow this page to integrate your tool with the SDO solution.
  • The Terraform script allows you to quickly get started by installing a copy of the configured applications. After the initial setup, if you need additional copies of the Sumo Logic applications, you can install them from the Sumo Logic App catalog.
  • If you plan to integrate Jenkins with this solution, you need to complete additional configuration. The Terraform script does not configure Jenkins. See the following guides to install and configure the Jenkins Sumo Logic plugin:
  • This script configures Jira Server WebHooks and creates resources in Sumo Logic. Jira Server Logs collection needs to be configured as explained in Step 1 here. Configure the log collection and update the variable jira_server_access_logs_sourcecategory in atlassian.auto.tfvars with the selected source category.
  • If you plan to integrate CircleCI with the SDO solution, you need to complete additional configuration. The Terraform script does not configure CircleCI. Use the following steps configure the CircleCI Sumo Logic plugin. Once configured, this plugin will send CircleCI Workflow and Job related data to Sumo Logic:
    • Since the SDO dashboards require ‘environment’, ‘team’, and ‘service’ fields for lighting up panels, you need to send them as custom-data in the configuration file of the pipeline.

    • Add the sumo orb in the configuration file of the project to send custom-data elements to Sumo:
      circleci-custom-data.png
      See this sample CircleCI Configuration file which sends CircleCI data, including custom-data, to Sumo Logic.

    • The Terraform script also allows you to only install the CircleCI app from the app catalog. It creates a source with placeholder value for _sourceCategory from the sumologic.auto.tfvars file.

  • Sumo Logic to Jira and Sumo Logic to Opsgenie Webhooks are in Beta. To participate, contact your Sumo account executive.

SDO Script File Groups

The SDO solution script is organized into following groups of files (GitHub repository):

Group Files
Configuration Files sumologic.auto.tfvars
sumologic_fer.auto.tfvars
sumologic_webhooks.auto.tfvars
pagerduty.auto.tfvars
github.auto.tfvars
gitlab.auto.tfvars
atlassian.auto.tfvars
circleci.auto.tfvars
Sumo Logic Resource Creation Files sumologic_collection_content.tf
sumologic_fer.tf
sumologic_webhooks.tf
Other Systems Resource Creation Files atlassian.tf
pagerduty.tf
github.tf
gitlab.tf
System Files outputs.tf
providers.tf
variables.tf
Test Files integration_test.go
fileutil.go
unit_tests.sh

Set Up the Terraform environment

Before you run the Terraform script, please perform the following actions on a server machine of your choice:

  1. Install Terraform version equal or greater than 0.12.20 and lower than 0.13.0. 
  2. Install the latest version of curl.
  3. Install Python version 2.7 or later.
  4. Install the following third-party Terraform providers:

    Rest API Terraform Provider version 1.12 and above:

    1. Download terraform-provider-restapi binary from Github
    2. Move the terraform-provider-restapi binary to $HOME/.terraform.d/plugins directory:

      mkdir -p $HOME/.terraform.d/plugins

      mv terraform-provider-restapi $HOME/.terraform.d/plugins/terraform-provider-restapi

      chmod +x terraform-provider-restapi

    If you plan to use Jira, install Jira Terraform Provider version  0.1.11 and above:

    1. Download terraform-provider-jira binary from GitHub.
    2. Unzip the zip file.
    3. Move the terraform-provider-jira binary to $HOME/.terraform.d/plugins directory:

      mkdir -p $HOME/.terraform.d/plugins

      mv terraform-provider-jira $HOME/.terraform.d/plugins/terraform-provider-jira

      chmod +x terraform-provider-jira

Configure the Terraform script

  1. Clone the GitHub repository:
    $ git clone https://github.com/SumoLogic/sumologic-solution-templates
  2. Initialize the Terraform working directory by navigating to the directory sumologic-solution-templates/software-development-optimization-terraform. and running terraform init. This will install the required Terraform providers: Sumo Logic Terraform Provider, Template, Null, BitBucket Terraform Provider, GitHub Terraform Provider, GitLab Terraform Provider, and Pagerduty Terraform Provider.
  3. Choose which Sumo Logic Applications to configure by updating the values of the following variables in the sumologic.auto.tfvars file:
    • install_jira_cloud
    • install_jira_server
    • install_bitbucket_cloud
    • install_opsgenie
    • install_github
    • install_gitlab
      • This is only used for installing Gitlab app from the app catalog and uses webhook.
      • Valid options for this are : ‘all’,’app’,’collection’,’fer’,’none’:
        • ‘all’  - It’s a default option, for installing all the components of the setup.
        • ‘app’ - This is for only installing the app
        • 'collection' - for configuring collection in Sumo Logic (fers, the sources) and/or in other systems (webhooks).
        • ‘fer’ - This will only create the fer.
        • 'none' - for skipping the entire integration/installation.
    • install_pagerduty
    • install_jenkins
    • install_sdo
      • The options for variable values are 'all', 'app', 'fer', 'collection', and 'none'. For SDO, 'app' and 'none' are the valid options.
      • For Jenkins, collection is not configured in Jenkins. Choosing 'collection' will create the source in Sumo and set up the field extraction rules.
      • If you do not want to install the GitHub collection or application, rename the file github.tf to github.tf_backup.
      • If you do not want to install the GitLab collection or application, rename the file gitlab.tf to gitlab.tf_backup.
    • install_circleci
      • This is only used for installing the CircleCI app from the app catalog and uses a webhook.
      • Valid options for this values are : 'all', 'app', 'collection', and 'none'. If you choose:
        • ‘all’  - It’s a default option, for installing every component of the setup
        • ‘app’ - for only installing the app
        • 'collection' - for configuring collection in Sumo Logic (fers, the sources) and/or in other systems (webhooks)
        • 'none' - for skipping the entire integration/installation
    • install_circleci_SDO_plugin
      • This is used to integrate CircleCI into the SDO solution.
      • valid options for this variable  are:  ‘all’ ,’none’ ,’fer’  and ‘collection’. If you choose:
        • ‘all’  - It’s a default option, for installing every component of the setup
        • ‘fer’ - for only installing the FERs
        • 'collection' - for configuring collection in Sumo Logic (fers, the sources) and/or in other systems (webhooks)
        • 'none' - for skipping the entire integration/installation
      • Choosing ‘collection’ will create two sources in Sumo and set up the field extraction rules.
  4. You can choose which Webhooks to configure by updating the variables in sumologic.auto.tfvars.
    • install_sumo_to_opsgenie_webhook
    • install_sumo_to_jiraserver_webhook
    • install_sumo_to_jiraservicedesk_webhook 
    • install_sumo_to_jiracloud_webhook
    • install_sumo_to_pagerduty_webhook 
  5. Update the following placeholder values in the sumologic.auto.tfvars file: atlassian.auto.tfvars, pagerduty.auto.tfvars, github.auto.tfvars, gitlab.auto.tfvars, sumologic_fer.auto.tfvars, and sumologic_webhooks.auto.tfvars so they correspond with your Sumo Logic, Atlassian, GitHub, GitLab, CircleCI, and Pagerduty environments. See Configurable Parameters for the full list of input parameters.

Install the Software Development Optimization (SDO) Solution

You have the following available methods to install the Software Development Optimization solution using the Terraform script:

Option 1: Install the complete SDO solution

This procedure will configure collection in other systems (for example Jira Cloud) and create sources and FERs/Fields and connections in Sumo Logic along with App installation:

  1. Set install_(app) variables as 'all' for the required applications in sumologic.auto.tfvars, as defined in this section. This configuration will result in installing Sumo Logic apps, configuring collectors, and creating field extraction rules and fields in Sumo Logic.
  2. Set install_(app) variable as 'none' for the tools that are not applicable to your environment.
  3. To install the outbound connections in Sumo Logic, configure the install_sumo_to_(app)_webhook variables as 'true' as defined in this section.
  4. Configure required variables in sumologic.auto.tfvars, atlassian.auto.tfvars, github.auto.tfvars, gitlab.auto.tfvars, pagerduty.auto.tfvars, and circleci.auto.tfvars.
  5. Navigate to the directory sumologic-solution-templates/software-development-optimization-terraform and execute the commands below:

$ terraform plan
$ terraform apply

Option 2: Install one or more parts of the solution

You have the following options when installing parts of the solution: installing only the Sumo Logic Apps, only the FERs, and only connections.

  • Install only the Sumo Logic Apps:  Choose this method if you are already collecting data and have set up Webhooks. This will not configure collectors, sources, FERs, and connections in Sumo Logic and will not configure webhooks in other systems.
  • Install only the FERs: Choose this method if you have already configured and set up the collection and have installed the relevant apps. This will not configure collectors, sources, App installation, and connections in Sumo Logic and will not configure Webhooks in other systems.
  • Install outgoing connections: Choose this method if you want to only configure Sumo Logic connections. This will not configure collectors, sources, App installation, and FERs in Sumo Logic and will not configure Webhooks in other systems.

Install only the Apps 

To use existing sources, FERs/Fields, or configure these manually and install only the applications: 

  1. Other than the SDO app, configure collection in Sumo Logic if not already done Based on the app documentation
  2. Populate source categories that you set up during the collection phase in sumologic.auto.tfvars, as defined in this section.
  3. Set install_(app) variables as ‘app’ for the required applications in sumologic.auto.tfvars, as defined in this section.
  4. Setting this flag as ‘app’ will result in app installation in Sumo Logic, it will not configure collection in other systems (for example Jira Cloud) and it will not create sources and FERs/Fields in Sumo Logic.
  5. Set install_(app) variables as ‘none’ for the applications which should not be configured.
  6. Set install_sumo_to_(app)_webhook variables as ‘false’ for the Sumo Logic connections if you do not wish to configure the outgoing connections in Sumo Logic.
  7. Navigate to the directory sumologic-solution-templates/software-development-optimization-terraform and execute the commands:

$ terraform plan

$ terraform apply

Install FERs

To use existing sources and apps, or configure these manually:

  1. Configure sources and source categories in Sumo Logic.
  2. Configure collection in respective systems for example Jira Cloud.
  3. Populate source categories in sumologic.auto.tfvars, as defined in this section.
  4. Set install_(app) variable as ‘fer’ for the required applications in sumologic.auto.tfvars, as defined in this section.
  5. Setting install_(app) variable as ‘fer’ will result in FER configuration in Sumo Logic, it will not configure collection in other systems (for example Jira Cloud) and it will not create sources and Fields in Sumo Logic and will not install Apps.
  6. Navigate to the directory 
    sumologic-solution-templates/software-development-optimization-terraform and execute the commands below:

$ terraform plan

$ terraform apply

Install the Sumo Logic outgoing connections

To configure the Sumo Logic connections only:

  1. Set install_(app)_webhook variables as ‘none’ for all the applications except Opsgenie. Opsgenie connection cannot be installed without configuring Opsgenie collection as there is a cyclic dependency.
  2. Set install_sumo_to_(app)_webhook variables as ‘true’ for the Sumo Logic connections which should be configured in Sumo Logic.
  3. Configure required variables in sumologic_webhooks.auto.tfvars.
  4. Navigate to the directory sumologic-solution-templates/software-development-optimization-terraform and execute the commands:
    $ terraform plan
    $ terraform apply

Complete Post-Setup Configurations

After completing configuration, instrument your DevOps pipeline to specially identify and send build and deploy events in your pipeline to Sumo Logic. These events are correlated with other events in the DevOps lifecycle.
Complete the configuration for the build and deploy tool you use.

Bitbucket for build and deploy

  1. Access the Sumo Logic Platform and navigate to Manage Data -> Collection page.

  2. Search for Software Development Optimization Collector.

  3. Under this Collector, click on Show URL for the source Bitbucket Cloud. Make a note of this URL and use this URL to configure the Bitbucket CI/CD Pipeline to collect deploy events:

    • Deploy: Follow the steps outlined in this document to configure the Bitbucket CI/CD Pipeline to collect deploy events.

Jenkins for build and deploy

  1. Install the latest Jenkins Plugin as described here

  2. Access the Sumo Logic Platform and navigate to Manage Data -> Collection page.

  3. Search for Software Development Optimization Collector.

  4. Under this Collector, click on Show URL for the source Jenkins. Make a note of this URL and Source Category, you will use these to configure the Jenkins Plugin :

    • Build Pipeline Stages: Follow Configure Jenkins Plugin, and optionally Optional - Advance Configuration to configure the Jenkins Sumo Logic plugin.

    • Build: Follow this doc to modify your Jenkins plugin to explicitly identify, enrich, and send Build Events to Sumo Logic.

    • Deploy: Follow this doc to modify your Jenkins plugin to explicitly identify, enrich, and send Deploy Events to Sumo Logic.

CircleCI for build and deploy

  1. Create three environment variables in project settings of CircleCI environment :

    • CIRCLE_TOKEN = <API personal token created in CircleCi>

    • JOB_HTTP_SOURCE = <url of job-collector source created by the Terraform script output>

    • WORKFLOW_HTTP_SOURCE = <url of workflow-collector source created by the Terraform script output>

CircleCI app installation

  1. Create a webhook connection in project settings of CircleCI environment : 

    • Receiver URL = <url of circleci source created by the Terraform script>

Other Tools for build and deploy

If you are using tools other than Jenkins and Bitbucket pipelines for Build and Deploy phases:

  1. Build: Configure your tool to construct and send events using the build event schema in the section Build Event. 

Deploy: Configure your tool to construct events using the deploy event schema in the section Deploy Events.

Uninstalling the Solution

To uninstall the solution, navigate to the directory sumologic-solution-templates/software-development-optimization-terraform and execute the command:

$ terraform destroy

Configurable Parameters

Configure the following parameters in specific files according to app or service.

Sumo Logic

Configure these parameters in sumologic.auto.tfvars.

Parameter Description Default
sumo_access_id Sumo Logic Access ID  
sumo_access_key Sumo Logic Access Key  
deployment Sumo Logic Deployment us1
sumo_api_endpoint Sumo Logic API Endpoint. Make sure the trailing "/" is present. https://api.sumologic.com/api/v1/
app_installation_folder The Sumo Logic apps will be installed in a folder under your personal folder in Sumo Logic. Software Development Optimization
install_jira_cloud Install Sumo Logic Application and WebHooks for Jira Cloud. Options: app, collection, fer, all, and none. all
install_jira_server Install Sumo Logic Application and WebHooks for Jira Server. Options: app, collection, fer, all, and none. all
install_bitbucket_cloud Install Sumo Logic Application and WebHooks for BitBucket Cloud. Options: app, collection, fer, all, and none. all
install_opsgenie Install Sumo Logic Application and WebHooks for Opsgenie. Options: app, collection, fer, all, and none. all
install_github Install Sumo Logic Application and WebHooks for Github. Options: app, collection, fer, all, and none.  If you do not wish to install the GitHub collection or application, rename the file github.tf to github.tf_backup. all
install_gitlab Install Sumo Logic Application and WebHooks for GitLab. Options: app, collection, fer, all, and none.  If you do not wish to install the Gitlab collection or application, rename the file gitlab.tf to gitlab.tf_backup. all
install_pagerduty Install Sumo Logic Application and WebHooks for Pagerduty. Options: app, collection, fer, all, and none. all
install_jenkins Install Sumo Logic Application for Jenkins. Options: app, collection, fer, all, and none.
The Terraform script does not configure the Jenkins Sumo Logic plugin, choosing ‘collection’ will create http source in Sumo Logic for Jenkins and will configure the Jenkins FERs.
 
all
install_sdo Install Sumo Logic Application for Software Development Optimization. Options: app or none. app
install_circleci

Install Sumo Logic Application for CircleCI. Options: app, collection, all, and none.

all
install_circleci_SDO_plugin Install Sumo Logic Collection and FER for CircleCI SDO integration. all
install_sumo_to_opsgenie_webhook Install Sumo Logic to Opsgenie WebHook. install_opsgenie should be true for this option to be true. true
install_sumo_to_jiracloud_webhook Install Sumo Logic to Jira Cloud WebHook. true
install_sumo_to_jiraserver_webhook Install Sumo Logic to Jira Server WebHook true
install_sumo_to_jiraservicedesk_webhook Install Sumo Logic to Jira Service Desk WebHook true
install_sumo_to_pagerduty_webhook Install Sumo Logic to Pagerduty WebHook true
jira_cloud_sc Source Category for Jira Cloud SDO/Jira/Cloud
jira_server_sc Source Category for Jira Server SDO/Jira/Server/Events
bitbucket_sc Source Category for BitBucket Cloud SDO/Bitbucket
opsgenie_sc Source Category for Opsgenie SDO/Opsgenie
pagerduty_sc Source Category for Pagerduty SDO/Pagerduty
github_sc Source Category for GitHub SDO/Github
gitlab_sc Source Category for GitLab SDO/Gitlab
jenkins_sc Source Category for Jenkins SDO/Jenkins
circlecl_app_sc Source Category for CircleCI SDO/CircleCI

Sumo Logic Field Extraction Rules

Configure these parameters in sumologic_fer.auto.tfvars. There is a set of FER's for the SDO Apps. Each FER needs scope and a parse expression. In most cases default values will suffice, if required you can edit the scope and parse expression as per your requirements.

Parameter Description
(app)_pull_request_fer_scope A keyword search expression that points to the subset of logs you'd like to parse. For more details see FER.
(app)_pull_request_fer_parse A valid parse expression with supported parse and search operators. For more details see FER.

Jira Cloud

Configure these parameters in atlassian.auto.tfvars.

Parameter Description
jira_cloud_url Jira Cloud URL
jira_cloud_user Jira Cloud Username
jira_cloud_password Jira Cloud Password or API Key
jira_cloud_jql Jira Cloud Query Language Example: "project = Sumo"
jira_cloud_events Jira Cloud Events

Sumo Logic to Jira Cloud Webhook

Configure these parameters in webhooks.auto.tfvars.

Parameter Description
jira_cloud_auth

To generate Authorization Header follow this doc

jira_cloud_user Jira Cloud Username
jira_cloud_password Jira Cloud Password or API Key
jira_cloud_jql Jira Cloud Query Language Example: "project = Sumo"
jira_cloud_events Jira Cloud Events

Sumo Logic to Jira Service Desk Webhook

Configure these parameters in webhooks.auto.tfvars.

Parameter Description
jira_servicedesk_url Jira Service Desk URL, can be same as Jira Cloud URL
jira_server_auth Basic Authorization Header
jira_server_projectkey Jira Server Project Key
jira_server_issuetype Jira Server Issue Type, for example 'Bug'
jira_server_priority Issue Priority. For Example, 3

Jira Server

Configure these parameters in atlassian.auto.tfvars.

Parameter Description
jira_server_access_logs_sourcecategory Jira Server Access Logs Source Category, default "SDO/Jira/Server*", refer to this link.
jira_server_url Jira Server URL
jira_server_user Jira Server Username
jira_server_password Needs to be the password. API Key is not supported on Jira Server yet.
jira_server_jql Jira Server Query Language Example: "project = Sumo"
jira_servicedesk_priority Jira Server Events

Sumo Logic to Jira Server Webhook

Configure these parameters in webhooks.auto.tfvars.

Parameter Description
jira_server_auth Basic Authorization Header
jira_server_projectkey Jira Server Project Key
jira_server_issuetype Jira Server Issue Type, for example 'Bug'
jira_server_priority Issue Priority, for example 3

Bitbucket

Configure these parameters in atlassian.auto.tfvars.

Parameter Description
bitbucket_cloud_user Bitbucket Username
bitbucket_cloud_password Bitbucket password or App Password
bitbucket_cloud_owner The owner of the repositories. Can be you or any team having write access.
bitbucket_cloud_repos Specify the repositories for which WebHooks should be created. Format: ["repo1","repo2"]
bitbucket_cloud_desc The name / description to show in the UI
bitbucket_cloud_events Bitbucket Events to track

Opsgenie

Configure these parameters in atlassian.auto.tfvars.

Parameter Description
opsgenie_api_url Opsgenie API URL. Do not add the trailing "/". If using the EU instance of Opsgenie, the URL needs to be https://api.eu.opsgenie.com for requests to be executed.
opsgenie_key Opsgenie API Key

Sumo Logic to Opsgenie Webhook

Configure these parameters in webhooks.auto.tfvars.

Parameter Description
opsgenie_priority Opsgenie Alert Priority

PagerDuty

Configure these parameters in pagerduty.auto.tfvars.

Parameter Description
pagerduty_api_key Pagerduty API Key.
pagerduty_services_pagerduty_webhooks List of Pagerduty Service IDs. Example, ["P1QWK8J","PK9FKW3"]. You can get these from the URL after opening a specific service in Pagerduty. These are used for Pagerduty to Sumo Logic webhooks.

Sumo Logic to PagerDuty Webhook

Configure these parameters in webhooks.auto.tfvars.

Parameter Description
pagerduty_services_sumo_webhooks Sumo Logic to Pagerduty Webhook. List of Pagerduty Service IDs. Example, ["P1QWK8J","PK9FKW3"]. You can get these from the URL after opening a specific service in Pagerduty. These are used for Sumo Logic to Pagerduty Webhooks.

GitHub

Configure these parameters in github.auto.tfvars. If you do not wish to install the GitHub collection or application, rename the file github.tf to github.tf_backup.

Parameter Description
github_token Github Token
github_organization Organization Name.
github_repo_webhook_create Create webhooks at repo level. Default "true".
github_repository_names List of repository names for which webhooks need to be created. Example, ["repo1","repo2"]
github_org_webhook_create Create webhooks at org level. Default "false".
github_repo_events List of repository events which should be sent to Sumo Logic. Example, ["create","delete","fork"]
github_org_events List of organization level events which should be sent to Sumo Logic. Example, ["create","delete","fork"]

GitLab

Configure these parameters in gitlab.auto.tfvars. If you do not wish to install the Gitlab collection or application, rename the file gitlab.tf to gitlab.tf_backup.

Parameter Description
gitlab_token GitLab Token
gitlab_project_webhook_create Create webhooks at project level. Default "true".
gitlab_project_names List of project names for which webhooks need to be created. Example, ["project1","project2"]

Jenkins

The Terraform script does not configure the Jenkins Sumo Logic plugin, it creates an HTTP source in Sumo Logic for Jenkins and configures the Jenkins FERs.

Parameter Description
jenkins_sc Jenkins Source Category.

CircleCI

The Terraform script does not configure the CircleCI Sumo Logic plugin, it creates HTTP sources in Sumo Logic for CircleCI and configures the CircleCI FERs.

It also allows to install the CircleCI app that can be configured using a CircleCI webhook.

Parameter Description
circlesi_app_sc CircleCI App Source Category
circleci_build_jobname CircleCI build job name for SDO FER
circleci_deploy_jobname CircleCI deploy job name for SDO FER