Skip to main content
Sumo Logic

Collect Logs and Metrics for Jenkins

  1. Last updated
  2. Save as PDF
This page provides instructions for configuring log and metric collection for the Sumo App for Jenkins.

This page provides instructions for configuring log and metric collection for the Sumo Logic App for Jenkins.

Collection overview

Configuring log and metric collection for the Jenkins App includes the following tasks:

  1. Configure a Collector
  2. Configure HTTP Logs and Metric source
  3. Install Jenkins Plugin
  4. Configure Jenkins Plugin
  5. Optional - Advanced Configuration

Configure a Collector

To create a new Sumo Logic hosted collector, perform the steps in the Configure a Hosted Collector section of the Sumo Logic documentation.

Configure an HTTP Log and Metric Source

To create a new HTTP logs and metric source, do the following:

  1. Create a new HTTP Logs and Metrics Source in the hosted collector created above by following these instructions.
  2. Make a note of HTTP Source URL and Source Category, as you will need them later in the configuration process.

Install the Jenkins Plugin

This section walks you through the ways in which you can install the Jenkins plugin:

Updating the Jenkins plugin

This section shows you how to update the version of the Jenkins plugin you have running on your system.

To update the Jenkins plugin, do the following:

  1. In the Jenkins console, go to Manage Jenkins > Manage Plugins.

  2. Click the Updates tab > Check now.

Jenkins_Update-1.png

  1. Select "Sumologic Publisher".

  2. Click Download now and install after restart. You may need to restart Jenkins for the plugin to show up.

Jenkins_Update-2.png

Installing with sumologic publisher for the first time

This section shows you how to install the Jenkins plugin, if this is your first time using sumologic publisher.

To perform a first time installation, do the following:

  1. In the Jenkins console, go to Manage Jenkins > Manage Plugins.

Jenkins_1st-time-install.png

  1. Click the Available tab.

  2. Search for "Sumologic Publisher".

  3. Click "Sumologic Publisher".

  4. Click Download now and install after restart. You may need to restart Jenkins for the plugin to show up.

Jenkins_1st-time-install-1.png

Manually installing the Jenkins plugin

This section shows you how to manually install the Jenkins plugin.

To install the Jenkins Plugin, do the following:

  1. Download the Jenkins plugin HPI file from this link.

  2. In the Jenkins console, go to Manage Jenkins > Manage Plugins. 

  3. Go to Advanced > Upload Plugin and click Choose File. 

Jenkins_Upload_Plugin.png

  1. Select the downloaded Sumologic-publisher.hpi file from your machine and click Upload.

Jenkins_Upload_button.png

A page similar to the following appears once the plugin is installed.

Jenkins_Plugin_Success.png

  1. Select Restart Jenkins when installation is complete and no jobs are running. You may need to restart Jenkins for the plugin to appear.

Configure Jenkins Plugin

This task walks you through configuring the Jenkins Plugin for Sumo Logic.

To configure the Jenkins Plugin, do the following:

  1. Go to Jenkins > Manage Jenkins and select Configure System.

Jenkins_Plugin_Configure-System.png

  1. Search for the Sumo Logic Jenkins Plugin.
  2. In the Sumo Logic Jenkins Plugin dialog,  specify the following:
  • SumoLogic Portal Name - This is the domain name of the hosted collector. For example, service.sumologic.com or service.us2.sumologic.com.
  • Metric Data Prefix: Enter the name of the Jenkins Master on which plugin is installed.
  • HTTP Source URL: HTTP Source URL from step 2. The HTTP source URL will look something like this: https://sumologic_url/receiver/v1/http/SECRET_STRING
  • Source Category: This is the Source Category you defined for the source.
  • Keep Old Configuration for Jobs: Enable this option to send old configuration for jobs.

Jenkins_Plugin_Configure-Sumo-Logic.png 

  1. Specify the types of data you want to send to Sumo Logic:
  • Metric Data - Sends metric information.
  • Audit Logs - Sends audit information like login, logouts, login failures, configuration changes to jobs and the Jenkins servers.
  • Periodic Logs - Sends periodic information like node information, master information, shutdown events, Jenkins system logs.
  • SCM logs - Send Source Control Management logs related to builds.
  1. Specify the following:
  • Job Status Information: Select to send status for all jobs
  • Console Logs Information: Select to send console logs for all jobs
  1. Jenkins plugin can be configured with an HTTP proxy. Once this is configured all the logs which are being sent from the Jenkins plugin will go through the proxy before getting ingested in sumo logic. Following are the parameters which are introduced to support the same:
  • Enable Proxy Setting (checkbox): Check to Enable proxy setting
  • Proxy Host: User to input the proxy server host-name here.
  • Proxy port: Specify the port of the proxy server.
  • Enable Proxy Authentication(checkbox):  Check to enable Authentication for the proxy. This can be used if the proxy is configured with authentication.
  • Username: Username to be used for the proxy Authentication.
  • Password: Password to be used for proxy authentication.clipboard_ed1a9556faa1acf2c8d6ec51392a3119b.png
  1. Click Apply, and then click Save.

Optional - Advanced Configuration

This section provides instructions for configuring Sumo Logic Jenkins Plugin for specific projects using Jenkins configuration as code. For more information see configuration as code.

To configure Sumo Logic Jenkins Plugin for specific projects using configuration as code, do the following:

  1. The plugin can be configured using configuration as code provided by Jenkins as detailed in this document.
  2. In case of specific jobs, do the following:
    1. For freestyle and maven Projects, go to Job Configuration and in the Post-build Actions, select Sumo Logic Build Logger.

Jenkins_Plugin_Advanced-config-1.png

  1. For Pipeline projects, do the following, 
  1. In the pipeline configuration, for normal script make the following as the top level.
SumoPipelineLogCollection {
// your script

Jenkins_Plugin_Pipeline1.png

  1. Then, in the pipeline configuration, for declarative pipeline script update the option.
options {
SumoPipelineLogCollection()
}

Jenkins_Plugin_Pipeline2nd.png

  1. Click Apply, and then click Save.

Sample Log Message

The following table shows sample log messages for different log types.
 

Log Type

Sample

Authentication

{

  "userName": "Sourabh Jain",

  "auditEventType": "User_Login",

  "userId": "sourabh",

  "message": "sourabh Logged in.",

  "logType": "Audit_Event",

  "eventTime": "2019-06-06 05:55:38,609 +00:00"

}

Configuration Change

{

  "userName": "demoexample@sumologic.com",

  "auditEventType": "Config_Change",

  "userId": "demoexample@sumologic.com",

  "message": "demoexample@sumologic.com changed configuration for the file jobs/demoexample-gradle-dep-check/config.xml.",

  "fileDetails": {

    "Old_File_Data": "PD94bWwgdmVyc2lvbj0nMS4xJyBlbmNvZGluZz0nVVRGLTgnPz4KPGZsb3ctZGVmaW5pdGlvbiBwbHVnaW49IndvcmtmbG93LWpvYkAyLjMyIj4KICA8YWN0aW9ucz4KICAgIDxvcmcuamVua2luc2NpLnBsdWdpbnMucGlwZWxpbmUubW9kZWxkZWZpbml0aW9uLmFjdGlvbnMuRGVjbGFyYXRpdmVKb2DwvYXV0aFRva2VuPgogIDxkaXNhYmxlZD5mYWxzZTwvZGlzYWJsZWQ+demoexamplefile==",

    "Current_File_Data": "PD94bWwgdmVyc2lvbj0nMS4xJyBlbmNvZGluZz0nVVRGLTgnPz4KPGZsb3ctZGVmaW5pdGlvbiBwbHVnaW49IndvcmtmbG93LWpvYkAyLjMyIj4KICA8YWN0aW9ucz4KICAgIDxvcmcuamVua2luc2NpLnBsdWdpbnMucGlwZWxpbmUubW9kZWxkZWZpbml0aW9uLmFjdGlvbnMuRGVjbGFyYXRpdnZ2Vycy8+CiAgPGF1dGhUb2tlbj5zd29yZGZpc2g8L2F1dGhUb2tlbj4KICA8ZGlzYWJsZWQ+ZmFsc2U8L2Rpc2FibGVkdemoexamplefile="

  },

  "logType": "Audit_Event",

  "eventTime": "2019-06-05 15:17:28,237 +00:00"

}

Shutdown

{

  "logType": "Slave_Event",

  "eventTime": "2019-06-05 11:00:12,256 +00:00",

  "eventSource": "Shutdown"

}

Jenkins Log

{

  "threadId": 33675,

  "logType": "Jenkins_Log",

  "logLevel": "INFO",

  "logMessage": "Finished Sumo Logic Periodic Data Publisher. 102 ms",

  "logSource": "com.sumologic.jenkins.jenkinssumologicplugin.sender.SumoPeriodicPublisher",

  "eventTime": "2019-06-06 07:08:06,908 +00:00"

}

Periodic Update

{

  "numberOfSlaves": 0,

  "numberOfExecutors": 4,

  "numberOfFreeExecutors": 2,

  "nodeName": "build-jenkins-executor (i-07e3500c4cda1f30b)",

  "nodeLabel": "gradle docker",

  "nodeStatus": "updated",

  "isIdle": false,

  "isOnline": true,

  "isRemoved": false,

  "isConnecting": false,

  "nodeURL": "https://localhost:8080/computer/buil...00c4cda1f30b)/",

  "eventSource": "Periodic_Update",

  "monitorData": {

    "SwapSpaceMonitor": "Memory:48519/63624MB  Swap:0/0MB",

    "ClockMonitor": "In sync",

    "DiskSpaceMonitor": "85.71 GB",

    "ResponseTimeMonitor": "13ms",

    "ArchitectureMonitor": "Linux (amd64)",

    "TemporarySpaceMonitor": "85.71 GB"

  },

  "logType": "Slave_Event",

  "eventTime": "2019-06-06 07:05:06,702 +00:00"

}

In Queue

{

  "numItemsInQueue": 0,

  "numBlockedItemsInQueue": 0,

  "maxWaitingTime": 0,

  "averageWaitingTime": 0,

  "queueId": 3060904,

  "queueTime": 56.093,

  "isBlocked": true,

  "reasonForBlock": "All nodes of label ‘docker’ are offline",

  "isConcurrentBuild": false,

  "jobName": "part of Module ITs: vault #142",

  "jobURL": "https://localhost:8080/job/module-it-vault/142/",

  "logType": "Queue_Event",

  "eventTime": "2019-06-06 06:50:06,668 +00:00"

}

In Progress

{

  "name": "TestComplexPipeline",

  "result": "In_Progress",

  "number": 11,

  "start_time": 0,

  "duration": 0,

  "jobStartTime": "2019-06-04 03:39:57,790 +00:00",

  "jobRunDuration": 13.175,

  "jobBuildURL": "http://localhost:8080/job/TestComplexPipeline/11/",

  "label": "Master",

  "nodeName": "master"

}

SCM Logs

{

  "buildNumber": 3,

  "jobName": "master",

  "scmType": "hudson.plugins.git.GitSCM",

  "scmURLs": "git https://github.com/SumoSourabh/build...ne-project.git",

  "revision": "7bb6c1d7b1e3d7d65a39bade4f2ad68196322b2c",

  "branches": "master ",

  "changeLog": [

    "1559619601000 commit:aa058c84927345885838c17bb7db448910885c6b author:Sourabh Jain message:commit",

    "1559621063000 commit:7bb6c1d7b1e3d7d65a39bade4f2ad68196322b2c author:Sourabh Jain message:commit"

  ],

  "logType": "Scm_Status",

  "eventTime": "2019-06-04 04:05:15,723 +00:00"

}

Job Status Logs

{

  "name": "MyTest",

  "hudsonVersion": "2.174",

  "result": "UNSTABLE",

  "description": "This is a testPlease . alsncns",

  "number": 35,

  "start_time": 1560239332099,

  "duration": 17401,

  "logType": "Job_Status",

  "user": "sourabh",

  "jobStartTime": "2019-06-11 07:48:52,096 +00:00",

  "jobType": "Maven project",

  "jobRunDuration": 17.401,

  "jobBuildURL": "http://ec2-13-234-181-219.ap-south-1...job/MyTest/35/",

  "upstreamJobURL": "",

  "triggerCauses": "Started by user Sourabh Jain",

  "label": "Master",

  "nodeName": "master",

  "jobMetaData": {

    "NODES": "1,2,3",

    "ENV": "qa",

    "APPS": "app01,app02"

  },

  "testResult": {

    "failures": 24,

    "passes": 238,

    "skips": 59,

    "total": 321,

    "totalDuration": 0.10700000333599746

  }

}

Console Logs

[2019-06-04 09:02:37,386 +00:00] channel stopped

Test Result

{

  "logType": "Test_Result",

  "number": 35,

  "name": "MyTest",

  "testResult": [

    {

      "className": "TwentyTwoTest",

      "testName": "test4",

      "errorStackTrace": "java.lang.AssertionError: oops\n\tat org.junit.Assert.fail(Assert.java:88)\n\tat Base.BaseTestClassMethod(Base.java:14)\n\tat TwentyTwoTest.test4(TwentyTwoTest.java:21)\n",

      "errorDetails": "oops",

      "status": "Failed",

      "duration": 0

    },

    {

      "className": "TwentyTwoTest",

      "testName": "test5",

      "status": "Passed",

      "duration": 0

    }

  ]

}

Pipeline Stages

{

  "logType": "Pipeline_Stages",

  "number": 23,

  "name": "TestComplexPipeline",

  "stages": [

    {

      "id": 1,

      "stageId": "10",

      "name": "parallel stage",

      "status": "SUCCESS",

      "startTime": "2019-06-11 07:55:50,240 +00:00",

      "duration": 0.174,

      "pauseDuration": 0

    },

    {

      "id": 2,

      "stageId": "14",

      "name": "build",

      "status": "SUCCESS",

      "startTime": "2019-06-11 07:55:50,414 +00:00",

      "duration": 0.056,

      "pauseDuration": 0,

      "steps": [

        "StepName - Print Message,StepStatus - SUCCESS,StepDuration - 0.005,StepArguments - Building the app [app01] on node [1],StepExecutedOn - (master)"

      ]

    },

    {

      "id": 3,

      "stageId": "19",

      "name": "deploy",

      "status": "SUCCESS",

      "startTime": "2019-06-11 07:55:50,492 +00:00",

      "duration": 0.052,

      "pauseDuration": 0,

      "steps": [

        "StepName - Print Message,StepStatus - SUCCESS,StepDuration - 0.005,StepArguments - Deploying the app app01] on node [1],StepExecutedOn - (master)"

      ]

    }

  ]

}

Query sample 

This sample Query is from the Jobs in Progress panel of the Jenkins - Job Overview dashboard.

_sourceCategory=Labs/Jenkins/Sourabh/Logs In_Progress
| json "name", "result", "number" as Job_Name, Result, Job_Number
| where Result = "In_Progress"
| first(_raw) by Job_Name, Job_Number
| json field=_first "jobBuildURL", "label", "nodeName", "jobStartTime", "jobRunDuration" as Job_URL, Job_Label, Node_Name, Job_Start_Time, Job_Run_Duration nodrop
| where ![subquery : _sourceCategory=Labs/Jenkins/Sourabh/Logs Job_Status
| json "name", "number" as Job_Name, Job_Number   
| count by Job_Name, Job_Number
| compose Job_Number, Job_Name]
| tourl(Job_URL, Job_Name) as Job_URL
| count by Job_Name, Job_URL, Job_Label, Node_Name, Job_Start_Time, Job_Run_Duration, Job_Number
| fields Job_Name, Job_URL, Job_Label, Node_Name, Job_Start_Time, Job_Run_Duration, Job_Number