Skip to main content
Sumo Logic

Using collectd for Metrics

Use Sumo's collectd plugin to collect Carbon 2.0 metrics.

Overview

For metric collection, we recommend that you either stream your metric data using an installed collector, or use an open source metric collection agent, such as collectd, to send your metrics directly to an HTTP endpoint in the Sumo service, using our HTTP Source for Logs and Metrics.  We recommend collectd for metric collection when the Sumo collector does not support your requirements.

Sumo's collectd plugin sends data from collectd to an HTTP endpoint in the multi-dimensional Carbon 2.0 metric format.

The Sumo collectd plugin is available at https://github.com/SumoLogic/sumologic-collectd-plugin

Prerequisites

Python v2.6 or later is required. Download the desired Python version from the Python download page.

Step 1: Install collectd

If collectd is already installed, skip this step. Otherwise, follow the instructions for download and installation on the collectd download site. For additional details, see the First Steps page on the the collectd Wiki.

Debian / Ubuntu

sudo apt-get install collectd

Step 2: Download and install Sumo collectd plugin

Use one of the options below to install the Sumo collectd plugin.

Option A: Install plugin as a library

To install the collectd plugin as a library, enter the following command in a terminal window:

sudo pip install sumologic_collectd_metrics

All required dependencies will be automatically installed with a library installation.

Option B: Install plugin with source code

The Sumo collectd plugin source code can be saved in a directory anywhere on your system. Download it from PythonIdex > Package Index > sumologic_collectd_metrics, or:

git clone https://github.com/SumoLogic/sumologic-collectd-plugin.git

The Sumo collectd plugin uses requests and retry libraries to submit HTTPS requests. If they are not installed, install them using pip:

sudo pip install requests
sudo pip install retry

Step 3: Configure an HTTP Source in Sumo

In this step you create, on the Sumo service, an HTTP endpoint to receive your metrics. This process involves creating an HTTP source on a hosted collector in Sumo. In Sumo, collectors use sources to receive data.

  1. If you don’t already have a Sumo account, you can create one by clicking the Free Trial button on https://www.sumologic.com/.
  2. Create a hosted collector, following the instructions on Configure a Hosted Collector in Sumo help. (If you already have a Sumo hosted collector that you want to use, skip this step.)
  3. Create an HTTP source on the collector you created in the previous step. For instructions, see HTTP Logs and Metrics Source in Sumo help.
  4. When you have configured the HTTP source, Sumo will display the URL of the HTTP endpoint. Make a note of the URL. You will use it when you configure the collectd plugin to send data to Sumo.

Step 4: Configure Sumo collectd plugin

You configure the behavior of the Sumo collectd plugin by defining the values of parameters in the collectd.conf file in the /etc/collectd/ directory. The subsections below describe required, recommended, and advanced parameters. You must define the required parameters. It is a very good idea to define the recommended parameters. Default values for advanced parameters are listed in advanced parameters. You can set different values as desired in the collectd.conf file.

The excerpt from collectd.conf below is an example configuration for the plugin.

LoadPlugin python
<Plugin python>
    # Uncomment and update the following line if sumologic collectd plugin installed with source code
    # ModulePath "/path/to/sumologic-collectd-plugin"
    LogTraces true
    Interactive false
    Import "sumologic_collectd_metrics"

    <Module "sumologic_collectd_metrics">
        TypesDB "/path/to/your/collectd/share/collectd/types.db", "/path/to/my_own_types.db"  
        # At lease one types.db file must be specified
        URL "https://<deployment>.sumologic.com/receiver/v1/http/<source_token>"  
        # URL must be specified
        # Uncomment and update the following lines to override the metadata configured in the the Sumo HTTP source.
        # SourceName "my_source"
        # SourceHost "my_host"
        # SourceCategory "my_category"
        # Uncomment and update the following lines to add additional key=value pairs
        # Dimensions "my_dim_key1"="my_dim_val1", "my_dim_key2"="my_dim_val2"
        # Metadata "my_meta_key1"="my_meta_val1", "my_meta_key2"="my_meta_val2"
    </Module>
</Plugin>

Required parameters

The parameters below are required and must be specified in the module config.

Name Description Type Required
URL The URL to send metrics to. This URL is presented to you when you create the HTTP source in the Sumo web app, as described in Step 3: Configure an HTTP Source in Sumo.    String   Yes
TypesDB Data-set specification for collectd raw data. More information about types.db is available in collectd types.db.

Collectd ships with a default types.db file in the collectd root directory, for example, /usr/share/collectd/types.db.
Strings in the format:

"types1.db", "types2.db"
Yes

Recommended parameters

The parameters below are not strictly required, however we recommend you configure them to categorize your metrics and make it easier to search for them in Sumo. 

Name Description Type Required
SourceName Name of the metrics source. _sourceName can be used to search metrics from this source.

The value you specify overrides the one you configured for the HTTP source in Step 3: Configure an HTTP Source in Sumo.  
String No
SourceHost Name of metrics host. _sourceHost can be used to search metrics from this host.

The value you specify overrides the one you configured for the HTTP source in Step 3: Configure an HTTP Source in Sumo.  
String No
SourceCategory Category of the collected metrics. _sourceCategory can be used to search metrics from this category. It will override the default configured in the the Sumo source configuration. 

The value you specify overrides the one you configured for the HTTP source in Step 3: Configure an HTTP Source in Sumo.  
String No
Dimensions Key value pairs that help identify a metric. collectd data have intrinsic dimensions with keys like hostpluginplugin_instancetypetype_instanceds_name, and ds_type.

You can specify additional dimensions to help distinguish metrics from this collectd instance with metrics from other collectd instances.

Dimensions cannot contain reserved symbols or keywords. For more information, see Reserved characters and keywords.
Srings in the format: "key1"="val1", "key2"="val2"... No
Metadata Key value pairs that help identify a metric. collectd data have intrinsic dimensions with keys like hostpluginplugin_instancetypetype_instanceds_name, and ds_type.

You can specify additional dimensions to help distinguish metrics from this collectd instance with metrics from other collectd instances.

Dimensions cannot contain reserved symbols or keywords. For more information, see Reserved characters and keywords.
Srings in the format:"key1"="val1", "key2"="val2"... No

Advanced parameters

You can configure the Sumo collectd plugin by overriding default values for plugin parameters.

Name Description Type Default Unit
MaxBatchSize The plugin batches metrics before sending them over https. MaxBatchSize defines the upper limit of metrics per batch. Positive Integer 5000 NA
MaxBatchInterval The plugin batches metrics before sending them via HTTPS. MaxBatchInterval defines the upper limit of duration to construct a batch. Positive Integer 1 Second
HttpPostInterval The plugin schedules https post requests at fixed intervals. HttpPostInterval defines the frequency for the scheduler to run. If no metrics batch is available at the time, the scheduler immediately returns. If multiple metrics batches are available, then the oldest batch is picked to be sent. Positive Float 0.1  
MaxRequestsToBuffer The plugin buffers failed and delayed metrics batch requests. MaxRequestsToBuffer specifies the maximum number of these requests to buffer. After the buffer becomes full, the request with oldest metrics batch will be dropped to make space for new metrics batch. Positive Integer 1000 NA
RetryInitialDelay The plugin retries on recoverable exceptions. RetryInitialDelay specifies the initial delay before a retry is scheduled. For more information, see retry library. Non-negative Integer 0 Second
RetryMaxAttempts The plugin retries on recoverable exceptions. RetryMaxAttempts specifies the upper limit of retries before the current retry logic fails. The metric batch is then either put back for the next run (when metrics buffer specified by MaxRequestsToBuffer is not full), or dropped (when metrics buffer is full). For more information, see retry library. Positive Integer 10 NA
RetryMaxDelay The plugin retries on recoverable exceptions. RetryMaxDelay specifies the upper limit of delay before the current retry logic fails. Then the metric batch either is put back for the next run (when metrics buffer specified by MaxRequestsToBuffer is not full), or dropped (when metrics buffer is full). For more information, see retry library. Positive Integer 100 Second
RetryJitterMin The plugin retries on recoverable exceptions. RetryJitterMin specifies the minimum extra seconds added to delay between attempts. For more information, see retry library. Non-negative Integer 0 Second
RetryJitterMax The plugin retries on recoverable exceptions. RetryJitterMax specifies the maximum extra seconds added to delay between attempts. For more information, see retry library. Non-negative Integer 10 Second
ContentEncoding The content encoding used to compress HTTP entity-body. Allowable values: "deflate", "gzip", or "none". String "deflate" NA

Reserved characters and keywords

In collectd.conf, the equals sign and space character are reserved: "=", " "

The following keywords are reserved: _sourcehost, _sourcename, _sourcecategory, _collectorid, _collector, _source, _sourceid, _contenttype, _rawname

Step 5: Configure input plugins (Optional)

We recommend you configure the log and csv input plugins. For information on these plugins, see Plugin:CSV and Plugin:LogFile on the collecd Wiki.

You may also want to configure the plugins available for collecting cpu, memory, disk, and network metrics. 

As with the output plugin, you configure input plugins in collectd.conf:  

LoadPlugin logfile
<Plugin logfile>
        LogLevel "info"
        File "/var/log/collectd.log"
        Timestamp true
        PrintSeverity true
</Plugin>
LoadPlugin csv
<Plugin csv>
        DataDir "/usr/local/var/lib/collectd/csv"
</Plugin>
LoadPlugin cpu
LoadPlugin memory
LoadPlugin disk
LoadPlugin interface

For a complete list of collectd plugins, see Table of Plugins on the collectd Wiki.

Step 6: Start sending metrics

Start sending metrics by running collectd. For example, (the command will differ depending on your collectd installation):

sudo service collectd start

Step 7: View logs

If you installed the logfile input plugin, you can view logs by tailling the collectd.log file. For example (the command may vary depend in your collectd installation):

tail -f /var/log/collectd.log

Step 8: View metrics

To view the metrics sent by the collectd plugin, log into Sumo Logic and open a Metrics tab. Query for metrics using either dimensions or metadata, for example:

_sourceName=my_source _sourceHost=my_host _sourceCategory=my_category plugin=cpu

Metrics should appear in the chart area of the page. 

Reference information

Naming Schema

collectd uses a powerful naming schema to identify each statistics value. It has been proven to be very generic and flexible, but may be confusing to new users. To learn more, see Naming schema on the collectd Wiki.

Compression

Metrics are batched and compressed before they are sent. The compression algorithm is "deflate". The algorithm is explained in An Explanation of the Deflate Algorithm. Alternately, you can specify "gzip" for gzip compression and "none" for no compression.

Error handling

The Sumo collectd plugin retries on exceptions by default. When all retries fail, the request is either scheduled for a future attempt or dropped based on the buffer status. By default, 1,000 requests are buffered. If the buffer becomes full, then requests failed after all retries will be dropped. Otherwise, it is put back to the processing queue for the next run.