Skip to main content
Sumo Logic

Collect Metrics with Telegraf

Telegraf is a plugin-driven server agent for collecting and reporting metrics. Like CollectD, Logstash, and FluentD, it follows the input->output flow. It consists of numerous Input Plugins to read metrics from various systems, and numerous Output Plugins to send your data to. While Sumo Logic does not currently have an output plugin, there are multiple existing plugins that can be used to deliver data to Sumo Logic.

Collect Carbon 2.0 metrics using the Serializer and HTTP output plugins

This section has instructions for converting Telegraf metrics to the Carbon 2.0 format, and sending the metrics to a Sumo Logic hosted collector using the Telegraf  HTTP Output plugin.

We recommend this approach over collecting Graphite format metrics from Telegraf.  It is less effort to configure, and Carbon 2.0 metrics are easier to query. 

  1. Set up a Hosted Collector with an HTTP Source.
  2. Set up Telegraf to output data to the HTTP Source.

    This is an example configuration for the Telegraf agent using the HTTP Output plugin:

    [[outputs.http]]
      url = "https://collectors.sumologic.com/rec...ttp/XXXXXXXXXX"
      data_format = "carbon2"
      [outputs.http.headers]
        Content-Type = "application/vnd.sumologic.carbon2"
        X-Sumo-Category = "prod/someapp/metrics"

With this configuration, any data flowing into Telegraf from Input Plugins can be sent to Sumo Logic. In this example:

  • url  is the URL of the Sumo Logic HTTP Source.
  • data_format configures Telegraf to use the Carbon 2.0 Serializer plugin, which translates the Telegraf metric format to Carbon 2.0.
  • The Content-Type header is required to tell the Sumo Logic HTTP Source that it is receiving Carbon2 data.
  •  X-Sumo-Category sets the source category for collected metrics to  prod/someapp/metrics.  With that setting, you could search for collected metrics by using the following in your query:

    _sourceCategory=prod/someapp/metrics

Collect Graphite metrics using the Graphite output plugin

Telegraf has a Graphite Output plugin that can deliver metrics to Sumo Logic. You can set up an Installed Collector with a Streaming Metric Source, and configure Telegraf to send metrics to the source. The flow is as follows:

  1. Set up an Installed Collector with a Streaming Metrics Source.
  2. Set up Telegraf to Output data to that Streaming Metrics Source.

    This is an example configuration for the Telegraf agent using the Graphite Output plugin:

    [[outputs.graphite]]
      servers = ["localhost:2003"]
      data_format = "graphite"

In the above configuration, any data flowing into Telegraf from Input Plugins can be sent to Sumo Logic. In this example, the servers property is the URL to the Installed Collector. In this case, we assume the Telegraf agent and Sumo collector are on the same box, but they do not have to be. The data_format should be set to graphite so the Installed Collector can properly ingest the data from Telegraf. See the Graphite Output plugin for details on other configurable parameters.

Graphite output issues on Windows

If your Syslog data is missing any carriage returns or line feed characters (a CR LF, or \r \n), the following message will be written to the collector.log file in the Collector's log directory.

2018-12-18 03:22:52,091 -0500 [Thread-737420] ERROR com.sumologic.scala.collector.input.syslog.EventInput - Received event: Exception. server com.sumologic.scala.collector.input.syslog.TCPSyslogServer@5725403c, socketAddress /10.48.0.14:63557

java.net.SocketTimeoutException: Read timed out

        at java.net.SocketInputStream.socketRead0(Native Method)

        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)

This message is generated when the connection to the Collector times out awaiting an end of line character. You can resolve this problem by ensuring that the Syslog data includes carriage returns or line feed characters.

The problem relates to a known issue with the Graphite output plugin: 

https://github.com/influxdata/telegraf/issues/3510

Collect Graphite metrics using the HTTP output plugin

Telegraf has an HTTP Output plugin that can deliver metrics to Sumo Logic. Using a Hosted Collector with an HTTP Source, Telegraf can be configured to send data to the HTTP Source in Sumo Logic. The flow is as follows:

  1. Set up a Hosted Collector with an HTTP Source.
  2. Set up Telegraf to output data to the HTTP Source.

This is an example configuration for the Telegraf agent using the HTTP Output plugin:

[[outputs.http]]
  url = "https://collectors.sumologic.com/rec...ttp/XXXXXXXXXX"
  data_format = "graphite"
  [outputs.http.headers]
    Content-Type = "application/vnd.sumologic.graphite"
    X-Sumo-Category = “prod/someapp/metrics”

In the above configuration, any data flowing into Telegraf from their many Input Plugins can be sent to Sumo Logic. In this example:

  •  url property is the URL if the HTTP Source.
  • data_format is set to graphite
  • Content-Type header is required to tell the Sumo Logic HTTP Source that it is receiving Graphite data.
  • We can also set additional headers to control the Sumo Logic metadata. The example sets the X-Sumo-Categoryto prod/someapp/metrics so in Sumo Logic, you can find these metrics using _sourceCategory=prod/someapp/metrics.