Skip to main content
Sumo Logic

Collect Logs and Metrics for Non-Kubernetes environments

This page assists to collect Logs and Metrics for Non-Kubernetes environments.

We use the Telegraf operator for HAProxy metric collection and Sumo Logic Installed Collector for collecting HAProxy logs. The diagram below illustrates the components of the HAProxy collection in a non-Kubernetes environment. Telegraf runs on the same system as HAProxy, and uses the HAProxy input plugin to obtain HAProxy metrics, and the Sumo Logic output plugin to send the metrics to Sumo Logic. Logs from HAProxy on the other hand are sent to either a Sumo Logic Local File source or Syslog source.

This section provides instructions for configuring metrics collection for the Sumo Logic App for HAProxy. Follow the below instructions to set up the metric collection:

  1. Configure Metrics Collection

    1. Configure a Hosted Collector

    2. Configure an HTTP Logs and Metrics Source

    3. Install Telegraf

    4. Configure and start Telegraf

  2. Configure Logs Collection

    1. Configure logging in HAProxy

    2. Configure Sumo Logic Installed Collector

Step 1 Configure Metrics Collection

  1. Configure a Hosted Collector:
    To create a new Sumo Logic hosted collector, perform the steps in the Configure a Hosted Collector section of the Sumo Logic documentation.

  2. Configure an HTTP Logs and Metrics Source:
    Create a new HTTP Logs and Metrics Source in the hosted collector created above by following these instructions. Make a note of the HTTP Source URL.

  3. Install Telegraf:
    Use the following steps to install Telegraf.

  4. Configure and start Telegraf:
    As part of collecting metrics data from Telegraf, we will use the HAProxy input plugin to get data from Telegraf and the Sumo Logic output plugin to send data to Sumo Logic. 

    Create or modify telegraf.conf and copy and paste the text below:

Please enter values for the following parameters (marked in bold above):

  • In the input plugins section: 
    • servers - The URL to the HAProxy server. This can be a comma-separated list to connect to multiple HAProxy servers. Please see this doc for more information on additional parameters for configuring the HAProxy input plugin for Telegraf.
    • In the tags section [inputs.haproxy.tags]
      • environment - This is the deployment environment where the HAProxy server identified by the value of servers resides. For example: dev, prod or qa. While this value is optional we highly recommend setting it. 
      • proxy_cluster - Enter a name to identify this HAProxy cluster. This cluster name will be shown in the Sumo Logic dashboards. 
  • In the output plugins section: 
    • url - This is the HTTP source URL created in step 2. Please see this doc for more information on additional parameters for configuring the Sumo Logic Telegraf output plugin.

Here’s an explanation for additional values set by this Telegraf configuration that we request you please do not modify as they will cause the Sumo Logic apps to not function correctly.

  • data_format - “prometheus” In the output plugins section for which metrics are sent in the Prometheus format to Sumo Logic.
  • proxy_system: “haproxy” - In the input plugins section:  This value identifies the proxy system.
  • component: “proxy” - In the input plugins section: This value identifies application components.

For all other parameters please see this doc for more properties that can be configured in the Telegraf agent globally.

Once you have finalized your telegraf.conf file, you can start or reload the telegraf service using instructions from the doc.

At this point, HAProxy metrics should start flowing into Sumo Logic.

Step 2 Configure Logs Collection

This section provides instructions for configuring log collection for HAProxy running on a non-kubernetes environment for the Sumo Logic App for HAProxy. 

By default, HAProxy logs are forwarded to Syslog. Configuration in the file /etc/haproxy/haproxy.cfg can be modified to send logs to files.

Sumo Logic supports collecting logs both via Syslog and a local log file. Utilizing Sumo Logic Cloud Syslog will require TCP TLS Port 6514 to be open in your network. Local log files can be collected via Installed collectors which will require you to allow outbound traffic to Sumo Logic endpoints for collection to work. For detailed requirements for Installed collectors, see this page.

Based on your infrastructure and networking setup choose one of these methods to collect HAProxy logs and follow the instructions below to set up log collection:

  1. Configure logging in HAProxy
  2. Configure local log file or syslog collection
  3. Configure a Collector
  4. Configure a Source
  • emerg - Errors such as running out of operating system file descriptors.
  • alert - Some rare cases where something unexpected has happened, such as being unable to cache a response
  • info - TCP connection and http request details and errors
  • err - Errors such as being unable to parse a map file, being unable to parse the HAProxy configuration file, and when an operation on a stick table fails
  • warning - Certain important, but non-critical, errors such as failing to set a request header or failing to connect to a DNS nameserver
  • notice - Changes to a server’s state, such as being UP or DOWN or when a server is disabled. Other events at startup, such as starting proxies and loading modules are also included. Health check logging, if enabled, also uses this level)
  • debug (a lot of information, useful for development/testing)

All logging settings are located in Haproxy.conf

The log global directive basically says, use the log line that was set in the global section. Putting a log global directive into the defaults section is equivalent to putting it into all of the subsequent proxy sections.

Configuring HAProxy logs to go to log files

Follow the steps below to enable HAProxy logs to go to log files :

  1. You can enable HAProxy logs to syslog by adding the following line in the global section of /etc/haproxy/haproxy.cfg file. This means that HAProxy will send its messages to rsyslog on 127.0.0.1.

global
   log 127.0.0.1  local2
defaults
   log global
  1. By default, rsyslog doesn’t listen to any address. Uncomment or add following lines in /etc/rsyslog.conf. This will make rsyslog listen on UDP port 514 for all IP addresses.

$ModLoad imudp
$UDPServerRun 514
  1. Now create a /etc/rsyslog.d/haproxy.conf file containing below lines.

local2.*    /var/log/haproxy.log
local2.=info     /var/log/haproxy-info.log
local2.=notice     /var/log/haproxy-notice.log
  1. Restart HAProxy and rsyslog server to enforce configuration changes.

  2. Verify that the files are created in /var/log location.

  1. Add a  Syslog source in the installed collector configured in the previous step.

  2. Configure the Syslog Source fields as follows:

    • Name. (Required)

    • Description. (Optional)

    • Protocol: UDP

    • Port: 514 (as entered while configuring logging in Step b.)

    • Source Category. Enter any string to tag the output collected from this Source, such as Haproxy/Logs. (The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.)

    • Fields. Set the following fields:
      • component = proxy
      • proxy_system = haproxy
      • proxy_cluster = <Your_Haproxy_Cluster_Name>
      • environment = <Environment_Name>, such as Dev, QA or Prod.

  1. Configure the Advanced section:

    • Enable Timestamp Parsing. Select Extract timestamp information from log file entries.
    • Time Zone. Choose the option, Ignore time zone from log file and instead use, and then select your HAProxy Server’s time zone.
    • Timestamp Format. The timestamp format is automatically detected.
    • Encoding. Select UTF-8 (Default).
  2. Click Save.
  1. Add a Local File Source.

  2. Configure the Local File Source fields as follows:
    • Name. (Required)
    • Description. (Optional)
    • File Path (Required). Enter the path to your error.log or access.log. The files are typically located in /var/log/haproxy*.log. If you are using a customized path, check the haproxy.conf file for this information.
    • Source Host. Sumo Logic uses the hostname assigned by the OS unless you enter a different host name.
    • Source Category. Enter any string to tag the output collected from this Source, such as Haproxy/Logs. (The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.)
    • Fields. Set the following fields:
      • component = proxy
      • proxy_system = haproxy
      • proxy_cluster = <Your_Haproxy_Cluster_Name>
      • environment = <Environment_Name>, such as Dev, QA or Prod.

  1. Configure the Advanced section:

    • Enable Timestamp Parsing. Select Extract timestamp information from log file entries.
    • Time Zone. Choose the option, Ignore time zone from log file and instead use, and then select your HAProxy Server’s time zone.
    • Timestamp Format. The timestamp format is automatically detected.
    • Encoding. Select UTF-8 (Default).
    • Enable Multiline Processing. Detect messages spanning multiple lines
      • Infer Boundaries - Detect message boundaries automatically

4. Click Save.