Skip to main content
Sumo Logic

Collect Apache Logs and Metrics for Non-Kubernetes environments

We use the Telegraf Operator for Apache metrics collection and the Sumo Logic Installed Collector for collecting Apache logs. The diagram below illustrates the components of the Apache collection in a non-Kubernetes environment for each web server. Telegraf runs on the same host as Apache, and uses the Apache input plugin to obtain Apache metrics, and the Sumo Logic output plugin to send the metrics to Sumo Logic. Apache logs are sent to a Sumo Logic Local File source of an installed collector.. 

This section provides instructions for configuring metrics collection for the Sumo Logic App for Apache. Follow the instructions to set up metrics collection for each server belonging to a Apache server farm:

  1. Configure Metrics Collection

    1. Configure Metrics in Apache

    2. Configure a Hosted Collector

    3. Configure a HTTP Logs and Metrics Source

    4. Install Telegraf

    5. Configure and start Telegraf

  2. Configure Logs Collection

    1. Configure logging in Apache

    2. Configure Sumo Logic Installed Collector

    3. Configure a Local File Source

Step 1 Configure Collection of Metrics from a Apache Server

  1. Configure Metrics in Apache

Before you can configure Sumo Logic to ingest metrics, you must turn on server-status for Apache. For this edit the Apache conf file (httpd.conf)

  • Uncomment following line if not already done in the httpd.conf
    • LoadModule status_module libexec/apache2/mod_status.so
  • Add following lines in the httpd.conf after that
  <IfModule status_module>
    ExtendedStatus On
    <Location /server-status>
      Sethandler server-status
      order deny,allow
      allow from localhost
    </Location>
  </IfModule>
  • You may need to update ServerName in the httpd.conf file
    • Example: ServerName localhost:80
  • Save httpd.conf file
  • Verify configuration is working as expected by running the following command
    • apachectl configtest

Syntax OK

  1. Configure a Hosted Collector

To create a new Sumo Logic hosted collector, perform the steps in the Configure a Hosted Collector section of the Sumo Logic documentation.

  1. Configure an HTTP Logs and Metrics Source

Create a new HTTP Logs and Metrics Source in the hosted collector created above by following these instructions. Make a note of the HTTP Source URL.

  1. Install Telegraf

Follow the steps in this document to install Telegraf.

  1. Configure and start Telegraf

As part of collecting metrics data from Telegraf, we will use the Apache input plugin to get data from Telegraf and the Sumo Logic output plugin to send data to Sumo Logic. 

Create or modify the telegraf.conf file and copy and paste the text below in the relevant sections:

[[inputs.apache]]
  urls = ["http://localhost/server-status?auto"]
  response_timeout = "5s"
  [inputs.apache.tags]
    environment = "prod"
    component = "webserver"
    webserver_system = "apache"
    webserver_farm = "your_apache_webserver_farmname"

[[outputs.sumologic]]
  url = "<URL Created in Step 3>"
  data_format = "prometheus"

[agent]
  interval = "60s"
  flush_interval = "60s"

Please enter values for the following parameters (marked in bold above):

  • In the input plugins section i.e. : 

    • urls - The URL to the Apache server. Please see this doc for more information on additional parameters for configuring the Apache input plugin for Telegraf.

    • Configure metrics to collect by uncommenting the following lines. Please see  this document for more information

      • response_timeout = "5s"

    • In the tags section i.e.  [inputs.apache.tags]

      • webserver_farm - Enter a name to uniquely identify this Apache web server farm. This web server farm name will be shown in the Sumo Logic dashboards. 

      • environment - This is the deployment environment where the Apache web server farm identified by the value of urls resides. For example: dev, prod or qa. While this value is optional we highly recommend setting it. 

  • In the output plugins section i.e. : 

    • url - This is the HTTP source URL created in step 3. Please see this doc for more information on additional parameters for configuring the Sumo Logic Telegraf output plugin.

  • In the agent section i.e [agent]

    • Set interval and flush_interval to “60s” to collect metric every 60 seconds.

Here’s an explanation for additional values set by this Telegraf configuration that we request you to please not modify these values as they will cause the Sumo Logic apps to not function correctly.

  • data_format = “prometheus”, In the output plugins section i.e.   Metrics are sent in the Prometheus format to Sumo Logic

  • component = “webserver” - In the input plugins section i.e. - This value is used by Sumo Logic apps to identify application components.

  • webserver_system = “apache” - In the input plugins section i.e. -  This value identifies the webserver system.

For all other parameters please see this doc for more properties that can be configured in the Telegraf agent globally.

Once you have finalized your telegraf.conf file, you can start or reload the telegraf service via the instructions described in their documentation.

At this point, Apache metrics should start flowing into Sumo Logic.

Step 2 Configure Collection of Logs from a Apache server

This section provides instructions for configuring collection of logs from Apache running on a non-Kubernetes environment. 

Apache logs (access logs and error logs) are stored in log files. 

Sumo Logic supports collecting logs via a local log file. Local log files can be collected via Sumo Logic Installed collectors, which requires you to allow outbound traffic to Sumo Logic endpoints for collection to work.

Follow the instructions below to set up log collection:

  1. Configure Apache to log to a local file(s)
  2. Configure an Installed Collector
  3. Configure a Local File source for apache access logs
  4. Configure a Local File source for apache error logs

 

  1. Configure Apache to log to a local file(s)

Apache logs written to a log file can be collected via the Local File Source of a Sumo Logic Installed collector. Before you can configure Sumo Logic to ingest logs, you must configure the logging of access logs and error logs via the instructions described in their documentation.

To configure the Apache log file(s), locate your local httpd.conf configuration file in the Apache directory. After determining the location of the conf file, modify the httpd.conf configuration file logging parameters if required.

For access logs, the following directive is to be noted

  • CustomLog: access log file path and format (standard common and combined)

For error logs, following directives are to be noted

  • ErrorLog: error log file path

  • LogLevel: to control the number of messages logged to the error_log

  1. Configure an Installed Collector

To add an Installed collector, perform the steps as defined on the page Configure an Installed Collector.

  1. Configure a Local File Source for Apache access logs

To add a Local File Source for Apache access log do the following

  1. Add a Local File Source in the installed collector configured in the previous step.

  2. Configure the Local File Source fields as follows:

  • Name. (Required)

  • Description. (Optional)

  • File Path (Required). Enter the path to your apache access logs. The files are typically located in /var/log/apache2/access_log. If you are using a customized path, check the httpd.conf file for this information. 

  • Source Host. Sumo Logic uses the hostname assigned by the OS unless you enter a different host name

  • Source Category. Enter any string to tag the output collected from this Source, such as Prod/Apache/Access. (The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.)

  • Fields. Set the following fields. For more information on fields please see this document:

    • component = webserver

    • webserver_system = apache

    • webserver_farm = <your_apache_webserver_farmname>

    • environment = <Environment_Name>, such as dev, qa or prod.

  • Configure the Advanced Options for Logs section:

  • Enable Timestamp Parsing. Select Extract timestamp information from log file entries.

  • Time Zone. Select Use time zone form log file, if none is detected use “Use Collector Default”

  • Timestamp Format. Select Automatically detect the format.

  • Encoding. Select UTF-8 (Default).

  • Apache Access logs are single-line logs, uncheck Detect messages spanning multiple lines.

  1. Click Save.

At this point, Apache access logs should start flowing into Sumo Logic.

  1. Configure a Local File Source for Apache error logs

To add a Local File Source for Apache error log do the following

  1. Add a Local File Source in the installed collector configured in the previous step.

  2. Configure the Local File Source fields as follows:

  • Name. (Required)

  • Description. (Optional)

  • File Path (Required). Enter the path to your error_log. The files are typically located in /var/log/apache2/error_log. If you are using a customized path, check the httpd.conf file for this information. 

  • Source Host. Sumo Logic uses the hostname assigned by the OS unless you enter a different host name

  • Source Category. Enter any string to tag the output collected from this Source, such as Prod/Apache/Error. (The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.)

  • Fields. Set the following fields. For more information on fields please see this document:

    • component = webserver

    • webserver_system = apache

    • webserver_farm = <your_apache_webserver_farmname>

    • environment = <Environment_Name>, such as dev, qa or prod.

  • Configure the Advanced Options for Logs section:

  • Enable Timestamp Parsing. Select Extract timestamp information from log file entries.

  • Time Zone. Select Use time zone form log file, if none is detected use “Use Collector Default”

  • Timestamp Format. Select Automatically detect the format.

  • Encoding. Select UTF-8 (Default).

  • Apache Error logs are multiline-line logs, Select Detect messages spanning multiple lines and Boundary Regex - Expression to match message boundary.

    • If error messages starts like [Mon May 17 14:12:14.462704 2021] then use boundary regex as below 

      ^\[\S{3}\s\S{3}\s\d{1,2}\s[^\]]+\].*

  1. Click Save.

At this point, Apache Error logs should start flowing into Sumo Logic.