Skip to main content
Sumo Logic

Collect Nginx Logs and Metrics for Non-Kubernetes environments

Sumo Logic uses the Telegraf operator for Nginx metric collection and the Installed Collector for collecting Nginx logs. The diagram below illustrates the components of the  Nginx collection in a non-Kubernetes environment. Telegraf uses the Nginx input plugin to obtain Nginx metrics and the Sumo Logic output plugin to send the metrics to Sumo Logic. Logs from Nginx are collected by a Local File Source.

The process to set up collection for Nginx data is done through the following steps:

  1. Configure Logs Collection
    1. Configure logging in Nginx
    2. Configure Sumo Logic Installed Collector
    3. Configure a local file source
    4. Save
  2. Configure Metrics Collection
    1. Configure a Hosted Collector
    2. Configure an HTTP Logs and Metrics Source
    3. Install Telegraf
    4. Configure and start Telegraf

Configure Logs Collection

Nginx app supports the default access logs and error logs format.

  1. Configure logging in Nginx.
    Before you can configure Sumo Logic to ingest logs, you must configure the logging of errors and processed requests in NGINX Open Source and NGINX Plus. For instructions, refer to the following documentation
  2. Configure an Installed Collector. If you have not already done so, install and configure an installed collector for Windows by following the documentation.
  3. Configure a Collector
    Use one of the following Sumo Logic Collector options:
    1. To collect logs directly from the Nginx machine, configure an Installed Collector.
    2. If you are using a service like Fluentd, or you would like to upload your logs manually, Create a Hosted Collector.
  4. Configure a local file source

For an Installed Collector

To collect logs directly from your Nginx machine, use an Installed Collector and a Local File Source.  

  1. Add a Local File Source.
  2. Configure the Local File Source fields as follows:
    • Name. (Required)
    • Description. (Optional)
    • File Path (Required). Enter the path to your error.log or access.log. The files are typically located in /var/log/nginx/error.log. If you are using a customized path, check the nginx.conf file for this information. If you are using Passenger, you may have instructed Passenger to log to a specific log using the passenger_log_file option.
    • Source Host. Sumo Logic uses the hostname assigned by the OS unless you enter a different hostname.
    • Source Category. Enter any string to tag the output collected from this Source, such as Nginx/Access or Nginx/Error. (The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.)
    • Fields. Add the following fields, as show in the screenshot below.
      ​​​​
      component = webserver
      webserver_system = nginx
      webserver_farm = <Your_nginx_farm_Name>
      Enter Default if you do not have one.
      environment = <Your_Environment_Name> (for example, Dev, QA, or Prod)
  3. Configure the Advanced section:
    • Enable Timestamp Parsing. Select Extract timestamp information from log file entries.
    • Time Zone. Automatically detect.
    • Timestamp Format. The timestamp format is automatically detected.
    • Encoding. Select UTF-8 (Default).
    • Enable Multiline Processing
      • Error logs. Select Detect messages spanning multiple lines and Infer Boundaries - Detect message boundaries automatically.
      • Access logs. These are single-line logs, uncheck Detect messages spanning multiple lines.
  4. Click Save.

For a Hosted Collector

If you are using a service like Fluentd, or you would like to upload your logs manually, use a Hosted Collector and an HTTP Source.

  1. Add an HTTP Source.
  2. Configure the HTTP Source fields as follows:
    • Name. (Required)
    • Description. (Optional)
    • Source Host. Sumo Logic uses the hostname assigned by the OS unless you enter a different hostname.
    • Source Category. Enter any string to tag the output collected from this Source, such as Nginx/Access or Nginx/Error. (The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.)
  3. Configure the Advanced section:
    • Enable Timestamp Parsing. Select Extract timestamp information from log file entries.
    • Time Zone. For Access logs, use the time zone from the log file. For Error logs, make sure to select the correct time zone.
    • Timestamp Format. The timestamp format is automatically detected.
    • Enable Multiline Processing
      • Error logs: Select Detect messages spanning multiple lines and Infer Boundaries - Detect message boundaries automatically.
      • Access logs: These are single-line logs, uncheck Detect messages spanning multiple lines.
  4. Click Save.
  5. When the URL associated with the HTTP Source is displayed, copy the URL so you can add it to the service you are using, such as Fluentd.

Configure Metrics Collection

Setup a Sumo Logic HTTP Source
  1. Configure a Hosted Collector for Metrics.
    To create a new Sumo Logic hosted collector, perform the steps in the Create a Hosted Collector documentation.
  2. Configure an HTTP Logs & Metrics source:
    1. On the created Hosted Collector on the Collection Management screen, select Add Source.
    2. Select HTTP Logs & Metrics.
      1. Name. (Required). Enter a name for the source.
      2. Description. (Optional).
    3. Source Category (Recommended). Be sure to follow the Best Practices for Source Categories. A recommended Source Category may be Prod/Webserver/Nginx/Metrics.
  3. Select Save.
  4. Take note of the URL provided once you click Save. You can retrieve it again by selecting the Show URL next to the source on the Collection Management screen.
Setup Telegraf
  1. Install Telegraf if you haven’t already. Use the following steps to install Telegraf.
  2. Configure and start Telegraf.
  3. As part of collecting metrics data from Telegraf, we will use the nginx input plugin to get data from Telegraf and the Sumo Logic output plugin to send data to Sumo Logic.
[[inputs.nginx]]
  urls = ["http://IP_TO_BE_CHANGED/nginx_status"]
  response_timeout = "5s"

  [inputs.nginx.tags]
  environment="env_TO_BE_CHANGED"
  component="webserver"
  webserver_system="nginx"
  webserver_farm="<nginx_TO_BE_CHANGED>"

[[outputs.sumologic]]
  url = "<URL_from_HTTP_Logs_and_Metrics_Source>"
  data_format = "prometheus"

Enter values for fields annotated with <VALUE_TO_BE_CHANGED> to the appropriate values. Do not include the brackets (<>) in your final configuration

  • Input plugins section, which is [[inputs.nginx]]:
    • urls - An array of Nginx stub_status URI to gather stats. For more information on additional parameters to configure the Nginx input plugin for Telegraf see this doc.
  • In the tags section, which is [inputs.nginx.tags]:
    • environment - This is the deployment environment where the Nginx farm identified by the value of servers resides. For example; dev, prod, or QA. While this value is optional we highly recommend setting it. 
    • webserver_farm - Enter a name to identify this Nginx farm. This farm name will be shown in our dashboards. 
  • In the output plugins section, which is [[outputs.sumologic]]
    • URL - This is the HTTP source URL created previously. See this doc for more information on additional parameters for configuring the Sumo Logic Telegraf output plugin.

Here’s an explanation for additional values set by this Telegraf configuration.

  • data_format: “prometheus” - In the output [[outputs.sumologic]] plugins section. Metrics are sent in the Prometheus format to Sumo Logic.
  • Component - “webserver” - In the input [[inputs.nginx]] plugins section. This value is used by Sumo Logic apps to identify application components.
  • webserver_system - “nginx” - In the input plugins sections. This value identifies the webserver system.

See this doc for all other parameters that can be configured in the Telegraf agent globally.

At this point, Telegraf should start collecting the Nginx metrics and forward them to the Sumo Logic HTTP Source.