Skip to main content
Sumo Logic

Collect Logs and Metrics for Nginx ULM

Learn how to collect logs and metrics for  Sumo Logic  App for Nginx ULM.

This page provides instructions for configuring log and metric collection for the Sumo Logic App for Nginx ULM.

Collection Process Overview

Configuring log and metric collection for the Nginx ULM App includes the following tasks:

Step 1: Collect Logs for Nginx

Non-Kubernetes

This section provides instructions for configuring log collection for the Sumo Logic App for Nginx. Follow the below instructions to set up the Log collection.

  1. Configure logging in Nginx 
  2. Configure a Collector
  3. Configure a Source
1. Configure logging in Nginx

Before you can configure Sumo Logic to ingest logs, you must configure the logging of errors and processed requests in NGINX Open Source and NGINX Plus. For instructions, refer to the following documentation:

https://www.nginx.com/resources/admin-guide/logging-and-monitoring/

2. Configure a Collector

Use one of the following Sumo Logic Collector options:

  1. To collect logs directly from the Nginx machine, configure an Installed Collector.
  2. If you are using a service like Fluentd, or you would like to upload your logs manually, configure a Hosted Collector
3. Configure a Source
For an Installed Collector

To collect logs directly from your Nginx machine, use an Installed Collector and a Local File Source. 

  1. Add a Local File Source.
  2. Configure the Local File Source fields as follows:
    • Name. (Required)
    • Description. (Optional)
    • File Path (Required). Enter the path to your error.log or access.log. The files are typically located in /var/log/nginx/error.log. If you are using a customized path, check the nginx.conf file for this information. If you are using Passenger, you may have instructed Passenger to log to a specific log using the passenger_log_file option.
    • Source Host. Sumo Logic uses the hostname assigned by the OS unless you enter a different hostname.
    • Source Category. Enter any string to tag the output collected from this Source, such as Nginx/Access or Nginx/Error. (The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.)
  3. Configure the Advanced section:
    • Enable Timestamp Parsing. Select Extract timestamp information from log file entries.
    • Time Zone. Automatically detect.
    • Timestamp Format. The timestamp format is automatically detected.
    • Encoding. Select UTF-8 (Default).
    • Enable Multiline Processing. 
      • Error logs. Select Detect messages spanning multiple lines and Infer Boundaries - Detect message boundaries automatically.
      • Access logs. These are single-line logs, uncheck Detect messages spanning multiple lines.
  4. Click Save.
For a Hosted Collector

If you are using a service like Fluentd, or you would like to upload your logs manually, use a Hosted Collector and an HTTP Source.

  1. Add an HTTP Source.
  2. Configure the HTTP Source fields as follows:
    • Name. (Required)
    • Description. (Optional)
    • Source Host. Sumo Logic uses the hostname assigned by the OS unless you enter a different hostname.
    • Source Category. Enter any string to tag the output collected from this Source, such as Nginx/Access or Nginx/Error. (The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.)
  3. Configure the Advanced section:
    • Enable Timestamp Parsing. Select Extract timestamp information from log file entries.
    • Time Zone. For Access logs, use the time zone from the log file. For Error logs, make sure to select the correct time zone.
    • Timestamp Format. The timestamp format is automatically detected.
    • Enable Multiline Processing. 
      • Error logs: Select Detect messages spanning multiple lines and Infer Boundaries - Detect message boundaries automatically.
      • Access logs: These are single-line logs, uncheck Detect messages spanning multiple lines.
  4. Click Save.
  5. When the URL associated with the HTTP Source is displayed, copy the URL so you can add it to the service you are using, such as Fluentd.

Kubernetes

1. Configure logging in Nginx ULM

Before you can configure Sumo Logic to ingest logs, you must configure the logging of errors and processed requests in NGINX Open Source and NGINX Plus. For instructions, refer to the following documentation:

https://www.nginx.com/resources/admin-guide/logging-and-monitoring/

2. Use the Sumologic-Kubernetes-Collection, to send the logs to Sumologic. For more information, visit.

Step 2: Collect Metrics for Nginx ULM

Non-Kubernetes

This section provides instructions for configuring metrics collection for the Sumo Logic App for Nginx. Follow the below instructions to set up the metric collection.

  1. Configure Metrics in Nginx
  2. Configure a Hosted Collector
  3. Configure a Http Logs and Metrics Source
  4. Install Telegraf
  5. Configure Telegraf and Forward Metrics to Sumo Logic
1. Configure Metrics in Nginx 

Before you can configure Sumo Logic to ingest metrics, you must enable the stub status module to expose metrics in NGINX Open Source. 

2. Configure a Hosted Collector

To create a new Sumo Logic hosted collector, perform the steps in the Configure a Hosted Collector section of the Sumo Logic documentation.

3. Configure an Http Logs and Metrics Source

Create a new HTTP Logs and Metrics Source in the hosted collector created above by following these instructions. 

Make a note of HTTP Source URL.

4. Install Telegraf.

Use the following steps to install Telegraf.

5. Configure and start Telegraf.

Create a file called telegraf.conf and add the appropriate configuration. The following is a basic example:

[agent]
  interval = "60s"
[[inputs.nginx]]
  urls = ["http://localhost:8080/nginx_status"]
  namepass = ["nginx"]
  fieldpass = ["accepts", "active", "handled", "reading", "requests", "waiting", "writing"]
[[outputs.sumologic]]
  url = "<URL Created in Step 3>"
  data_format = "prometheus"
  • interval - This is the frequency to send data to Sumo Logic, in this example, we will send the metrics every 60 seconds. Please refer to this doc for more properties that can be configured in the Telegraf agent globally.
  • urls - The url to the Nginx server with the Stub Status enabled. This can be a comma-separated list to connect to multiple Nginx servers. Please refer to this doc for more information on configuring the Nginx input plugin for Telegraf.
  • url - This is the HTTP source URL created in step 3. Please refer to this doc for more information on configuring the Sumo Logic Telegraf output plugin.
  • data_format = The format to use when sending data to Sumo Logic. Please refer to this doc for more information on configuring the Sumo Logic Telegraf output plugin.

Once you have finalized your telegraf.conf file, you can run the following command to start telegraf.

telegraf --config /path/to/telegraf.conf

Kubernetes

The following steps assume you are collecting Nginx metrics from a Kubernetes environment. In a Kubernetes environment, we use the Telegraf Operator, which is packaged with our Kubernetes collection.  You can learn more about this here.

  1. Before you can configure Sumo Logic to ingest metrics, you must enable the stub status module to expose metrics in NGINX Open Source. 
  2. Set up Kubernetes Collection with the Telegraf Operator.
  3. On your Nginx Pods, add the following annotations to configure Telegraf.
annotations:
        telegraf.influxdata.com/inputs: |+
          [[inputs.nginx]]
            urls = ["http://localhost:8080/nginx_status"]
        telegraf.influxdata.com/class: sumologic-prometheus
        prometheus.io/scrape: "true"
        prometheus.io/port: "9273"
  • telegraf.influxdata.com/inputs - This contains the required configuration for the Telegraf Nginx Input plugin. Please refer to this doc for more information on configuring the Nginx input plugin for Telegraf. Note since telegraf will be run as a sidecar the host should always be localhost
  • telegraf.influxdata.com/class: sumologic-prometheus - This instructs the Telegraf operator what output to use. This should not be changed.
  • prometheus.io/scrape: "true" - This ensures our Prometheus will scrape the metrics.
  • prometheus.io/port: "9273" - This tells Prometheus what ports to scrape on. This should not be changed.

Sample Log Message

Access Log Example

50.1.1.1 - example [23/Sep/2016:19:00:00 +0000] "POST /api/is_individual HTTP/1.1" 200 58 "-" 
"python-requests/2.7.0 CPython/2.7.6 Linux/3.13.0-36-generic"

Error Log Example

2016/09/23 19:00:00 [error] 1600#1600: *61413 open() "/srv/core/client/dist/client/favicon.ico" 
failed (2: No such file or directory), client: 101.1.1.1, server: _, request: "GET /favicon.ico 
HTTP/1.1", host: "example.com", referrer: "https://abc.example.com/"

Create Field Extraction Rules

Field Extraction Rules (FERs) tell Sumo Logic which fields to parse out automatically. For instructions, see Create a Field Extraction Rule

Nginx assumes the NCSA extended/combined log file format for Access logs and the default Nginx error log file format for error logs.

Both the parse expressions can be used for logs collected from Nginx Server running on Local or container-based systems.

FER for Access Logs

Use the following Parse Expression:

| json field=_raw "log" as nginx_log_message nodrop
| if (isEmpty(nginx_log_message), _raw, nginx_log_message) as nginx_log_message
| parse regex field=nginx_log_message "(?<Client_Ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
| parse regex field=nginx_log_message "(?<Method>[A-Z]+)\s(?<URL>\S+)\sHTTP/[\d\.]+\
"\s(?<Status_Code>\d+)\s(?<Size>[\d-]+)\s\"(?<Referrer>.*?)\"\s\"(?<User_Agent>.+?)\".*"

FER for Error Logs

Use the following Parse Expression:

| json field=_raw "log" as nginx_log_message nodrop
| if (isEmpty(nginx_log_message), _raw, nginx_log_message) as nginx_log_message
| parse regex field=nginx_log_message "\s\[(?<Log_Level>\S+)\]\s\d+#\d+:\s(?:\*\d+\s|)(?<Message>[A-Za-z][^,]+)(?:,|$)"
| parse field=nginx_log_message "client: *, server: *, request: \"* * HTTP/1.1\", host: 
\"*\"" as Client_Ip, Server, Method, URL, Host nodrop

Query Samples

This sample Query is from the Requests by Clients panel of the Nginx ULM - Overview dashboard.

_sourceCategory = Labs/Nginx/Logs
| json field=_raw "log" as nginx_log_message nodrop
| if (isEmpty(nginx_log_message), _raw, nginx_log_message) as nginx_log_message
| parse regex field=nginx_log_message "(?<Client_Ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
| parse regex field=nginx_log_message "(?<Method>[A-Z]+)\s(?<URL>\S+)\sHTTP/[\d\.]+\"\s(?<Status_Code>\d+)\s(?<Size>[\d-]+)\s\"(?<Referrer>.*?)\"\s\"(?<User_Agent>.+?)\".*"
| where _sourceHost matches "{{Server}}" and Client_Ip matches "{{Client_Ip}}" and Method matches "{{Method}}" and URL matches "{{URL}}" and Status_Code matches "{{Status_Code}}"
| count as count by Client_Ip
| sort count