Collect Logs and Metrics for Nginx Plus
This page provides instructions for configuring log and metric collection for the Sumo Logic App for Nginx Plus.
Collection Process Overview
Sumo Logic supports a collection of logs and metrics data from Nginx Plus in both Kubernetes and non-Kubernetes environments.
Please click on the appropriate links below based on the environment where your Nginx Plus servers are hosted.
Sample Log Message
Non Kubernetes environment
Access Log Example
50.1.1.1 - example [23/Sep/2016:19:00:00 +0000] "POST /api/is_individual HTTP/1.1" 200 58 "-" "python-requests/2.7.0 CPython/2.7.6 Linux/3.13.0-36-generic"
Error Log Example
2016/09/23 19:00:00 [error] 1600#1600: *61413 open() "/srv/core/client/dist/client/favicon.ico" failed (2: No such file or directory), client: 101.1.1.1, server: _, request: "GET /favicon.ico HTTP/1.1", host: "example.com", referrer: "https://abc.example.com/"
Kubernetes environment
Access Log Example
{"timestamp":1620821977736,"log":"10.244.0.132 - - [12/May/2021:12:19:28 +0000] \"GET //demo-index.html HTTP/1.1\" 200 8777 \"-\" \"curl/7.68.0\"","stream":"stdout","time":"2021-05-12T12:19:28.975861476Z"}
Error Log Example
{"timestamp":1620821977737,"log":"2021/05/12 12:19:36 [error] 7#7: *8192 upstream timed out (110: Connection timed out) while connecting to upstream, health check \"\" of peer 44.240.53.50:12345 in upstream \"stream_backend2\"","stream":"stderr","time":"2021-05-12T12:19:36.344706832Z"}
Create Field Extraction Rules
Field Extraction Rules (FERs) tell Sumo Logic which fields to parse out automatically. For instructions, see Create a Field Extraction Rule.
Nginx assumes the NCSA extended/combined log file format for Access logs and the default Nginx error log file format for error logs.
Both the parse expressions can be used for logs collected from Nginx Plus Server running on Local or container-based systems.
FER for Access Logs
Use the following Parse Expression:
| json field=_raw "log" as nginx_log_message nodrop | if (isEmpty(nginx_log_message), _raw, nginx_log_message) as nginx_log_message | parse regex field=nginx_log_message "(?<Client_Ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})" | parse regex field=nginx_log_message "(?<Method>[A-Z]+)\s(?<URL>\S+)\sHTTP/[\d\.]+\ "\s(?<Status_Code>\d+)\s(?<Size>[\d-]+)\s\"(?<Referrer>.*?)\"\s\"(?<User_Agent>.+?)\".*"
FER for Error Logs
Use the following Parse Expression:
| json field=_raw "log" as nginx_log_message nodrop | if (isEmpty(nginx_log_message), _raw, nginx_log_message) as nginx_log_message | parse regex field=nginx_log_message "\s\[(?<Log_Level>\S+)\]\s\d+#\d+:\s(?:\*\d+\s|)(?<Message>[A-Za-z][^,]+)(?:,|$)" | parse field=nginx_log_message "client: *, server: *, request: \"* * HTTP/1.1\", host: \"*\"" as Client_Ip, Server, Method, URL, Host nodrop
Query Samples
This sample Query is from the Responses Over Time panel of the Nginx Plus - Overview dashboard.
_sourcecategory=Labs/Nginx/Logs | json auto maxdepth 1 nodrop | if (isEmpty(log), _raw, log) as nginx_log_message | parse regex field=nginx_log_message "(?<Client_Ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})" | parse regex field=nginx_log_message "(?<Method>[A-Z]+)\s(?<URL>\S+)\sHTTP/[\d\.]+\"\s(?<Status_Code>\d+)\s(?<Size>[\d-]+)\s\"(?<Referrer>.*?)\"\s\"(?<User_Agent>.+?)\".*" | where _sourceHost matches "{{Server}}" and Client_Ip matches "{{Client_Ip}}" and Method matches "{{Method}}" and URL matches "{{URL}}" and Status_Code matches "{{Status_Code}}" | if(Status_Code matches "2*", 1, 0) as Successes | if(Status_Code matches "3*", 1, 0) as Redirects | if(status_code matches "4*", 1, 0) as Client_Errors | if(Status_Code matches "5*", 1, 0) as Server_Errors | timeslice by 5m | sum(Successes) as Successes, sum(Client_Errors) as Client_Errors, sum(Redirects) as Redirects, sum(Server_Errors) as Server_Errors by _timeslice | sort by _timeslice asc