Skip to main content
Sumo Logic

Collect Logs and Metrics for the IIS App

This page provides instructions for configuring log and metric collection for the Sumo Logic App for IIS.

This page provides instructions for configuring log and metric collection for the Sumo Logic App for IIS.

Collection Process Overview

Configuring log and metric collection for the IIS App includes the following tasks:

  • Configure Fields in Sumo Logic.
  • Configure Collection for IIS Server
    • Configure Logs Collection
    • Configure Metrics Collection

Configure Fields in Sumo Logic

Create the following Fields in Sumo Logic prior to configuring the collection. This ensures that your logs and metrics are tagged with relevant metadata, which is required by the app dashboards. For information on setting up fields, see the Fields help page.

  • component
  • environment
  • webserver_system
  • webserver_farm
  • pod

Configure Collection for IIS

Sumo Logic supports the collection of logs and metrics data from IIS server in standalone environments. 

  • Configure Log Collection.
    • Enable Logging on IIS Server Side
    • Set up Collector and Sources on Sumo Logic Side
      • Set up local file source for IIS Access Logs
      • Set up local file source for IIS Error Logs
      • Set up Source for IIS Performance (Perfmon) Logs
  • Configure Metrics Collection.
    • Set up HTTP Metrics Source
    • Configure Telegraf (telegraf.conf), and start it.

Collect Internet Information Services (IIS) Logs and Metrics for Standalone environments

Sumo Logic uses the Telegraf operator for IIS metric collection and the Installed Collector for collecting IIS logs. The diagram below illustrates the components of the IIS collection in a standalone environment. Telegraf uses the Windows Performance Counters Input Plugin to obtain IIS metrics and the Sumo Logic output plugin to send the metrics to Sumo Logic. Logs from IIS Server are collected by a Local File Source.

 

The process to set up collection for IIS data is done through the following steps:

  1. Configure Logs Collection
    1. Log Types
    2. Make sure logging is turned on in IIS Server
    3. Configure an Installed Collector
    4. Configure Sources
  2. Configure Metrics Collection
    1. Configure a Hosted Collector
    2. Configure an HTTP Logs and Metrics Source
    3. Install Telegraf
    4. Configure and start Telegraf

Configure Logs Collection

This section provides instructions for configuring log collection for IIS running on a standalone environment for the Sumo Logic App for IIS.

  1. Log Types

This section covers the following default log formats for IIS 10 and IIS 8.5:

  • IIS Access Logs (W3C format)
  • HTTP Error Logs
  • Performance Logs

Default log formats are used by IIS App. IIS allows you to choose which fields to log in IIS access logs. To understand the various fields and their significance see this link.

IIS Log files are generated as local files. For a standard Windows Server, the default log location is as follows: %SystemDrive%\inetpub\logs\LogFiles

For example:

c:\inetpub\logs\LogFiles\

Within the folder, you will find subfolders for each site configured with IIS. The logs are stored in folders that follow a naming pattern like W3SVC1, W3SVC2, W3SVC3, etc. The number at the end of the folder name corresponds to your site ID. For example, W3SVC2 is for site ID 2.

  • IIS Access Logs (W3C default format)
    Sumo Logic expects logs in W3C format with following fields:
    #Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken
    IIS allows you to choose fields to log in IIS access logs. For explanations on the various fields and their significance see this link.

  • HTTP Error Logs
    Sumo Logic expects Error logs in following format :
    #Fields: date time c-ip c-port s-ip s-port protocol_version verb cookedurl_query protocol_status siteId Reason_Phrase Queue_Name 
    For information on how to configure HTTP Error Logs, and for explanations on the various HTTP Error Log fields and their significance see this link.

  • Performance Logs
    These logs are output of Perfmon queries which will be configured at Installed Collector, "Windows Performance" Source.

  1. Make sure logging is turned on in IIS Server.

  • Enable logging on your IIS Server
    Perform the following task, if logging on your IIS Server is not already enabled.
    To enable logging on your IIS Server, do the following:

  1. Open IIS Manager.

  2. Select the site or server in the Connections pane, and then double-click Logging.

  1. In the Format field under Log File, select W3C and then click Select Fields. IIS App works on default fields selection.

  2. Select following fields, if not already selected. Sumo Logic expects these fields in IIS logs for the IIS App to work by default:
    date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken
    For more information about IIS log format and log configuration refer link.

  • Verify that log files are created
    Perform the following task to ensure that log files are being created.

To confirm log files are being created, do the following:

  1. Open a command-line window and change directories to C:\inetpub\Logs\LogFiles. This is the same path you will enter when you configure the Source to collect these files.
  2. Under the \W3SVC1 directory, you should see one or more files with a .log extension. If the file is present, you can collect it.
  • Enable HTTP Error Logs on your Windows Server
    Perform the following task to enable HTTP Error Logs on your Windows Server, that is hosting the IIS Server.

To enable HTTP Error Logs on the Windows Server hosting IIS Server, do the following:

  1. To configure HTTP Error Logging, refer to this document link.
  2. To understand HTTP Error Log format, refer to this document link.
    HTTP Error Log files are generated as local files. The default HTTP Error log file location is: C:\Windows\System32\LogFiles\HTTPERR
  1. Configure an Installed Collector. 

If you have not already done so, install and configure an installed collector for Windows by following the documentation.

  1. Configure Sources.

This section demonstrates how to configure sources for the following log types:

  • IIS Access Logs
  • HTTP Error Logs
  • IIS Performance (Perfmon) Logs
  • Configure Source for IIS Access Logs

To configure a local file source for IIS Access Logs, do the following:

  1. Configure a Local File Source.
  2. Specify Local File Source Fields as follows:
    1. Name: Required (for example, "IIS")
    2. Description. (Optional)
    3. File Path (Required).C:\inetpub\Logs\LogFiles\W3SVC*\*.log
    4. Collection start time. Choose how far back you would like to begin collecting historical logs. For example, choose 7 days ago to being collecting logs with a last modified date within the last seven days.
    5. Source Host. Sumo Logic uses the hostname assigned by the operating system by default, but you can enter a different host name.
    6. Source Category (Required). For example, Webserver/IIS/Access. 
    7. Fields. Set the following fields:
      component = webserver
      webserver_system = iis
      webserver_farm = <Your_IISserver_farm_Name>. Enter Default if you do not have one.
      environment = <Your_Environment_Name> (for example, Dev, QA, or Prod)
  3. Configure the Advanced section:
    1. Timestamp Parsing Settings: Make sure the setting matches the timezone on the log files.
    2. Enable Timetamp Parsing: Select Extract timestamp information from log file entries.
    3. Time Zone: Select the option to Use time zone from log file. If none is present use: and set the timezone to UTC.
    4. Timestamp Format: Select the option to Automatically detect the format.
    5. Encoding. UTF-8 is the default, but you can choose another encoding format from the menu if your IIS logs are encoded differently.
    6. Enable Multiline Processing. Uncheck the box to Detect messages spanning multiple lines. Since IIS Error logs are single line log files, disabling this option will ensure that your messages are collected correctly.
  4. Click Save.
    After a few minutes, your new Source should be propagated down to the Collector and will begin submitting your IIS log files to the Sumo Logic service.
  • Configure Source for HTTP Error Logs

This section demonstrates how to configure a Local File Source for HTTP Error Logs, for use with an Installed Collector

To configure a local file source for HTTP Error Logs, do the following:

  1. Configure a Local File Source.
  2. Specify the Local File Source Fields as follows:
    1. Name: Required (for example, "HTTP Error Logs")
    2. Description. (Optional)
    3. File Path (Required). C:\Windows\System32\LogFiles\HTTPERR\*.*
    4. Collection start time. Choose how far back you would like to begin collecting historical logs. For example, choose 7 days ago to being collecting logs with a last modified date within the last seven days.
    5. Source Host. Sumo Logic uses the hostname assigned by the operating system by default, but you can enter a different host name.
    6. Source Category (Required). For example, Webserver/IIS/Error. 
    7. Fields. Set the following fields:
      component = webserver
      webserver_system = iis
      webserver_farm = <Your_IISserver_farm_Name>. Enter Default if you do not have one.
      environment = <Your_Environment_Name> (for example, Dev, QA, or Prod)
  3. Configure the Advanced section:
    1. Timestamp Parsing Settings: Make sure the setting matches the timezone on the log files.
    2. Enable Timetamp Parsing: Select Extract timestamp information from log file entries.
    3. Time Zone: Select the option to Use time zone from log file. If none is present use: and set the timezone to UTC.
    4. Timestamp Format: Select the option to Automatically detect the format.
    5. Encoding. UTF-8 is the default, but you can choose another encoding format from the menu if your IIS logs are encoded differently.
    6. Enable Multiline Processing. Uncheck the box to Detect messages spanning multiple lines. Since IIS Error logs are single line log files, disabling this option will ensure that your messages are collected correctly.
  4. Click Save.
    After a few minutes, your new Source should be propagated down to the Collector and will begin submitting your IIS HTTP Error log files to the Sumo Logic service.
  • Configure Source for IIS Performance (Perfmon) Logs

This section demonstrates how to configure a Windows Performance Source, for use with an Installed Collector.

Use the appropriate source for your environment:

To configure a Source for IIS Performance Logs, do the following:

  1. Configure a Local Windows Performance Monitor Log Source.
  2. Configure the Local Windows Performance Source Fields as follows:
    • Name: Required (for example, "IIS Performance")
    • Source Category (Required). For example, Webserver/IIS/PerfCounter. 
    • Frequency: Every Minute (you may custom choose frequency)
    • Description. (Optional)
    • Fields. Set the following fields:
      component = webserver
      webserver_system = iis
      webserver_farm = <Your_IISserver_farm_Name>
      . Enter Default if you do not have one.
      environment = <Your_Environment_Name> (for example, Dev, QA, or Prod)
  3. Under Perfmon Queries Click Add Query.
  4. Add the following two queries:
    • Query 1:
      1. For Name, enter WebServices
      2. For Query, enter Select TotalMethodRequestsPerSec, GetRequestsPerSec, PostRequestsPerSec, CurrentConnections, CurrentAnonymousUsers, CurrentNonAnonymousUsers, CGIRequestsPerSec, ISAPIExtensionRequestsPerSec, BytesReceivedPerSec, BytesSentPerSec, FilesReceivedPerSec, FilesSentPerSec, ServiceUptime, BytesTotalPerSec from Win32_PerfFormattedData_W3SVC_WebService
    • Query 2:
      1. For Name, enter HTTPServiceRequestQueues
      2. For Query, enter Select ArrivalRate, CurrentQueueSize, CacheHitRate, RejectionRate, MaxQueueItemAge from Win32_PerfFormattedData_Counters_HTTPServiceRequestQueues
  5. Click Save.

Configure Metrics Collection

Setup a Sumo Logic HTTP Source
  1. Configure a Hosted Collector for Metrics.
    To create a new Sumo Logic hosted collector, perform the steps in the Create a Hosted Collector documentation.

  2. Configure an HTTP Logs & Metrics source:

    1. On the created Hosted Collector on the Collection Management screen, select Add Source.

    2. Select HTTP Logs & Metrics.

      1. Name. (Required). Enter a name for the source.

      2. Description. (Optional).

      3. Source Category (Required). For example,  Prod/Webserver/IIS/Metrics.

    3. Select Save.

    4. Take note of the URL provided once you click Save. You can retrieve it again by selecting the Show URL next to the source on the Collection Management screen.

Setup Telegraf
  1. Install Telegraf if you haven’t already. Use the following steps to install Telegraf.

  2. Configure and start Telegraf.
    As part of collecting metrics data from Telegraf, we will use the Windows Performance Counters Input Plugin to get data from Telegraf and the Sumo Logic output plugin to send data to Sumo Logic.

    Create or modify telegraf.conf and copy and paste the text below:  

[[inputs.win_perf_counters]]
  [[inputs.win_perf_counters.object]]
    # HTTP Service request queues in the Kernel before being handed over to User Mode.
    ObjectName = "HTTP Service Request Queues"
    Instances = ["*"]
    Counters = [
"CurrentQueueSize",
"RejectedRequests"
]
    Measurement = "win_http_queues"
  [[inputs.win_perf_counters.object]]
    # IIS, ASP.NET Applications
    ObjectName = "ASP.NET Applications"
    Counters = [
"Cache Total Entries",
"Cache Total Hit Ratio",
"Cache Total Turnover Rate",
"Output Cache Entries",
"Output Cache Hits",
"Output Cache Hit Ratio",
"Output Cache Turnover Rate",
"Compilations Total",
"Errors Total/Sec",
"Pipeline Instance Count",
"Requests Executing",
"Requests in Application Queue",
"Requests/Sec"]
    Instances = ["*"]
    Measurement = "win_aspnet_app"
  [[inputs.win_perf_counters.object]]
    # IIS, ASP.NET
    ObjectName = "ASP.NET"
    Counters = [
"Application Restarts",
"Applications Running",
"Request Execution Time",
"Request Wait Time",
"Requests Current",
"Requests Queued",
"Requests Rejected",
"State Server Sessions Abandoned",
"State Server Sessions Active",
"State Server Sessions Timed Out",
"State Server Sessions Total",
"Worker Process Restarts",
"Worker Processes Running"
]
    Instances = ["*"]
    Measurement = "win_aspnet"
  [[inputs.win_perf_counters.object]]
    # IIS, Web Service
    ObjectName = "Web Service"
    Counters = [
"Service Uptime",
"Current Connections",
"Bytes Sent/sec",
"Total Bytes Sent",
"Bytes Received/sec",
"Total Bytes Received",
"Bytes Total/sec",
"Total Bytes Transferred",
"Get Requests/sec",
"Total Get Requests",
"Post Requests/sec",
"Total Post Requests",
"Put Requests/sec",
"Total Put Requests",
"Delete Requests/sec",
"Total Delete Requests",
"Head Requests/sec",
"Options Requests/sec",
"Total Head Requests",
"Total Method Requests",
"Total Options Requests",
"Anonymous Users/sec",
"NonAnonymous Users/sec",
"Files Sent/sec",
"Total Files Sent",
"Files Received/sec",
"Total Files Received",
"Files/sec",
"Total Files Transferred",
"Not Found Errors/sec",
"Locked Errors/sec",
"Total Method Requests/sec",
"Total Allowed Async I/O Requests",
"Total Blocked Async I/O Requests",
"Current Blocked Async I/O Requests",
"Current CGI Requests",
"Current ISAPI Extension Requests",
"Current NonAnonymous Users",
"Total CGI Requests",
"Total Connection Attempts (all instances)",
"Total ISAPI Extension Requests",
"Total Locked Errors",
"Total Logon Attempts",
"Total NonAnonymous Users",
"Total Not Found Errors",
"Total Rejected Async I/O Requests",
"Total count of failed CAL requests for authenticated users",
"Total count of failed CAL requests for SSL connections"
]
    Instances = ["*"]
    Measurement = "win_websvc"
  [[inputs.win_perf_counters.object]]
    # Web Service Cache / IIS
    ObjectName = "Web Service Cache"
    Counters = [
"Current Files Cached",
"Active Flushed Entries",
"Total Files Cached",
"Total Flushed Files",
"File Cache Hits",
"File Cache Misses",
"File Cache Hits %",
"File Cache Flushes",
"Current File Cache Memory Usage",
"Maximum File Cache Memory Usage",
"Current URIs Cached",
"Total URIs Cached",
"Total Flushed URIs",
"URI Cache Hits",
"URI Cache Misses",
"URI Cache Hits %",
"URI Cache Flushes",
"Current Metadata Cached",
"Total Metadata Cached",
"Total Flushed Metadata",
"Metadata Cache Hits",
"Metadata Cache Misses",
"Metadata Cache Hits %",
"Metadata Cache Flushes",
"Output Cache Current Flushed Items",
"Output Cache Current Hits %",
"Output Cache Current Items",
"Output Cache Current Memory Usage",
"Output Cache Total Flushed Items"
]
    Instances = ["*"]
    Measurement = "win_websvc_cache"
  [[inputs.win_perf_counters.object]]
    # APP POOL WAS
    ObjectName = "APP_POOL_WAS"
    Counters = [
"Current Application Pool State",
"Current Application Pool Uptime",
"Current Worker Processes",
"Maximum Worker Processes",
"Recent Worker Process Failures",
"Time Since Last Worker Process Failure",
"Total Application Pool Recycles",
"Total Application Pool Uptime",
"Total Worker Process Failures",
"Total Worker Process Ping Failures",
"Total Worker Process Shutdown Failures",
"Total Worker Process Startup Failures",
"Total Worker Processes Created"
]
    Instances = ["*"]
    Measurement = "win_app_pool_was"
  [inputs.win_perf_counters.tags]
    environment="env_TO_BE_CHANGED"
    component="webserver"
    webserver_system="iis"
    webserver_farm="iisserver_TO_BE_CHANGED"


[[outputs.sumologic]]
  url = "<URL_from_HTTP_Logs_and_Metrics_Source>"
  data_format = "prometheus"

Enter values for fields annotated with <VALUE_TO_BE_CHANGED> to the appropriate values. Do not include the brackets (<>) in your final configuration

  • Input plugins section, which is [[inputs.win_perf_counters]]:

Configure the Windows Performance Counters Input Plugin for Telegraf see this doc.

  • In the tags section, which is [inputs.win_perf_counters.tags]:
    • environment - This is the deployment environment where the IIS Server farm identified by the value of servers resides. For example; dev or QA. While this value is optional we highly recommend setting it. 
    • webserver_farm - Enter a name to identify this IIS Server farm This farm name will be shown in our dashboards. Use “default” if none is present.
  • In the output plugins section, which is [[outputs.sumologic]]
    • URL - This is the HTTP source URL created previously. See this doc for more information on additional parameters for configuring the Sumo Logic Telegraf output plugin.

Here’s an explanation for additional values set by this Telegraf configuration.

  • data_format: “prometheus” - In the output [[outputs.sumologic]] plugins section. Metrics are sent in the Prometheus format to Sumo Logic.
  • component - “webserver” - In the input [[inputs.win_perf_counters]] plugins section. This value is used by Sumo Logic apps to identify application components.
  • webserver_system- “iis” - In the input plugins sections. This value identifies the webserver system.

See this doc for all other parameters that can be configured in the Telegraf agent globally.

At this point, Telegraf should start collecting the IIS Server metrics and forward them to the Sumo Logic HTTP Source.