JSON Parameters for Installed Sources

This topic describes JSON Source parameters for installed Collectors. For more information about configuring sources in JSON files, and how to configure Sumo to read the JSON files, see Local Configuration File Management.

info

JSON files need to be UTF-8 encoded following RFC 8259.

Source types for installed collectors

Each Source can have its own unique fields in addition to the generic fields listed in Use JSON to configure sources. The sourceType field determines the type of Source (and the associated parameters). The next table lists the valid field types. The sections that follow list the unique parameters for each and associated JSON examples.

Log source parameters for installed collectors

Field Type	Type Value
Local File Source	LocalFile
Remote File Source	RemoteFileV2
Local Windows Event Log Source	LocalWindowsEventLog
Remote Windows Event Log Source	RemoteWindowsEventLog
Local Windows Performance Source	LocalWindowsPerfMon
Remote Windows Performance Source	RemoteWindowsPerfMon
Windows Active Directory Inventory Source	ActiveDirectory
Syslog Source	Syslog
Script Source	Script
Docker Log Source	DockerLog
Docker Stats Source	DockerStats

Local file source

In addition to the common parameters, the following parameters are for local file source.

Parameter	Type	Required?	Default	Description	Access
`sourceType`	String	Yes		LocalFile	not modifiable
`pathExpression`	String	Yes		A valid path expression (full path) of the file to collect. For files on Windows systems (not including Windows Events, enter the absolute path including the drive letter. Escape special characters and spaces with a backslash [`\`]. If you are collecting from Windows using CIFS/SMB, see Prerequisites for Windows Log Collection. Use a single asterisk wildcard [``] for file or folder names. Example: `[var/foo/.log]`. Use two asterisks [``] to recurse within directories and subdirectories. Example: `[var//*.log]`.	modifiable
`denylist`	String Array	No	[ ]	Comma-separated list of valid path expressions from which logs will not be collected. Example: `["/var/log/*/.bak","/var/oldlog/*.log"]`	modifiable
`encoding`	String	No	UTF-8	Defines the encoding form. Default is `"UTF-8"`; options include `"UTF-16"`; `"UTF-16BE"`; `"UTF-16LE"`.	modifiable

Local File Source JSON example with cutoffTimestamp:

{
   "api.version":"v1",
   "sources":[{
    "name":"Test-Chef",
    "category":"Chef",
    "automaticDateParsing":true,
    "multilineProcessingEnabled":false,
    "useAutolineMatching":false,
    "forceTimeZone":false,
    "timeZone":"UTC",
    "filters":[],
    "cutoffTimestamp":1426057200000,
    "encoding":"UTF-8",
    "pathExpression":"/home/ubuntu/chef*.log",
    "denylist":[],
    "sourceType":"LocalFile"
  }]
}

Remote file source

In addition to the common parameters, the following parameters are for remote file source.

Parameter	Type	Required?	Default	Description	Access
`sourceType`	String	Yes		RemoteFileV2	not modifiable
`remoteHosts`	List	Yes		Host name of remote machine. Make sure to enclose IP addresses in brackets. Example: `["192.168.0.1","10.0.1.16", "192.168.1.234"]`.	modifiable
`remotePort`	Int	Yes		Port of remote machine (SSH)	modifiable
`remoteUser`	String	Yes		User account to connect to the remote machine.	modifiable
`remotePassword`	String	Yes		Password used to connect to remote machine. Required only when authMethod is set to `"password"`.	modifiable
`keyPath`	String	Yes		Path to SSH key used to connect to the remote machine. Required only when authMethod is set to `"key"`.	modifiable
`keyPassword`	String	No	Null	Password to SSH key to connect to the remote machine, required only with authMethod is set to `"password"`.	modifiable
`pathExpression`	String	Yes		Path expression of the files to collect.	modifiable
`authMethod`	String	Yes		Authentication method used to connect to the remote machine. Options are `"password"` to connect with a password or `"key"` to connect with an SSH key.	modifiable
`denylist`	List	No	[ ]	List of valid path expression to skip. Default is `[ ]`.	modifiable

Remote file source JSON example:

{
   "api.version":"v1",
   "sources":[
      {
         "sourceType":"RemoteFileV2",
         "name":"Example1",
         "remoteHosts":[
            "192.168.0.1",
            "10.0.1.16",
            "192.168.1.234"
         ],
         "remotePort":22,
         "remoteUser":"user",
         "remotePassword":"password",
         "keyPath":"",
         "keyPassword":"",
         "pathExpression":"/var/log/somelog.log",
         "authMethod":"password",
         "denylist":[
            "/var/log/*.out.log",
            "/var/log/*.tmp.log"
         ]
      }
   ]
}

Local Windows event log source

In addition to the common parameters, except hostName, the following parameters are for local Windows event log source. The Source Host (_sourceHost) value is parsed and applied to your event logs automatically. The value is parsed from the field Computer in your event logs.

Parameter	Type	Required?	Default	Description	Access
`sourceType`	String	Yes		LocalWindowsEventLog	not modifiable
`logNames`	List	Yes		List of Windows log types to collect. For example, `"Security"` or `"Application"`. To obtain the list of available logs on a given machine, use the PowerShell command `Get-WinEvent -ListLog *` or the legacy command `wevtutil el`. We do not support `"Analytic"` or `"Debug"` ETW logs.	modifiable
`renderMessages`	Boolean	No, only applicable to the legacy format.	true	When using legacy format, this indicates if full event messages are collected (`true`) or just core event built-in metadata (`false`).	modifiable
`eventFormat`	Integer	No	0	Sets how you want your event logs formatted. Use 0 for the legacy format. Where events retain their default XML format from Windows. Use 1 for JSON format. Where events are formatted into JSON that is designed to work with Sumo Logic features, making it easier for you to reference your data.	modifiable
`eventMessage`	Integer	Yes if eventFormat is 0.		Use 0 for the complete message. It will ingest the entire event content along with metadata. Use 1 for the message title. It will ingest the first line of event messages along with all of the metadata. Use 2 for metadata only. It will ingest metadata fields from each event, including event ID and timestamp.	modifiable
`allowlist`	String array	No		Available in Collector version 19.351-4 and later. You can set allow and deny Windows Event ID filters to only collect important events. We recommend only using one at a time. Your list needs to be a comma-separated list of event IDs.	modifiable
`denylist`	String array	No		Available in Collector version 19.351-4 and later. You can set allow and deny Windows Event ID filters to only collect important events. We recommend only using one at a time. Your list needs to be a comma-separated list of event IDs.	modifiable

Local Windows event log source JSON example:

{
   "api.version":"v1",
   "sources":[
      {
         "sourceType":"LocalWindowsEventLog",
         "name":"Example1",
         "allowlist":"100,101,102,103,104,105",
         "logNames":["Security","Application","System"],
         "eventFormat":1,
         "eventMessage":1
      }
   ]
}

Remote Windows event log source

In addition to the common parameters, except hostName, the following parameters are for remote Windows event log source. The Source Host (_sourceHost) value is parsed and applied to your event logs automatically. The value is parsed from the field Computer in your event logs.

Parameter	Type	Required?	Default	Description	Access
`sourceType`	String	Yes		RemoteWindowsEventLog	not modifiable
`domain`	String	Yes		Windows domain from which logs will be created.	modifiable
`username`	String	Yes		Username needed to connect to the remote machine.	modifiable
`password`	String	Yes		Password needed to connect to the remote machine.	modifiable
`hosts`	List	Yes		List of hosts to collect from.	modifiable
`logNames`	List	Yes		List of Windows log types collected.	modifiable
`logNames`	List	Yes		List of Windows log types to collect. For example, `"Security"` or `"Application"`. To obtain the list of available logs on a given machine, use the PowerShell command `Get-WinEvent -ListLog *` or the legacy command `wevtutil el`. We do not support `"Analytic"` or `"Debug"` ETW logs.	modifiable
`renderMessages`	Boolean	No, only applicable to the legacy format.	true	When using legacy format, this indicates if full event messages are collected (`true`) or just core event built-in metadata (`false`).	modifiable
`eventFormat`	Integer	No	0	Sets how you want your event logs formatted. Use 0 for the legacy format. Where events retain their default XML format from Windows. Use 1 for JSON format. Where events are formatted into JSON that is designed to work with Sumo Logic features, making it easier for you to reference your data.	modifiable
`eventMessage`	Integer	Yes if eventFormat is 0.		Use 0 for the complete message. It will ingest the entire event content along with metadata. Use 1 for the message title. It will ingest the first line of event messages along with all of the metadata. Use 2 for metadata only. It will ingest metadata fields from each event, including event ID and timestamp.	modifiable
`allowlist`	String array	No		Available in Collector version 19.351-4 and later. You can set allow and deny Windows Event ID filters to only collect important events. We recommend only using one at a time. Your list needs to be a comma-separated list of event IDs.	modifiable
`denylist`	String array	No		Available in Collector version 19.351-4 and later. You can set allow and deny Windows Event ID filters to only collect important events. We recommend only using one at a time. Your list needs to be a comma-separated list of event IDs.	modifiable

Remote Windows event log source JSON example:

{
   "api.version":"v1",
   "sources":[
      {
         "sourceType":"RemoteWindowsEventLog",
         "name":"Example1",
         "domain":"mydomain",
         "username":"user",
         "password":"password",
         "hosts":[
            "myremotehost1",
            "myremotehost2"
         ],
         "logNames":["Security","Application","System"],
         "eventFormat":1,
         "eventMessage":1
      }
   ]
}

Local Windows performance source

In addition to the common parameters, the following parameters are for local Windows performance source.

Parameter	Type	Required?	Default	Description	Access
`sourceType`	String	Yes		LocalWindowsPerfMon	not modifiable
`wmiQueries`	List	Yes		List of queries to be executed. Each query is an object with two fields: name and query	modifiable

Example response:

{
   "api.version":"v1",
   "sources":[
      {
         "sourceType":"LocalWindowsPerfMon",
         "name":"Example1",
         "wmiQueries":[
            {
               "name":"query_1",
               "query":"select * from Win32_PerfFormattedData_PerfOS_Processor"
            },
            {
               "name":"query_2",
               "query":"select * from NonExistence"
            }
         ]
      }
   ]
}

Remote Windows performance source

In addition to the common parameters, the following parameters are for remote Windows performance source.

Parameter	Type	Required?	Default	Description
`sourceType`	String	Yes	RemoteWindowsPerformance	not modifiable
`domain`	String	Yes	Windows domain from which logs will be created.	modifiable
`remoteUser`	String	Yes	User name needed to connect to the remote machine.
`remotePassword`	String	Yes	Password needed to connect to the remote machine.
`remoteHosts`	List	Yes	List of hosts to collect from.
`wmiQueries`	List	Yes	List of queries to be executed. Each query in an object with two fields: name and query

Remote Windows performance source JSON example:

{
   "api.version":"v1",
   "sources":[
      {
         "sourceType":"RemoteWindowsPerformance",
         "name":"Example1",
         "domain":"mydomain",
         "remoteUser":"user",
         "remotePassword":"password",
         "remoteHosts":[
            "myremotehost1",
            "myremotehost2"
         ],
         "wmiQueries":[
            {
               "name":"query_1",
               "query":"select * from Win32_PerfFormattedData_PerfOS_Processor"
            },
            {
               "name":"query_2",
               "query":"select * from NonExistence"
            }
         ]
      }
   ]
}

Windows performance metric example

This example shows how to use WMI queries to collect performance metrics from Windows systems.

{
   "api.version":"v1",
   "sources":[
      {
         "name":"Windows Performance",
         "sourceType":"LocalWindowsPerfMon"
         "automaticDateParsing":false,
         "multilineProcessingEnabled":false,
         "useAutolineMatching":false,
         "forceTimeZone":false,
         "filters":[],
         "cutoffTimestamp":0,
         "encoding":"UTF-8",
         "interval":300000,
         "wmiQueries":[
            {
               "name":"CPU",
               "query":"select * from Win32_PerfFormattedData_PerfOS_Processor"
            },
            {
               "name":"Logical Disk",
               "query":"select * from Win32_PerfFormattedData_PerfDisk_LogicalDisk"
            },
            {
               "name":"Physical Disk",
               "query":"select * from Win32_PerfFormattedData_PerfDisk_PhysicalDisk"
            },
            {
               "name":"Memory",
               "query":"select * from Win32_PerfFormattedData_PerfOS_Memory"
            },
            {
               "name":"Network",
               "query":"select * from Win32_PerfFormattedData_Tcpip_NetworkInterface"
            }
         ]
      }
   ]
}

Windows Active Directory Inventory Source

In addition to the common parameters, the following parameters are for a Windows Active Directory Inventory Source.

Parameter	Type	Required?	Default	Description
`sourceType`	String	Yes	ActiveDirectory	not modifiable
`nameSuffixes`	String	No	Semi-colon separated list of Distinguished Name suffixes. When set, the source won't ingest any records that contain the Distinguished Name suffixes specified.	modifiable
`filter`	String	No	Specifies a filter to use when searching for Domain Objects in Active Directory.	modifiable
`additionalAttr`	String	No	Semi-colon separated list of the LDAP Names of Active Directory attributes to report, in addition to the default list.	modifiable
`excludedAttr`	String	No	Semi-colon separated list of the LDAP Names of Active Directory attributes to be excluded from the report.	modifiable
`interval`	Integer	No	Seconds to check for new data. By default, Active Directory is queried for data every 24 hours. You can select a more frequent interval down to every minute.	modifiable

Windows Active Directory Inventory Source JSON example:

{
  "api.version":"v1",
  "source":{
    "name":"AD",
    "category":"{{sys.PATH}}-{{sys.YourEnvVar}}",
    "automaticDateParsing":false,
    "multilineProcessingEnabled":false,
    "useAutolineMatching":false,
    "forceTimeZone":false,
    "filters":[],
    "cutoffTimestamp":0,
    "encoding":"UTF-8",
    "fields":{

    },
    "nameSuffixes":"CN=System",
    "filter":"(objectClass=user)",
    "interval":60000,
    "additionalAttr":"name",
    "excludedAttr":"whenCreated;objectGUID",
    "sourceType":"ActiveDirectory"
  }
}

Syslog source

Syslog Sources do not support Multiline Detection, which means the common parameters multilineProcessingEnabled, useAutolineMatching and manualPrefixRegexp are not applicable. If you provide these in the Syslog Source configuration they are ignored.

note

Syslog sources break a syslog message on each newline character and send each line as individual messages to the service.

Parameter	Type	Required?	Description	Access
`sourceType`	String	Yes	Syslog	not modifiable
`protocol`	String	Yes	Protocol that syslog should use. Both UDP and `TCP` are supported.	modifiable
`port`	Integer	Yes	Port that syslog should use to connect to the machine. Recommended ports: `514` or `1514`	modifiable

Syslog source JSON example:

{
   "api.version":"v1",
   "sources":[
      {
         "sourceType":"Syslog",
         "name":"Example1",
         "protocol":"UDP",
         "port":514
      }
   ]
}

Script source

In addition to the common parameters, the following parameters are for script source.

Parameter	Type	Required?	Default	Description	Access
`sourceType`	String	Yes		Script	not modifiable
`commands`	List	Yes	`[ ]`	List of command line arguments.	modifiable
`file`	String	No	null	Path to script file to run	modifiable
`workingDir`	String	No	null	Working directory for commands/script.	modifiable
`timeout`	Long	No	0	Script timeout (in milliseconds). By default, this is set to 0.	modifiable
`script`	String	No	null	Script contents (if no file is provided).	modifiable
`cronExpression`	String	Yes		Schedule for running the script. Must be a valid Quartz cron expression.	modifiable

Script Source JSON Example:

{
   "api.version":"v1",
   "sources":[
      {
         "sourceType":"Script",
         "name":"Example1",
         "commands":[
            "/bin/bash"
         ],
         "file":"/usr/local/bin/getlogs.log",
         "workingDir":"/var/log",
         "timeout":60000,
         "script":"",
         "cronExpression":"0 * * * *"
      }
   ]
}

Docker log source

In addition to the common parameters, the following parameters are for Docker log source.

Parameter	Type	Required?	Description	Access
`sourceType`	String	Yes	DockerLog
`uri`	String	Yes	URI of the Docker daemon.	modifiable
`specifiedContainers`	List		Comma-separated list of Docker containers. Collection will be only from running containers. If the list contains stopped containers, the source can start collecting from these containers if they are started later. For advanced container filtering options, see More about defining container filters.	modifiable
`allContainers`	Boolean	Yes	Flag indicating whether the Source includes all running containers (`true`) or only the containers listed in `specifiedContainers` (`false`).	modifiable
`certPath`	String	`*`	Enter the path to the cert files on the local machine where the Collector is running. Required if the URI uses HTTPS.	modifiable
`collectEvents`	Boolean	Yes	Must be set to `true` to collect the Docker logs.

Example source JSON with all containers:

{
   "api.version":"v1",
   "sources":[
      {
         "sourceType":"DockerLog",
         "name":"Example1",
         "uri":"https://54.165.12.163:2376",
         "allContainers":true,
         "certPath":"/home/ec2-user/.docker/machine/machines/wmad-docker",
         "collectEvents":true
      }
   ]
}

Example source JSON with specified containers:

{
   "api.version":"v1",
   "sources":[
      {
         "sourceType":"DockerLog",
         "name":"Example1",
         "uri":"https://54.165.12.163:2376",
         "specifiedContainers":[
            "webserver",
            "mysql",
            "another-container"
         ],
         "allContainers":false,
         "certPath":"/home/ec2-user/.docker/machine/machines/wmad-docker",
         "collectEvents":true
      }
   ]
}
}

Docker stats source

In addition to the common parameters, the following parameters are for Docker stats source.

Parameter	Type	Required?	Default	Description	Access
`sourceType`	String	Yes		DockerStats	not modifiable
`contentType`	String	No		If value is empty or does not exist, it’s a json-based source, if value is “DockerMetrics”, it’s a metrics source.	modifiable
`metrics`	String	Array	No	All metrics List of metrics to be collected. Each metric is an object with two fields, category and metric. For a full list of available metrics, see Docker metrics definitions. When omitted, all available host metrics will be collected.	modifiable
`uri`	String	Yes		URI of the Docker daemon.	modifiable
`specifiedContainers`	List			Comma-separated list of Docker containers. Collection will be only from running containers. If the list contains stopped containers, the source can start collecting from these containers if they are started later. For advanced container filtering options, see More about defining container filters.	modifiable
`allContainers`	Boolean	Yes		Flag indicating whether the Source includes all running containers (true) or only the containers listed in specifiedContainers (false).	modifiable
`certPath`	String	*		Enter the path to the cert files on the local machine where the Collector is running. Required if the URI uses HTTPS.	modifiable
`pollInterval`	Long	No	Continuous (By default, polling occurs continuously, rather than on a periodic basis.)	The frequency, in milliseconds, at which stats are polled. The minimum acceptable polling frequency is 1 second.	modifiable

Example source JSON with all containers:

{
 "api.version":"v1",
 "source":{
   "name":"test",
   "category":"test",
   "automaticDateParsing":false,
   "multilineProcessingEnabled":false,
   "useAutolineMatching":false,
   "forceTimeZone":false,
   "filters":[],
   "cutoffTimestamp":0,
   "encoding":"UTF-8",
   "allContainers":true,
   "certPath":"",
   "uri":"unix:///var/run/docker.sock",
   "specifiedContainers":[],
   "pollInterval":60000,
   "sourceType":"DockerStats"   }
   ]
}

Example source JSON to collect metrics:

{
  "api.version":"v1",
  "Source":{
    "name":"docker-stats-metrics",
    "contentType":"DockerMetrics",
    "cutoffTimestamp":0,
    "allContainers":true,
    "certPath":"",
    "uri":"unix:///var/run/docker.sock",
    "metrics":[{"category":"CPU", "metric":"cpu_usage.percpu_usage"},{"category":"Memory", "metric":"usage"}],
    "specifiedContainers":[],
    "pollInterval":0,
    "sourceType":"DockerStats"
  }
}

Metric source parameters for installed collectors

Field Type	Type Value
Host metrics Source	SystemStats
Streaming Metrics Source	StreamingMetrics

Host metrics source

The following parameters are for a host metrics source.

Parameter	Type	Required?	Default	Description	Access
`name`	String	Yes		Type a desired name of the Source. The name must be unique per Collector. This value is assigned to the built-in metadata field _source.	modifiable
`description`	String	No	null	Type a description of the Source.	modifiable
`category`	String	No	null	Type a category of the source. This value is assigned to the metadata field `_sourceCategory`. See best practices for details.	modifiable
`sourceType`	String	Yes		SystemStats	not modifiable
`metrics`	String Array	No	all metrics	Comma-separated list of metrics to collect. Example: "metrics" : `["CPU_User", "CPU_Sys", "Mem_Used"]` For a full list of available metrics, see Host Metrics Source. When omitted, all available host metrics will be collected.	modifiable
`interval` (ms)	Integer	Yes		Time interval in milliseconds of the metrics collection. We recommend 60 second granularity (60000). The Sumo Logic UI offers some pre-defined values (10s, 15s, 30s, 1m, 5m).	modifiable
`hostName`	String	No		Host from which the metrics are collected.	modifiable

Host metrics source JSON example:

{
 "api.version": "v1",
 "sources": [{
   "sourceType" : "SystemStats",
   "name" : "Host_Metrics",
   "interval" : 60000,
   "hostName" : "my_host",
   "metrics" : ["CPU_User", "CPU_Sys", "Mem_Used"]
 }]
}

Streaming metrics source

The following parameters are for a streaming metrics source.

info

The streaming metrics source can receive Prometheus data as long as that data is pushed to it. This source cannot scrape Prometheus exporters. For that, we suggesting using Telegraf.

Graphite contentType JSON example:

{
   "api.version":"v1",
   "sources":[
      {
         "sourceType":"StreamingMetrics",
         "contentType":"Graphite",
         "name":"collectd",
         "protocol":"TCP",
         "port":2003
      }
   ]
}

Additional information

See the following topics for additional information:

Use JSON to configure Sources. The topic includes a list of common parameters for all log Source types. For Sources, the common parameter name must be unique per Collector.
JSON Source parameters for Hosted Collectors.
View or download Collector or Source JSON configuration from Sumo Logic.

Source types for installed collectors​

Log source parameters for installed collectors​

Local file source​

Remote file source​

Local Windows event log source​

Remote Windows event log source​

Local Windows performance source ​

Remote Windows performance source ​

Windows performance metric example​

Windows Active Directory Inventory Source ​

Syslog source​

Script source​

Docker log source​

Docker stats source​

Metric source parameters for installed collectors​

Host metrics source ​

Streaming metrics source ​

Additional information​