Skip to main content

JSON Parameters for Hosted Sources

This topic describes JSON Source parameters for Hosted Collectors. See the following topics for additional information:

note

JSON files need to be UTF-8 encoded following RFC 8259.

Common Parameters

See Use JSON to Configure Sources for a list of common parameters.

Source types for Hosted Collectors

Each Source can have its own unique fields in addition to the generic fields listed in Use JSON to Configure Sources. The sourceType field determines the type of Source (and the associated parameters). The next table lists the valid field types. The sections that follow list the unique parameters for each and associated JSON examples.

The Google Workspace Apps Audit Source cannot be created with JSON. This Source requires manual 0Auth confirmation which is only supported in the Sumo web interface.

Log Sources for Installed Collectors

Field TypeType Value
Local File SourceLocalFile
Remote File SourceRemoteFileV2
Local Windows Event Log SourceLocalWindowsEventLog
Remote Windows Event Log SourceRemoteWindowsEventLog
Local Windows Performance SourceLocalWindowsPerfMon
Remote Windows Performance SourceRemoteWindowsPerfMon
Windows Active Directory Inventory SourceActiveDirectory
Syslog SourceSyslog
Script SourceScript
Docker Log SourceDockerLog
Docker Stats SourceDockerStats

Metric Sources for Installed Collectors

Field TypeType Value
Host Metrics SourceSystemStats
Streaming Metrics SourceStreamingMetrics

Log Sources for Hosted Collectors

Field TypeType Value
Akamai SIEM API SourceUniversal
Amazon S3 SourcePolling
AWS S3 Archive SourcePolling
AWS CloudFront SourcePolling
AWS CloudTrail SourcePolling
AWS Elastic Load Balancing SourcePolling
AWS Kinesis Firehose for Logs SourceHTTP
Amazon S3 Audit SourcePolling
AWS Metadata (Tag) SourcePolling
Azure Event Hubs SourceUniversal
Carbon Black Cloud SourceUniversal
Carbon Black Inventory SourceUniversal
Cloud Syslog SourceCloudsyslog
Cisco AMP SourceUniversal
Cisco Meraki SourceUniversal
Crowdstrike FDR SourceUniversal
CrowdStrike SourceUniversal
Cloud SIEM AWS EC2 Inventory SourceUniversal
Cybereason SourceUniversal
Duo SourceUniversal
Google Cloud Platform SourceHTTP
HTTP SourceHTTP
Microsoft Graph Security API SourceUniversal
Mimecast SourceUniversal
Netskope SourceUniversal
Okta SourceUniversal
Palo Alto Cortex XDRUniversal
Proofpoint On Demand SourceUniversal
Proofpoint TAP SourceUniversal
Salesforce SourceUniversal
Sophos Central SourceUniversal
Tenable SourceUniversal

Metrics sources for hosted collectors

Field TypeType Value
AWS CloudWatch SourcePolling

Log Source parameters for Hosted Collectors

HTTP Source

In addition to the common parameters, the following parameters are for an HTTP Source.

ParameterTypeRequired?DefaultDescriptionAccess
fieldsJSON ObjectNoEnable Extended HTTP Metadata Collection by adding _convertHeadersToFields=true.modifiable
sourceTypeStringYesHTTPnot modifiable
messagePerRequestBooleanYesWhen set to true, only a single message will be sent for each HTTP request. To disable this feature, set to false.
You need to specify the common parameter multilineProcessingEnabled as false when setting messagePerRequest to true.
modifiable
urlStringNoURL assigned by Sumo for SourceThe Source's unique HTTP endpoint web address.not modifiable
HTTP Source JSON example
{
"api.version": "v1",
"source": {
"sourceType": "HTTP",
"name": "Example1",
"fields":{
"_convertHeadersToFields":"true"
},
"messagePerRequest": true,
"multilineProcessingEnabled": false
}
}

Cloud Syslog Source

In addition to the common parameters, the following parameters are for a Cloud Syslog Source.

ParameterTypeRequired?DefaultDescriptionAccess
sourceTypeStringYes Cloudsyslognot modifiable

Cloud Syslog Source JSON example: 

{
"api.version":"v1",
"source":{
"sourceType":"Cloudsyslog",
"name":"Example1"
}
}

Google Cloud Platform Source

In addition to the common parameters, the following parameters are for a Google Cloud Platform Source.

ParameterTypeRequired?DefaultDescriptionAccess
sourceTypeStringYesHTTPnot modifiable
thirdPartyRefNestedJSONYesSpecify the the serviceType as GoogleCloudLogs.

Google Cloud Platform Source JSON example: 

{
"source": {
"name": "Example-GCP",
"thirdPartyRef":{
"resources":[
{
"serviceType":"GoogleCloudLogs"
}
]
},
"sourceType": "HTTP"
}
}

AWS Log Sources

In addition to the common parameters, the following parameters are for all AWS log Sources except Kinesis.

ParameterTypeRequired?DefaultDescriptionAccess
sourceTypeStringYesPollingnot modifiable
contentTypeStringNoDefine based on the AWS Source you are creating.
Archive: AwsS3ArchiveBucket
S3: AwsS3Bucket
S3 Audit: AwsS3AuditBucket
CloudFront: AwsCloudFrontBucket
CloudTrail: AwsCloudTrailBucket
ELB: AwsElbBucket
Metadata: AwsMetadata
Kinesis: KinesisLog
not modifiable
scanIntervalLongYes300000Time interval of S3 bucket scans for new data, in milliseconds. Minimum value: 1000
For Automatic assign to: -1
modifiable
pausedBooleanYesfalseWhen set to true, the scanner for S3 bucket items is paused. To disable, set to false.modifiable
urlStringNoURLassigned by Sumo for SourceUsed to set up Event Based Notifications with AWS. This value is created and assigned by Sumo when the Source is created.
thirdPartyRefNestedJSONYesIncludes all required information for third-party integration, including the S3 bucket name, path expression for the S3 objects, and access credentials.
See examples below table.
snsTopicOrSubscriptionArn
This parameter goes in the thirdPartyRef nested JSON.
StringNoSNS topic/subscription ARN. If SNS has been successfully configured and has received a subscription confirmation request isSuccess will be true.
See example below table.
not modifiable
bucketName
This parameter goes in the thirdPartyRef nested JSON.
StringYesName of your Amazon S3 bucket.Modifiable
pathExpression
This parameter goes in the thirdPartyRef nested JSON.
StringYesWildcard pattern that matches the S3 objects you'd like to collect.Modifiable

IAM User authentication example:

"authentication": {
"type": "S3BucketAuthentication",
"awsId": "AKIAIOSFODNN7EXAMPLE",
"awsKey": "*******"
}

IAM Role authentication example:

"authentication": {
"type": "AWSRoleBasedAuthentication",
"roleARN": "arn:aws:iam::123456789012:role/myrole"
}

snsTopicOrSubscriptionArn example

"thirdPartyRef": {
"resources": [{
"serviceType": "#",
"path": {
"type": "#",
"bucketName": "#",
"pathExpression": "*",
"snsTopicOrSubscriptionArn": {
"isSuccess": true,
"arn": "arn:aws:sns:#:#:SumoSNSTopic-#:#"
}
},
"authentication": {...
}
}]
}

AWS S3 Archive Source

The parameters for this Source can be referenced in the AWS Log Sources section. This is an AWS S3 Archive Source JSON example:

{
"api.version":"v1",
"source":{
"name":"my_new_archive",
"automaticDateParsing":true,
"multilineProcessingEnabled":true,
"useAutolineMatching":true,
"contentType":"AwsS3ArchiveBucket",
"forceTimeZone":false,
"filters":[],
"cutoffTimestamp":0,
"encoding":"UTF-8",
"fields":{

},
"thirdPartyRef":{
"resources":[{
"serviceType":"AwsS3ArchiveBucket",
"path":{
"type":"S3BucketPathExpression",
"bucketName":"awsbucketname",
"pathExpression":"*"
},
"authentication":{
"type":"AWSRoleBasedAuthentication",
"roleARN":"arn:aws:iam::123456789012:role/myrole"
}
}]
},
"scanInterval":300000,
"paused":false,
"sourceType":"Polling"
}
}

Amazon S3 Source

The parameters for this Source can be referenced in the AWS Log Sources section. This is an Amazon S3 Source JSON example: 

{
"api.version":"v1",
"source":{
"sourceType": "Polling",
"name": "Example1",
"contentType": "AwsS3Bucket",
"thirdPartyRef":{
"resources":[
{
"serviceType": "AwsS3Bucket",
"path": {
"type": "S3BucketPathExpression",
"bucketName": "Bucket1",
"pathExpression": "*"
},
"authentication": {
"type": "AWSRoleBasedAuthentication",
"roleARN": "arn:aws:iam::123456789012:role/myrole"
}
}
]
},
"scanInterval": 300000,
"paused": false
}
}

AWS Elastic Load Balancing Source

The parameters for this Source can be referenced in the AWS Log Sources section. This is an AWS Elastic Load Balancing Source JSON example: 

{
"api.version": "v1",
"source": {
"sourceType": "Polling",
"name": "Example1",
"contentType": "AwsElbBucket",
"thirdPartyRef": {
"resources": [{
"serviceType": "AwsElbBucket",
"path": {
"type": "S3BucketPathExpression",
"bucketName": "Bucket1",
"pathExpression": "*"
},
"authentication": {
"type": "AWSRoleBasedAuthentication",
"roleARN": "arn:aws:iam::123456789012:role/myrole"
}
}]
},
"scanInterval": 300000,
"paused": false
}
}

AWS CloudFront Source

The parameters for this Source can be referenced in the AWS Log Sources section. This is an AWS CloudFront Source JSON example: 

{
"api.version": "v1",
"source": {
"sourceType": "Polling",
"name": "Example1",
"contentType": "AwsCloudFrontBucket",
"thirdPartyRef": {
"resources": [{
"serviceType": "AwsCloudFrontBucket",
"path": {
"type": "S3BucketPathExpression",
"bucketName": "Bucket1",
"pathExpression": "*"
},
"authentication": {
"type": "AWSRoleBasedAuthentication",
"roleARN": "arn:aws:iam::123456789012:role/myrole"
}
}]
},
"scanInterval": 300000,
"paused": false
}
}

AWS CloudTrail Source

The parameters for this Source can be referenced in the AWS Log Sources section. This is an AWS CloudTrail Source JSON example: 

{
"api.version": "v1",
"source": {
"sourceType": "Polling",
"name": "Example1",
"contentType": "AwsCloudTrailBucket",
"thirdPartyRef": {
"resources": [{
"serviceType": "AwsCloudTrailBucket",
"path": {
"type": "S3BucketPathExpression",
"bucketName": "Bucket1",
"pathExpression": "*"
},
"authentication": {
"type": "AWSRoleBasedAuthentication",
"roleARN": "arn:aws:iam::123456789012:role/myrole"
}
}]
},
"scanInterval": 300000,
"paused": false
}
}

Amazon S3 Audit Source

The parameters for this Source can be referenced in the AWS Log Sources section. This is an AWS S3 Audit Source JSON example: 

{
"api.version": "v1",
"source": {
"sourceType": "Polling",
"name": "Example1",
"contentType": "AwsS3AuditBucket",
"thirdPartyRef": {
"resources": [{
"serviceType": "AwsS3AuditBucket",
"path": {
"type": "S3BucketPathExpression",
"bucketName": "Bucket1",
"pathExpression": "*"
},
"authentication": {
"type": "AWSRoleBasedAuthentication",
"roleARN": "arn:aws:iam::123456789012:role/myrole"
}
}]
},
"scanInterval": 300000,
"paused": false
}
}

AWS Metadata (Tag) Source

The parameters for this Source can be referenced in the AWS Log Sources section. This is an AWS Metadata Source JSON example: 

{
"api.version": "v1",
"source": {
"sourceType": "Polling",
"name": "exampleName",
"description": "Poll metadata for data",
"automaticDateParsing": false,
"multilineProcessingEnabled": false,
"useAutolineMatching": false,
"contentType": "AwsMetadata",
"forceTimeZone": false,
"filters": [],
"cutoffTimestamp": 0,
"encoding": "UTF-8",
"fields": {

},
"thirdPartyRef": {
"resources": [{
"serviceType": "AwsMetadata",
"path": {
"type": "AwsMetadataPath",
"limitToRegions": ["ap-southeast-2", "us-east-1", "us-west-2"],
"limitToNamespaces": ["AWS/EC2"],
"tagFilters": ["tag1", "tag2", "tag3", "tag4", "name"]
},
"authentication": {
"type": "AWSRoleBasedAuthentication",
"roleARN": "arn:aws:iam::123456789012:role/myrole"
}
}]
},
"scanInterval": 60000,
"paused": false
}
}

AWS Kinesis Firehose for Logs Source

This is an AWS Kinesis Firehose for Logs Source JSON example. It uses the thirdPartyRef parameter to enable S3 Replay.

{
"source":
{
"name":"test-source",
"automaticDateParsing":true,
"multilineProcessingEnabled":true,
"useAutolineMatching":true,
"contentType":"AwsS3Bucket",
"forceTimeZone":false,
"filters":[],
"cutoffTimestamp":1634972400000,
"encoding":"UTF-8",
"fields":{},
"thirdPartyRef":{
"resources":[
{
"serviceType":"AwsS3Bucket",
"path":{
"type":"S3BucketPathExpression",
"bucketName":"test-kinesis-bucket-name",
"pathExpression":"http-endpoint-failed/*",
"useVersionedApi":true
},
"authentication":{
"type":"S3BucketAuthentication",
"awsId":"XXX",
"awsKey":"XXX"
}
}
]
},
"sourceType":"HTTP"
}
}

To disable S3 Replay use the NoPathExpression placeholder for path and authentication, for example:

{
"source":
{
"name": "test-kinesis-source",
"automaticDateParsing": true,
"multilineProcessingEnabled": true,
"useAutolineMatching": true,
"contentType": "KinesisLog",
"forceTimeZone": false,
"filters": [],
"cutoffTimestamp": 0,
"encoding": "UTF-8",
"fields": {},
"thirdPartyRef": {
"resources": [
{
"serviceType": "KinesisLog",
"path": {
"type": "NoPathExpression"
},
"authentication": {
"type": "NoAuthentication"
}
}
]
},
"messagePerRequest": false,
"sourceType": "HTTP"
}
}

Metrics Source parameters for Hosted Collectors

AWS CloudWatch Source

The following parameters are for an AWS CloudWatch Source.

ParameterTypeRequired?DefaultDescriptionAccess
nameStringYesType a desired name of the Source. The name must be unique per Collector. This value is assigned to the metadata field _source.modifiable
descriptionStringNonullType a description of the Source.modifiable
categoryStringNonullType a category of the source. This value is assigned to the metadata field _sourceCategory. See best practices for details.modifiable
sourceTypeStringYesPollingnot modifiable
contentTypeStringNoAwsCloudWatchnot modifiable
scanIntervalLongYes300000Time interval of S3 bucket scans for new data, in milliseconds.
Minimum value: 1000
modifiable
pausedBooleanYesfalseWhen set to true, the scanner for metrics is paused. To disable, set to false.modifiable
thirdPartyRefNestedJSONYesIncludes all required information for third-party integration, including the relevant Amazon regions, namespaces, and access credentials.

AWS CloudWatch Source JSON example: 

{
"api.version":"v1",
"source":{
"sourceType": "Polling",
"name": "Example1",
"contentType": "AwsCloudWatch",
"thirdPartyRef":{
"resources":[
{
"serviceType": "AwsCloudWatch",
"path": {
"type": "CloudWatchPath",
"limitToRegions": ["region-1", "region-2"],
"limitToNamespaces": ["AWS/ELB", "AWS/Route53", "AWS/OpsWork"]
},
"authentication": {
"type": "S3BucketAuthentication",
"awsId": "AKIAIOSFODNN7EXAMPLE",
"awsKey": "*******"
}
}
]
},
"scanInterval": 300000,
"paused": false
}
}

AWS Kinesis Firehose for Metrics Source

This is an AWS Kinesis Firehose for Metrics Source JSON example:

{
"api.version": "v1",
"source": {
"sourceType": "HTTP",
"name": "CloudWatch Metrics via Kinesis",
"contentType": "KinesisMetric",
"description": "Managed by Terraform in content-management / myDeployment",
"fields": {},
"thirdPartyRef": {
"resources": [
{
"serviceType": "KinesisMetric",
"authentication": {
"roleARN": "arn:aws:iam::123456789012:role/myrole",
"type": "AWSRoleBasedAuthentication"
},
"path": {
"type": "KinesisMetricPath",
"tagFilters": [
{
"namespace": "AWS/EBS",
"tags": [
"Deployment=myDeployment",
"Cluster=my_cluster_a;my_cluster_b",
"VolumeSet=commitlog;data"
],
"type": "TagFilters"
}
}
}
]
}
}
}
Legal
Privacy Statement
Terms of Use

Copyright © 2024 by Sumo Logic, Inc.