Skip to main content
Sumo Logic

Collect Logs for Amazon SES

Steps to collect logs from Amazon SES, and to ingest them into Sumo.

Collect Amazon SES Events Using CloudTrail

  1. Configure a Hosted Collector.
  2. To your Hosted Collector, add an AWS CloudTrail Source.
    1. Name. Enter a name to display for the new Source.
    2. Description. Enter an optional description.
    3. S3 Region. Select the Amazon Region for your SES S3 bucket.
    4. Bucket Name. Enter the exact name of your SES S3 bucket.
    5. Path Expression. Enter the string that matches the S3 objects you'd like to collect. You can use a wildcard (*) in this string. (DO NOT use a leading forward slash. See Amazon Path Expressions.)
    6. Source Category. Enter a source category. For example, AWS/Cloudtrail.
    7. Access Key ID and Secret Access Key. Enter your Amazon Access Key ID and Secret Access Key.
    8. Scan Interval. Use the default of 5 minutes. Alternately, enter the frequency Sumo Logic will scan your S3 bucket for new data.
    9. Enable Timestamp Parsing. Select the checkbox.
    10. Time Zone. Select Ignore time zone from log file and instead use, and select UTC.
    11. Timestamp Format. Select Automatically detect the format.
    12. Enable Multiline Processing. Select the checkbox, and select Infer Boundaries.
  3. Click Save.

Collect Amazon SES Notifications Using SNS Notifications

To configure notifications using the Amazon SES console, follow the steps in the AWS documentation here.

Raw_message_delivery1.png

Configure a Source

Configure HTTP source.

  • Name: Required
  • Source Category: AWS/SES/Notifications.
  • Timestamp Parsing Settings:
    • Enable Timestamp Parsing: True
    • Timezone:  Logs are sent in UTC by default and can be auto detected
    • Timestamp Format: Auto Detect
  • Enable One Message Per Request: True

Sample Log Message

CloudTrail log:

{ "eventVersion":"1.04", "userIdentity":{ "type":"IAMUser", "principalId":"AIDAI1234567890YGJ2G6", "arn":"arn:aws:iam::123456789033:user/mkmiller", "accountId":"123456789033", "accessKeyId":"ASI1234567890IHSAOIQ", "userName":"jbrown", "sessionContext":{ "attributes":{ "mfaAuthenticated":"true", "creationDate":"2017-12-12T11:18:58Z" } }, "invokedBy":"signin.amazonaws.com" }, "eventTime":"2018-01-02T19:45:18Z", "eventSource":"ses.amazonaws.com", "eventName":"GetIdentityMailFromDomainAttributes", "awsRegion":"us-west-3", "sourceIPAddress":"220.18.108.139", "userAgent":"signin.amazonaws.com", "requestParameters":{ "identities":[ "pwilson@sumologic.com", "amoore1@sumologic.com" ] }, "responseElements":{ "mailFromDomainAttributes":{ "mkmiller@sumologic.com":{ "behaviorOnMXFailure":"UseDefaultValue" }, "mperez1@sumologic.com":{ "behaviorOnMXFailure":"UseDefaultValue" } } }, "requestID":"9774b3e6-df4d-11e7-8e07-7d3a17657a4d", "eventID":"d36bd7a4-03f0-4245-a6b8-cdb56cfc8e91", "eventType":"AwsApiCall", "recipientAccountId":"123456789033" }

SES log:

{"notificationType":"Delivery","mail":{"timestamp":"2018-02-08T18:18:09.060Z","source":"Sumo Logic <service@sumologic.com>","sourceArn":"arn:aws:ses:us-west-3:123456789029:identity/service@sumologic.com","sourceIp":"19.171.22.2","sendingAccountId":"122226337001","messageId":"010001606dc7dea0-91abab6b-b5fc-47as-921f-813c92ac40ud-000000","destination":["bob@example.com"]},"delivery":{"timestamp":"2017-12-19T07:58:23.735Z","processingTimeMillis":865,"recipients":["jason@sumo.com"],"smtpResponse":"250 2.0.0 OK 1513670303 h58si3264405qta.418 - gsmtp","remoteMtaIp":"169.107.162.237","reportingMTA":"a9-19.smtp-out.amazonses.com"}}

{"notificationType":"Bounce","bounce":\{"bounceType":"Permanent","bounceSubType":"Suppressed","bouncedRecipients":[{"emailAddress":"larry@customer.com","action":"failed","status":"5.1.1","diagnosticCode":"Amazon SES has suppressed sending to this address because it has a recent history of bouncing as an invalid address. For more information about how to remove an address from the suppression list, see the Amazon SES Developer Guide: http://docs.aws.amazon.com/ses/lates...ssionlist.html "}],"timestamp":"2018-04-12T11:46:41.807Z","feedbackId":"010001606e10a2db-3807dda0-4311-4b62-b883-8e0cb4122954-000000","reportingMTA":"dns; amazonses.com"},"mail":\{"timestamp":"2017-12-19T09:17:52.309Z","source":"Sumo Logic <service@sumologic.com>","sourceArn":"arn:aws:ses:us-east-3:123456789029:identity/service@sumologic.com","sourceIp":"169.107.162.237","sendingAccountId":"123456789029","messageId":"010001606e109e93-29782834-7101-4a4a-abbd-2d3e971d1173-000000","destination":["naren@demo.com"]}}

{"notificationType":"Complaint","complaint":{"complainedRecipients":[{"emailAddress":"nathan@sumodemoacme.com"}],"timestamp":"2018-04-12T12:25:07.641Z","feedbackId":"01000162b539f06b-d701b0a8-bde5-48ea-85b2-a8a58e4de012-000000","userAgent":"AOL SComp","complaintFeedbackType":"abuse","arrivalDate":"2018-04-12T12:25:07.641Z"},"mail":{"timestamp":"2018-04-12T12:25:07.641Z","source":"Sumo Logic Information <service@sumologic.com>","sourceArn":"arn:aws:ses:us-west-2:123456789029:identity/service@sumologic.com","sourceIp":"147.106.118.104","sendingAccountId":"123456789029","messageId":"0100016292d33f2f-6a6d0111-cfb3-499b-a667-9edae2d901c5-000000","destination":(["jackson@longsumo.com"]}}

Query Sample

Top bounced email addresses

(_sourceCategory=aws-ses or _sourceCategory=AWS/SES/Notifications) "\"notificationType\":\"Bounce\""
| json "notificationType" nodrop
| json "bounce.bounceSubType" as bounceSubType nodrop
| json "bounce.bounceType" as bounceType nodrop
| json "bounce.bouncedRecipients" as bouncedRecipients nodrop
| parse regex field=bouncedRecipients "\"emailAddress\":\"(?[^\"]*)\"" multi
| parse field=BouncedemailAddress "*@*" as name, domain
| where notificationType="Bounce"
| count as eventCount by BouncedemailAddress
| sort by eventCount, BouncedemailAddress
| limit 10