Skip to main content
Sumo Logic

Collect Logs for Google Compute Engine

Instructions for configuring log collection for the Sumo Logic App for Google Compute Engine.

This page has instructions for configuring Google Compute Engine to send logs to Sumo.

Configure Google Compute Engine to export logs to Stackdriver   

If you haven't already done so, set up Google Cloud Audit to export logs to Stackdriver. For more information, see Overview of Logs Export in GCP.

Set up a Google-validated HTTP source and Pub/Sub topic   

In this step, you set up an HTTP source in Sumo, register it with Google, and create a Pub/Sub topic to send data to the HTTP source. Follow the instructions in Google Cloud Platform Source.

When you create the HTTP Source, assign a Source Category of “GCP”.

Create export of Google Compute Engine logs from Stackdriver

  1. Go to Logging and click Exports.
    gcp6.png
  2. Click Create Export.
    gcp7.png
  3. Select a GCP service to filter the logs. The recommended GCP service to create sinks for is "GCE VM Instance", which sends the service’s logs to Sumo Logic. In the Edit Export window on the right:

    1. Set the Sink Name. For example, "gce-vm-instance".
    2. Select "Cloud Pub/Sub" as the Sink Service.
    3. Set Sink Destination to the newly created Pub/Sub topic. For example, "pub-sub-logs".
    4. Click Create Sink.
      gcp8.png

Sample Log Message

{
 "message":{
   "data":{
     "insertId":"55E9891F381C2.A6AC1EA.F3043722",
     "logName":"projects/wk-dev/logs/cloudaudit.googleapis.com%2Factivity",
     "operation":{
       "first":true,
       "id":"operation-1511384259910-55e9891ee5970-33fdc63d-4bee6b10",
       "producer":"compute.googleapis.com"
     },
     "protoPayload":{
       "@type":"type.googleapis.com/google.cloud.audit.AuditLog",
       "authenticationInfo":{
         "principalEmail":"service-287993422434@dataflow-service-producer-prod.iam.gserviceaccount.com"
       },
       "authorizationInfo":[{
         "granted":true,
         "permission":"compute.instances.delete"
       }],
       "methodName":"v1.compute.instances.delete",
       "requestMetadata":{
         "callerSuppliedUserAgent":"Managed Infrastructure Mixer Client"
       },
       "resourceName":"projects/287993422434/zones/us-central1-f/instances/permissionlogs-yuanwang-1-11221246-d0b6-harness-p548",
       "response":{
         "@type":"compute.googleapis.com/operation",
         "id":"6917821783428586027",
         "insertTime":"2017-11-22T12:57:40.084-08:00",
         "name":"operation-1511384259910-55e9891ee5970-33fdc63d-4bee6b10",
         "operationType":"delete",
         "progress":"0",
         "selfLink":"https://www.googleapis.com/compute/v1/projects/wk-dev/zones/us-central1-f/operations/operation-1511384259910-55e9891ee5970-33fdc63d-4bee6b10",
         "status":"PENDING",
         "targetId":"7642006033207418043",
         "targetLink":"https://www.googleapis.com/compute/v1/projects/wk-dev/zones/us-central1-f/instances/permissionlogs-yuanwang-1-11221246-d0b6-harness-p548",
         "zone":"https://www.googleapis.com/compute/v1/projects/wk-dev/zones/us-central1-f"
       },
       "serviceName":"compute.googleapis.com"
     },
     "receiveTimestamp":"2017-11-22T20:57:41.0202444Z",
     "resource":{
       "labels":{
         "instance_id":"7642006033207418043",
         "project_id":"wk-dev",
         "zone":"us-central1-f"
       },
       "type":"gce_instance"
     },
     "severity":"NOTICE",
     "timestamp":"2017-11-22T20:57:39.896Z"
   },
   "attributes":{
     "logging.googleapis.com/timestamp":"2017-11-22T20:57:39.896Z"
   },
   "message_id":"174545382671298",
   "messageId":"174545382671298",
   "publish_time":"2017-11-22T20:57:42.118Z",
   "publishTime":"2017-11-22T20:57:42.118Z"
 },
 "subscription":"projects/wk-dev/subscriptions/sumo-test"
}

Query Sample

Top 10 users

_collector="HTTP Source for GCP Pub/Sub" logName resource timestamp
| json "message.data.resource.type" as type 
| parse regex "\s+\"logName\":\"(?<log_name>\S+)\"" 
| where type = "gce_instance" and log_name matches "projects/*/logs/cloudaudit.googleapis.com%2Factivity"
| parse regex "\s+\"resourceName\":\"projects/\S+/zones/(?<zone>\S+)/instances/(?<instance>\S+)\""
| json "message.data.resource.labels" as labels
| json field=labels "project_id" as project
| json "message.data.protoPayload.authenticationInfo.principalEmail" as user
| count as requests by user
| sort by requests
| limit 10