Skip to main content
Sumo Logic

Google Cloud Platform Source

You can export in real time all of the data collected by Stackdriver to Google Cloud Pub/Sub. Sumo Logic uses the  Pub/Sub integration to push logs to our platform in real time.

Log data for Google Cloud Platform (GCP) services is collected and exposed through the Google Cloud Stackdriver service. You can export, in real time,  the data collected by Stackdriver to Google Cloud Pub/Sub. We use this Pub/Sub integration to push logs to the Sumo service in real time.

Configure an HTTP source in Sumo

In this step you create the HTTP source to which you'll forward the log data from Google Pub/Sub.

  1. Select an existing hosted collector upon which to configure the source. If you don't already have a collector you'd like to use, create one, using the instructions on Configure a Hosted Collector.
  2. Create an HTTP Source.
    • Assign the source a Source Category, for example "gcp".

Add the source URL as an allowed domain in GCP

In this step, you add the HTTP source URL as an allowed domain to your GCP account. This source will be a Google Pub/Sub-only source, which means that it will only be usable for log data formatted as data coming from Google Pub/Sub.

  1. Open your Google Cloud Console.
  2. Select Products and services > API Manager > Credentials.
  3. Select Domain Verification > Add Domain.
    API Manager
  4. In the Configure webhook notifications for … dialog,  add the HTTP source URL as valid domain and click Add Domain.

  5. Click Take Me There to verify ownership of the URL at Google’s webmaster central page. You are taken to the Google’s Webmaster Central interface to verify the URL.  
  6. Click Add Property in the Webmaster Central site and add the HTTP Source URL.
  7. Do not click Verify yet. Download the HTML verification file and complete next steps before clicking Verify

Update source URL with HTML verification

In this step, you use the Sumo Logic Collector Management API to update the HTTP source with information from the HTML verification file that you downloaded from Webmaster Central.

  1. Later an Access Key and ID. Save them because you will need them later.

  2. Click on the information icon for the source to display the API usage information popup.

  3. Copy the Source API URL field.

  4. Using the Source API URL, make a GET request like this:
    curl -v -u "accessid:accesskey" -X GET https://yourendpoint/api/v1/collectors/collectorID/sources/sourceID >source.json

    You will see a response containing your etag:

    Response with etag

  5. Make note of the etag value in the response. You will need it.

  6. Edit the JSON file to insert the thirdPartyRef JSON block below, replacing the name field with the filename of the Google HTML Verification File, and the contents field with the full string from the body of the HTML Verification File.

  7. Update the HTTP source with the new thirdPartyRef field and fill in the Google verification information.

             "serviceType": "GoogleCloudLogs",
             "authentication": {
               "type": "GoogleCloudAuthentication",
               "validations": [
                   "name":"<google-validation-html-filename, for example google123a456b78c90d12.html>",
                   "content": "<google-validation-html-file-content, for example google-site-verification:google123a456b78c90d12.html >"

  8. Update the Source JSON file using the API with the following CURL command. You must add the etag value in the example below.
    curl -u "accessid:accesskey" -X PUT -H "Content-Type: application/json" -H "If-Match: \"etag\"" -T source.json
  9. After the JSON file is uploaded, return to the Google, and click Verify on the page as described in step 6 of the  Add the source URL as an allowed domain in GCP section above. It should verify successfully.

Configure a Pub/Sub topic for GCP projects

In this step, you configure a Pub/Sub topic in GCP and add a subscription to the above source URL. Once you configure the Pub/Sub, you can export data from Stackdriver to the Pub/Sub. For example, you can export Google App Engine logs, as described on Collect Logs for Google App Engine.

  1. In GCP, select Pub/Sub in the left navigation pane.
  2. In the Pub/Sub pane, select Topics, then click Create Topic in the Topics pane.
  3. Name the topic and click Create.

  4. Select the new topic in the Topics pane, and select New subscription from the options menu.

  5. In the Create a subscription pane:
    1. Subscription Name. Enter a name for the subscription.
    2. Delivery Type. Choose “Push into an endpoint url”, and enter the upload URL for the Sumo HTTP source you created above.
    3. Click Create.