Log data for Google Cloud Platform (GCP) services is collected and exposed through the Google Cloud Stackdriver service. You can export, in real time, the data collected by Stackdriver to Google Cloud Pub/Sub. We use this Pub/Sub integration to push logs to the Sumo service in real time.
Configure an HTTP source in Sumo
In this step you create the HTTP source to which you'll forward the log data from Google Pub/Sub.
Add the source URL as an allowed domain in GCP
In this step, you add the HTTP source URL as an allowed domain to your GCP account. This source will be a Google Pub/Sub-only source, which means that it will only be usable for log data formatted as data coming from Google Pub/Sub.
- Open your Google Cloud Console.
- Select Products and services > API Manager > Credentials.
- Select Domain Verification > Add Domain.
- In the Configure webhook notifications for … dialog, add the HTTP source URL as valid domain and click Add Domain.
- Click Take Me There to verify ownership of the URL at Google’s webmaster central page. You are taken to the Google’s Webmaster Central interface to verify the URL.
- Click Add Property in the Webmaster Central site and add the HTTP Source URL.
- Do not click Verify yet. Download the HTML verification file and complete next steps before clicking Verify.
Update source URL with HTML verification
In this step, you use the Sumo Logic Collector Management API to update the HTTP source with information from the HTML verification file that you downloaded from Webmaster Central.
Click on the information icon for the source to display the API usage information popup.
Copy the Source API URL field.
Using the Source API URL, make a GET request like this:
curl -v -u "accessid:accesskey" -X GET https://yourendpoint/api/v1/collectors/collectorID/sources/sourceID >source.json
You will see a response containing your etag:
Make note of the etag value in the response. You will need it.
Edit the JSON file to insert the
thirdPartyRefJSON block below, replacing the
namefield with the filename of the Google HTML Verification File, and the
contentsfield with the full string from the body of the HTML Verification File.
"name":"<google-validation-html-filename, for example google123a456b78c90d12.html>",
"content": "<google-validation-html-file-content, for example google-site-verification:google123a456b78c90d12.html >"
- Make an Update request with the Source JSON file using the Collector API with the following CURL command. You must add the etag value in the example below.
curl -u "accessid:accesskey" -X PUT -H "Content-Type: application/json" -H "If-Match: \"etag\"" -T source.json https://api.sumologic.com/api/v1/collectors/collectorID/sources/sourceID
- After the JSON file is uploaded, return to Google, and click Verify on the page as described in step 6 of the Add the source URL as an allowed domain in GCP section above. It should verify successfully.
Configure a Pub/Sub topic for GCP projects
In this step, you configure a Pub/Sub topic in GCP and add a subscription to the above source URL. Once you configure the Pub/Sub, you can export data from Stackdriver to the Pub/Sub. For example, you can export Google App Engine logs, as described on Collect Logs for Google App Engine.
- In GCP, select Pub/Sub in the left navigation pane.
- In the Pub/Sub pane, select Topics, then click Create Topic in the Topics pane.
Name the topic and click Create.
Select the new topic in the Topics pane, and select New subscription from the options menu.
- In the Create a subscription pane:
- Subscription Name. Enter a name for the subscription.
- Delivery Type. Choose “Push into an endpoint url”, and enter the upload URL for the Sumo HTTP source you created above.
- Click Create.