Skip to main content

Azure Stream Analytics

Thumbnail icon

Azure Stream Analytics is a fully managed stream processing engine that is designed to analyze and process large volumes of streaming data with sub-millisecond latencies. This integration helps in monitoring and troubleshooting your query and job performance.

Log and metric types​

For Azure Stream Analytics, you can collect the following logs and metrics:

  • Resource logs. Azure Stream Analytics captures below two categories of resource logs:
    • Authoring. These log events that are related to job authoring operations, such as job creation, and starting or stopping the job.
    • Execution. These events are related to job execution such as connectivity and data processing errors.

To learn more about the different resource log category types and schemas collected for Azure Stream Analytics, refer to the Azure documentation.

Setup​

Azure service sends monitoring data to Azure Monitor, which can then stream data to Eventhub. Sumo Logic supports:

You must explicitly enable diagnostic settings for each Azure Stream Analytics job you want to monitor. You can forward logs to the same event hub provided they satisfy the limitations and permissions as described here.

When you configure the event hubs source or HTTP source, plan your source category to ease the querying process. A hierarchical approach allows you to make use of wildcards. For example: Azure/StreamAnalytics/Logs, Azure/StreamAnalytics/Metrics.

Configure metrics collection​

In this section, you will configure a pipeline for shipping metrics from Azure Monitor to an Event Hub, on to an Azure Function, and finally to an HTTP Source on a hosted collector in Sumo Logic.

  1. Configure an HTTP Source.
  2. Configure and deploy the ARM Template.
  3. Export metrics to Event Hub. Perform below steps for each Azure Stream Analytics job that you want to monitor.
    • Choose Stream to an event hub as destination.
    • Select AllMetrics.
    • Use the Event hub namespace created by the ARM template in Step 2 above. You can create a new Event hub or use the one created by ARM template. You can use the default policy RootManageSharedAccessKey as the policy name.

Configure logs collection​

In this section, you will configure a pipeline for shipping diagnostic logs from Azure Monitor to an Event Hub.

  1. To set up the Azure Event Hubs source in Sumo Logic, refer to Azure Event Hubs Source for Logs.
  2. To create the Diagnostic settings in Azure portal, refer to the Azure documentation. Perform below steps for each Azure Stream Analytics job that you want to monitor.
    • Choose Stream to an event hub as the destination.
    • Select Execution and Authoring.
    • Use the Event hub namespace and Event hub name configured in previous step in destination details section. You can use the default policy RootManageSharedAccessKey as the policy name.

Troubleshooting​

HTTP Logs and Metrics Source used by Azure Functions​

To troubleshoot metrics collection, follow the instructions in Collect Metrics from Azure Monitor > Troubleshooting metrics collection.

Status
Legal
Privacy Statement
Terms of Use

Copyright © 2024 by Sumo Logic, Inc.