Azure Batch runs large-scale parallel and high-performance computing (HPC) batch jobs efficiently in Azure. This integration helps in monitoring health of the nodes; creation, update, and deletion events of individual Batch resources, such as a pool or task; and tracking the state (start/fail/complete) of jobs and tasks.
Log and metric types
For Azure Batch, you can collect the following logs and metrics:
Service Logs. Events emitted by the Batch service during the lifetime of an individual resource such as a pool or task.
Batch account metrics. Metrics at the Batch account level.
Azure service sends monitoring data to Azure Monitor, which can then stream data to Eventhub. Sumo Logic supports:
Logs collection from Azure Monitor using our Azure Event Hubs source.
Metrics collection using our HTTP Logs and Metrics source via Azure Functions deployed using the ARM template.
You must explicitly enable diagnostic settings for each Batch account you want to monitor. You can forward logs to the same event hub provided they satisfy the limitations and permissions as described here.
When you configure the event hubs source or HTTP source, plan your source category to ease the querying process. A hierarchical approach allows you to make use of wildcards. For example: Azure/Batch/Logs
Configure metrics collection
In this section, you will configure a pipeline for shipping metrics from Azure Monitor to an Event Hub, on to an Azure Function, and finally to an HTTP Source on a hosted collector in Sumo Logic.
Configure an HTTP Source.
Configure and deploy the ARM Template.
Export metrics to Event Hub. Choose Stream to an event hub
as destination and selectallMetrics
. Use the Event hub namespace created by the ARM template in Step 2 above. You can create a new Event hub or use the one created by ARM template. You can use the default policyRootManageSharedAccessKey
as the policy name.
Configure logs collection
In this section, you will configure a pipeline for shipping diagnostic logs from Azure Monitor to an Event Hub.
To set up the Azure Event Hubs cloud-to-cloud source in Sumo Logic portal, refer to our Azure Event Hubs cloud-to-cloud source guide documentation.
To create the Diagnostic settings in Azure portal, refer to the Azure documentation.
Choose Stream to an event hub
as the destination.
Use the Event hub namespace and Event hub name configured in previous step in destination details section. You can use the default policy RootManageSharedAccessKey
as the policy name.
Azure Event Hubs Source
Common error types are described here.
You can try restarting the source for ThirdPartyConfig
HTTP Logs and Metrics Source used by Azure Functions
To troubleshoot metrics collection, follow the instructions in Collect Metrics from Azure Monitor > Troubleshooting metrics collection.