Azure Storage

Azure Storage is Microsoft's cloud storage solution for modern data storage scenarios offering highly available, massively scalable, durable, and secure storage for a variety of data objects in the cloud. This integration helps in monitoring the transaction volume and read/write activity of all your storage accounts.

In Azure Storage, storage accounts allow you to create and manage the following storage services:

• Blob storage stores any type of text or binary data, such as a document, media file, or application installer. You can set Blob storage for private access or share contents publicly to the Internet. Blob storage serves the same purpose as both AWS S3 and EBS.
• Table storage. Stores structured datasets. Table storage is a NoSQL key-attribute data store that allows for rapid development and fast access to large quantities of data. Similar to AWS SimpleDB and DynamoDB services.
• Queue storage. Provides messaging for workflow processing and for communication between components of cloud services.
• File storage. Offers shared storage for legacy applications using the standard Server Message Block (SMB) protocol. File storage is used in a similar manner to EFS in the AWS platform.

Log and metric types

For Azure Storage, you can collect the following logs and metrics:

Resource logs, which provide an insight into operations that were performed within an Azure resource. For a complete schema for resource logs refer to the below documentation:

• Azure Blob Storage schema
• Azure File Storage schema
• Azure Queue Storage schema
• Azure File Storage schema

Requests made by the Blob storage service itself, such as log creation or deletion, aren't logged. For a full list of the logged data, see Storage logged operations and status messages.

Metrics for Azure Storage are in below namespaces:

• Microsoft.Storage/storageAccounts
• Microsoft.Storage/storageAccounts/blobServices
• Microsoft.Storage/storageAccounts/fileServices
  • Microsoft.Storage/storageAccounts/queueServices
  • Microsoft.Storage/storageAccounts/tableServices

Click on the above namespaces to learn more about the supported metrics. For a complete list of the dimensions that Azure Storage supports, refer to the below documentation.

• Azure Blob Storage Metrics dimensions
• Azure File Storage Metrics dimensions
• Azure Queue Storage Metrics dimensions
• Azure Table Storage Metrics dimensions

info

Capacity metrics are currently not supported via Diagnostic Settings.

Setup

Azure service sends monitoring data to Azure Monitor, which can then stream data to Eventhub. Sumo Logic supports:

• Logs collection from Azure Monitor using our Azure Event Hubs source.
• Metrics collection using our HTTP Logs and Metrics source via Azure Functions deployed using the ARM template.

You must explicitly enable diagnostic settings for each storage service (blob,queue,table and file) and each storage account that you want to monitor. You can forward logs to the same event hub provided they satisfy the limitations and permissions as described here.

When you configure the event hubs source or HTTP source, plan your source category to ease the querying process. A hierarchical approach allows you to make use of wildcards. For example: Azure/Storage/Logs, Azure/Storage/Metrics.

Metrics and logs in Azure Monitor support only Azure Resource Manager storage accounts. Azure Monitor doesn't support classic storage accounts. If you want to use metrics or logs on a classic storage account, you need to migrate to an Azure Resource Manager storage account. For more information, see Migrate to Azure Resource Manager.

Configure metrics collection

In this section, you will configure a pipeline for shipping metrics from Azure Monitor to an Event Hub, on to an Azure Function, and finally to an HTTP Source on a hosted collector in Sumo Logic.

1. Configure an HTTP Source.
2. Configure and deploy the ARM Template.
3. Export metrics to Event Hub. Perform below steps for each storage service (blob,queue,table and file) and each storage account that you want to monitor.
   • Choose Stream to an event hub as destination.
   • Select Transaction.
   • Use the Event hub namespace created by the ARM template in Step 2 above. You can create a new Event hub or use the one created by ARM template. You can use the default policy RootManageSharedAccessKey as the policy name.

Configure logs collection

In this section, you will configure a pipeline for shipping diagnostic logs from Azure Monitor to an Event Hub.

1. To set up the Azure Event Hubs cloud-to-cloud source in Sumo Logic portal, refer to our Azure Event Hubs source documentation.
2. To create the Diagnostic settings in Azure portal, refer to the Azure documentation. Perform below steps for each storage service (blob,queue,table and file) and each storage account that you want to monitor.
   • Choose Stream to an event hub as the destination.
   • Select allLogs.
   • Use the Event hub namespace and Event hub name configured in previous step in destination details section. You can use the default policy RootManageSharedAccessKey as the policy name.

Troubleshooting

Azure Event Hubs Source

Common error types are described here.

You can try restarting the source for ThirdPartyConfig errors.

HTTP Logs and Metrics Source used by Azure Functions

To troubleshoot metrics collection, follow the instructions in Collect Metrics from Azure Monitor > Troubleshooting metrics collection.