This page provides answers for frequently asked integration questions about Azure Event Hub.
What log sources are supported by Event Hub?
Exporting data from Azure Monitor: Includes the majority of services, including Network Watcher, SQL DB, EventHubs, Cosmos DB, Data Factory, KeyVault, and Stream Analytics export their logs and metrics to Azure Monitor.
What if there is an existing storage account or Event Hub?
- The storage account creation limit per region, per subscription, is 200.
- The limit for the number of Event Hub per namespace is 10.
- You need to update ConnectionStrings (StorageConnectionString for storage account and AzureEventHubConnectionString for Event Hub) and remove the corresponding resource from the template.
How is the function scaled?
To increase the number of parallel instances: the number of messages are ordered in one partition. By default, messages are distributed in round robin manner. Each Function instance is backed by 1 EventProcessorHost (EPH). EventHub only allows 1 EPH to hold a lease on a partition, but >1 partition can be assigned an EPH. This means the number of EPH <= number of partitions in EventHub. Hence, increasing number of partitions in Event Hub(default maxlimit is 32. You have to raise a support request to increasing it, and this will increase the consumption rate. For more information, see the Azure Trigger Scaling document.
To increase the number of fetched messages per function: increase the maxBatchSize(The maximum event count received per receive loop) property in host.json(default is 64) For more information, see the Azure Trigger - host.json properties document.
How much does it cost?
For specific details, see the following Azure pricing pages:
Function executes 3 million times during the month with memory consumption of 512 MB, execution duration 1 sec and 64KB is the message size.
Resource consumption: (512 MB / 1,024(in GB)) * 1sec * 3 million = 1.5 million GB-sec - 4,00,000 GB-s(free plan) = 1.1 million GB-sec => x 0.000016$/GB-s = 17.6$
Execution consumption:3 million - 1 million(free plan) = 2 million => x 0.20$ = 0.40$
Cost of Azure Function : 18$
Event Hub Example:
Ingress events(billed in multiples of 64 KB): 3 million * $0.028 per million events = 0.084$
Throughput units(1TU = 1MB/sec ingress events + 2MB/sec egress events + 84GB event storage aggregated across all eventhubs in a namespace): $0.015/hour * 30*24 = 10.8$
Cost of Event Hub = 10.88$
Storage Account Example:
Assuming General purpose v2 storage accounts in hot tier and worst case scenario of storing all event in storage with 1 message processed per receive loop by azure function
Storage prices: $0.0208 per GB => x (64KB*3 million)*/(1024*1024) Gb per month = 3.66$
Operation prices write operation $0.05 per 10000 requests => * 3 million = 15$
Cost of Storage Accounts = 18.66$
Total = 47.5$
How do I troubleshoot an Event Hub Integration?
Where can I find historic logs for the Azure function?
- Install Azure Storage Explorer.
- Log in using your Azure Account credentials.
Built-in logging uses the storage account specified by the connection string in the AzureWebJobsDashboard app setting.
- Go to functionname > Tables > AzureWebJobsHostLogs. The ones with the "I" partition key are invocation logs, as shown in the previous example.
What can I do if a function is timing out?
The default timeout for functions on a Consumption plan is 5 minutes. You can increase the value to 10 minutes for the Function App by changing the functionTimeout property in the host.json file.
How do I enable Application Insights?
Follow the instructions in the Microsoft Azure documentation for Enabling Application Insights integration.
How can I debug using print statements?
- Under the Function app edit mode section, change the edit mode in Function App Settings. You can now edit the function.
- Enter context.log(“debug statements”) and save.
- Run using sample logs, as described in Troubleshooting log collection.
How do I deploy the ARM template in Azure Gov cloud?
Use this template which has been customized for Azure Gov cloud to deploy the resources.
How do I ensure that the Event Hub is receiving log messages?
If events are not getting into the Event Hub, the event grid subscription publisher settings are not configured properly.
- Resources should be in the same region. Resource '/subscriptions/c088dc46-d692-42ad-a4b6-9a542d28ad2a/resourceGroups/AzureAuditEventHub/providers/Microsoft.Network/networkSecurityGroups/testhimvm-nsg' is in region 'eastus' and resource '/subscriptions/c088dc46-d692-42ad-a4b6-9a542d28ad2a/resourcegroups/testresourcegroup/providers/microsoft.eventhub/namespaces/sumoazureaudittf7grgv4prygw' is in region 'westus'.
This happens while exporting logs or metrics from Azure monitor to Event Hub. The service generating the logs and Event Hub should be deployed in the same region.
- Create or update activity logprofilesfailure. If you get this error message in Azure when setting up an Event Hub Export, do the following:
- Search for Subscriptions in all services.
- Select your subscription > Resource Providers.
- Search for and Enable microsoft.insights.