Introduction to Monitoring Azure OpenAI Service
Monitoring data Azure OpenAI collects monitoring data as: Activity log Alerts Metrics Diagnostic settings Insights Collection and routing Platform metrics and the Activity log are collected automatically Can be routed to other locations by using a diagnostic setting Configure diagnostic settings in the Azure portal Implement logging and monitoring for Azure OpenAI models - Architecture Diagram that shows an architecture that provides monitoring and logging for Azure OpenAI. Client applications access Azure OpenAI endpoints to perform text generation (completions) and model training (fine-tuning). Azure Application Gateway provides a single point of entry to Azure OpenAI models and provides load balancing for APIs. Azure API Management enables security controls and auditing and monitoring of the Azure OpenAI models. a. In API Management, enhanced-security access is granted via Azure Active Directory (Azure AD) groups with subscription-based access permissions. b. Auditing is enabled for all interactions with the models via Azure Monitor request logging. c. Monitoring provides detailed Azure OpenAI model usage KPIs and metrics, including prompt information and token statistics for usage traceability. API Management connects to all Azure resources via Azure Private Link. This configuration provides enhanced security for all traffic via private endpoints and contains traffic in the private network. Multiple Azure OpenAI instances enable scale-out of API usage to ensure high availability and disaster recovery for the service. More details : Monitoring Azure OpenAI Service - Azure AI services | Microsoft LearnIntroduction to Planning and Managing Costs of Azure OpenAI
& Manage Costs of Azure OpenAI Service Planning to manage costs for Azure OpenAI Service Estimate costs Understand the full billing model Other costs that might accrue with Azure OpenAI Service Using Azure Prepayment with Azure OpenAI Service Monitor costs Monitor costs from Cost analysis under Resource Management. To view Azure OpenAI costs in cost analysis: Sign in to the Azure portal. Select one of your Azure OpenAI resources. Under Resource Management select Cost analysis By default cost analysis is scoped to the individual Azure OpenAI resource. Screenshot of cost analysis dashboard scoped to an Azure OpenAI resource. Creating budgets Create alerts that notifies of overspending risks. Budgets can be created with filters for specific resources or services in Azure if you want more granularity present in your monitoring. Filters help ensure that you don't accidentally create new resources that cost you additional money. For more information about the filter options available when you create a budget, see Group and filter options.
