Forum Discussion
Introduction to Monitoring Azure OpenAI Service
Monitoring Azure OpenAI : Generated using Microsoft Designer
Monitoring data
Azure OpenAI collects monitoring data as:
Activity log
Alerts
Metrics
Diagnostic settings
Insights
Collection and routing
- Platform metrics and the Activity log are collected automatically
- Can be routed to other locations by using a diagnostic setting
- Configure diagnostic settings in the Azure portal
Implement logging and monitoring for Azure OpenAI models - Architecture
Diagram that shows an architecture that provides monitoring and logging for Azure OpenAI.
- Client applications access Azure OpenAI endpoints to perform text generation (completions) and model training (fine-tuning).
Azure Application Gateway provides a single point of entry to Azure OpenAI models and provides load balancing for APIs.
Azure API Management enables security controls and auditing and monitoring of the Azure OpenAI models.
a. In API Management, enhanced-security access is granted via Azure Active Directory (Azure AD) groups with subscription-based access permissions.
b. Auditing is enabled for all interactions with the models via Azure Monitor request logging.
c. Monitoring provides detailed Azure OpenAI model usage KPIs and metrics, including prompt information and token statistics for usage traceability.API Management connects to all Azure resources via Azure Private Link. This configuration provides enhanced security for all traffic via private endpoints and contains traffic in the private network.
Multiple Azure OpenAI instances enable scale-out of API usage to ensure high availability and disaster recovery for the service.
More details : Monitoring Azure OpenAI Service - Azure AI services | Microsoft Learn
