openai token usage log exports
Hi, I am trying to find a way to transfer the model token usage data to BigQuery, here are my attampts: 1. OpenAI model- deployment - metrics. It provides token usage directly, but I failed to find a way to export it on a scheduled basis. 2. API management service. Not working when using streaming responses with Azure OpenAI models because it requires more configuration to capture model completions. As I am developing a chatbot, the streaming feature is necessary. 3. Export native logs by diagnostic settings. Unfortunately there is no token usage related info included. So I wanna kindly ask for a potential solution for exporting the token usage to BigQuery with streaming function when using openai model. Thanks!Introduction to Monitoring Azure OpenAI Service
Monitoring data Azure OpenAI collects monitoring data as: Activity log Alerts Metrics Diagnostic settings Insights Collection and routing Platform metrics and the Activity log are collected automatically Can be routed to other locations by using a diagnostic setting Configure diagnostic settings in the Azure portal Implement logging and monitoring for Azure OpenAI models - Architecture Diagram that shows an architecture that provides monitoring and logging for Azure OpenAI. Client applications access Azure OpenAI endpoints to perform text generation (completions) and model training (fine-tuning). Azure Application Gateway provides a single point of entry to Azure OpenAI models and provides load balancing for APIs. Azure API Management enables security controls and auditing and monitoring of the Azure OpenAI models. a. In API Management, enhanced-security access is granted via Azure Active Directory (Azure AD) groups with subscription-based access permissions. b. Auditing is enabled for all interactions with the models via Azure Monitor request logging. c. Monitoring provides detailed Azure OpenAI model usage KPIs and metrics, including prompt information and token statistics for usage traceability. API Management connects to all Azure resources via Azure Private Link. This configuration provides enhanced security for all traffic via private endpoints and contains traffic in the private network. Multiple Azure OpenAI instances enable scale-out of API usage to ensure high availability and disaster recovery for the service. More details : https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/monitoring#monitoring-data
