This blog was originally published in the Cost Management updates blog post for September 2024.
As AI adoption accelerates across industries, organizations are increasingly integrating these technologies into their core operations. With the growing reliance on AI, it has become essential for our customers to manage their AI spend. In the Cost Management updates for August, we discussed hourly pricing for Azure OpenAI provisioned throughput units (PTUs) and the introduction of 1-month and 1-year Azure OpenAI provisioned reservations. This blog post covers the tools we offer in Cost Management for you to analyze, monitor, and optimize your Azure OpenAI costs. Please note that the tools mentioned below are also applicable for other Azure services.
Analyze costs
We know Cost analysis is your go-to tool for getting insights into your costs. Customizable views in Cost analysis enable you to group and filter by multiple cost attributes. You can view costs grouped by tags, resource groups, locations, and more and use filters to focus on the desired attributes. The below screenshot shows the customizable views in Cost analysis.
Screenshot of customizable views in Cost analysis.
The good news is that you can use these existing views that most of you are already familiar with to analyze your Azure OpenAI costs as well by using the filters below:
- Service name = Cognitive Services.
- Service tier/Meter subcategory = “Azure Open AI” or “Azure Open AI Reservation”
You could also use the “Resource type = OpenAI” filter but the view wouldn’t include reservation purchases. The screenshot below shows the Accumulated costs view in Cost analysis using the filters mentioned above and grouped by meter. Grouping by meter allows you to see input/output costs for your different models for token-based deployments and PTU costs for PTU-based deployments for the selected scope.
Screenshot of the Accumulated costs view filtered to Azure Open AI in Cost analysis
Monitor costs
There are multiple ways to monitor Azure OpenAI costs to ensure that they do not get out of control and stay within allotted budgets. Two approaches available in Cost Management are scheduled emails and budgets.
Scheduled emails
Getting email updates for your costs is a great way to stay on top of them and analyze trends and anomalies. You can subscribe to automated emails for your private or shared views in Cost analysis using the “Subscribe” button on top of your desired view (as seen in the screenshot above), through the Scheduled Actions API, or with Scheduled Actions bicep modules from the FinOps toolkit. You can also add members of your team to receive these emails on a daily, weekly, or monthly basis.
Budgets
If you do not want to be surprised by your costs and keep your teams accountable for their spend, you must create budgets. With budgets you get alerts when the actual or forecasted costs exceed the threshold you have defined. You could create a budget for your Azure OpenAI costs using the filters mentioned above. Budgets also support calling action groups when the threshold is met, which allow you to take automated actions like calling webhooks, creating tickets, and sending push notifications to the Azure mobile app so that you never miss an alert! Setting up budgets is really easy and can save you a lot of hassle explaining cost overruns. You can read more about budget creation in our documentation, which covers both the Cost analysis and API experiences.
If you prefer a video tutorial, discover how to create Azure Budgets.
Optimize costs
Your cost optimization journey starts with identifying the correct pricing model for your Azure OpenAI deployments. You have two models to choose from, provisioned throughput units (PTUs) and standard token-based deployments. You can use the pricing calculator to estimate costs for these models based on your predicted usage.
For PTU deployments, Azure OpenAI Service PTU reservations can help you save significant costs. You have the flexibility to go with a 1-month or a 1-year commitment. While purchasing reservations to optimize your costs is the first step in the right direction, it is equally important for you to monitor the utilization of these reservations to avoid any wastage. In Cost management, you can use the Reservations + Hybrid benefit page to monitor the utilization of all your reservations. As seen in the screenshot below, it can be found under Optimization on the left-hand side menu in Cost Management.
Screenshot of the Optimization section of the Cost Management menu in the Azure portal
You can also create reservation utilization alerts to be proactively alerted when the utilization is below your configured threshold to maximize your benefits.
We hope you can leverage all these tools to manage your spending on Azure OpenAI Service, and as I mentioned above, these are applicable for all Azure services. We encourage you to leave feedback in the Cost Management experience to let us know what you'd like to see next. We're eager to hear about your experiences and how we can improve your experience.
Microsoft
