FinOps for AI Workloads on Azure: Tokens, PTU and Pay-As-You-Go

Originally published on Substack: https://gisli78.substack.com/p/finops-for-ai-workloads-on-azure

Part 3 of my Azure AI landing zone series is about FinOps for AI workloads. Tokens are the AI bill most teams actually get, so the post covers:

• Picking the deployment type first: Global Standard, Data Zone, Batch and Flex (preview)
• Why quota isn't billing, and why max_tokens eats your TPM
• PTU vs pay-as-you-go, and when hourly PTU actually pays off
• Sizing PTU with Microsoft's formula, and how prompt caching helps
• Spillover: PTU for the baseline, pay-per-token for the peaks
• Showback per team with llm-token-limit and llm-emit-token-metric in API Management
• Cost Management: group by meter, tag your deployments, and remember budgets only alert

Read the full article on Substack: https://gisli78.substack.com/p/finops-for-ai-workloads-on-azure

Bicep sample: https://github.com/gislig/azure-ai-landing-zone/tree/main/examples/ai-finops