FinOps for AI Workloads on Azure: Tokens, PTU and Pay-As-You-Go
Originally published on Substack: https://gisli78.substack.com/p/finops-for-ai-workloads-on-azure Part 3 of my Azure AI landing zone series is about FinOps for AI workloads. Tokens are the AI bill most teams actually get, so the post covers: Picking the deployment type first: Global Standard, Data Zone, Batch and Flex (preview) Why quota isn't billing, and why max_tokens eats your TPM PTU vs pay-as-you-go, and when hourly PTU actually pays off Sizing PTU with Microsoft's formula, and how prompt caching helps Spillover: PTU for the baseline, pay-per-token for the peaks Showback per team with llm-token-limit and llm-emit-token-metric in API Management Cost Management: group by meter, tag your deployments, and remember budgets only alert Read the full article on Substack: https://gisli78.substack.com/p/finops-for-ai-workloads-on-azure Bicep sample: https://github.com/gislig/azure-ai-landing-zone/tree/main/examples/ai-finopsA secure Azure landing zone for AI: private endpoints, managed identity, APIM token limits (Bicep)
I wrote up a reference pattern for running Azure AI Foundry workloads inside a secure landing zone: Foundry (AIServices) behind a private endpoint with the three required private DNS zones, local auth disabled in favour of managed identity, and Azure API Management in front using llm-token-limit and llm-emit-token-metric to control and measure token usage. Article: https://gisli78.substack.com/p/a-secure-azure-landing-zone-for-ai Code (Bicep, with automated tests): https://github.com/gislig/azure-ai-landing-zone Feedback welcome, especially from anyone running this pattern in production. #mvpbuzz #azure #azug #microsoftHow to automate the deployment of an Elastic Job Agent with a Private Endpoint via Bicep
Environment Overview Recently, I worked on a case where our customer encountered an issue during the deployment of an Elastic Job Agent (EJA) and a Private Endpoint (PEP) using a Bicep deployment script. The process resulted in an endless deployment loop. In this article, I will guide you step by step through creating an EJA with a PEP and demonstrate how to automatically approve the connection using a Bicep deployment script. Technical Issue The main challenge is that the EJA PEP is fully managed by Microsoft, which means you cannot specify a name during the script authorization process—the system assigns one randomly. To address this, we will use deploymentScripts in combination with PowerShell to monitor and authorize the process automatically. For more details, please refer to the following resources: References Use deployment scripts in Bicep https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/deployment-script-bicep?tabs=azure-cli Elastic jobs in Azure SQL Database https://learn.microsoft.com/en-us/azure/azure-sql/database/elastic-jobs-overview?view=azuresql Elastic job private endpoints https://learn.microsoft.com/en-us/azure/azure-sql/database/elastic-jobs-overview?view=azuresql#elastic-job-private-endpoints The Idea Since it's not possible to specify the EJA PEP name as a variable for direct approval—unlike standard PEPs—the approach was to intercept the auto-generated PEP by its ID or name and then authorize it automatically. (Remember to always follow the Principle of Least Privilege (PoLP)); in my example, I used Contributor permissions for simplicity. ****************************************************************************************************************************************************************************************************** /*Disclaimer: ****************************************************************************************************************************************************************************************************** This script is not supported under any Microsoft standard support program or service. This script is provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the script and documentation remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the script be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the current script or documentation, even if Microsoft has been advised of the possibility of such damages. I’m not a programmer or a Bicep expert. I simply put together what I was able to reason through in order to meet our client’s request. Any suggestions or modifications are welcome. ****************************************************************************************************************************************************************************************************** This article was written in August 2025; code and modules may change over time. ****************************************************************************************************************************************************************************************************** */ param prefix string = 'bicep' param environment string = 'test' param sqlServerName string = 'bicep-test-sqleja-sqlsrv' param location string = 'westeurope' param forceUpdateTag string = utcNow() //let's check that the previously manually created server is present resource sqlServer 'Microsoft.Sql/servers@2021-11-01' existing = { name: sqlServerName } //create the database resource sqlDb 'Microsoft.Sql/servers/databases@2021-11-01' = { parent: sqlServer name: '${prefix}-${environment}-sqleja-sqldb' location: location properties: { requestedBackupStorageRedundancy: 'Local' maxSizeBytes: 5368709120 } sku: { name: 'S3' } } //create the Job Agent resource sqlJob 'Microsoft.Sql/servers/jobAgents@2024-05-01-preview' = { parent: sqlServer location: location name: '${prefix}-${environment}-sqleja-sqljobagent' properties: { databaseId: sqlDb.id } sku: { name: 'JA100' } } //defines a Target Group within a Job Agent resource targetGroup 'Microsoft.Sql/servers/jobAgents/targetGroups@2024-05-01-preview' = { parent: sqlJob name: '${prefix}-${environment}-sqleja-targetgroup' properties: { members: [ { membershipType: 'Include' serverName: sqlServer.name type: 'SqlServer' } ] } } //generate an UMI to auth and deploy the PS script resource scriptIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = { name: '${prefix}-${environment}-deployscript-umi' location: location } //assign the role to the script UMI //https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles resource scriptRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = { name: guid(scriptIdentity.id, 'script-role-assignment') properties: { roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'b24988ac-6180-42a0-ab88-20f7382dd24c') // Contributor principalId: scriptIdentity.properties.principalId } dependsOn: [ delayScript ] } //let's generate a delay to allow Azure AD permissions propagation resource delayScript 'Microsoft.Resources/deploymentScripts@2023-08-01' = { name: 'sleep-60s-bash' location: location kind: 'AzureCLI' identity: { type: 'UserAssigned' userAssignedIdentities: { '${scriptIdentity.id}': {} } } properties: { azCliVersion: '2.76.0' scriptContent: ''' echo "Sleeping for 60 seconds..." sleep 60 echo "Done sleeping." ''' retentionInterval: 'P1D' cleanupPreference: 'Always' timeout: 'PT5M' forceUpdateTag: forceUpdateTag } dependsOn: [ scriptIdentity ] } //create the eja pep resource privateEndpoint 'Microsoft.Sql/servers/jobAgents/privateEndpoints@2024-05-01-preview' = { parent: sqlJob name: '${prefix}-${environment}-sqleja-pep' properties: { targetServerAzureResourceId: sqlServer.id } } //PS script to approve the MS fully managed PEP resource approvePrivateEndpointConnection 'Microsoft.Resources/deploymentScripts@2023-08-01' = { name: 'ps-to-approve-eja-pep-by-script' location: location kind: 'AzurePowerShell' identity: { type: 'UserAssigned' userAssignedIdentities: { '${scriptIdentity.id}': {} } } properties: { azPowerShellVersion: '11.0' scriptContent: ''' Connect-AzAccount -Identity $SqlServerResourceIdForPrivateLink = '/subscriptions/<yourSubID>/resourceGroups/bicep-rg/providers/Microsoft.Sql/servers/bicep-test-sqleja-sqlsrv' $SQLprivateEndpoints = Get-AzPrivateEndpointConnection -PrivateLinkResourceId $SqlServerResourceIdForPrivateLink foreach ($privateEndpoint in $SQLprivateEndpoints) { if ($privateEndpoint.PrivateLinkServiceConnectionState.Status -eq "Pending") { $id = $privateEndpoint.id Write-Output "Approving Private Endpoint Connection: $id" Approve-AzPrivateEndpointConnection -ResourceId "${id}" -Description "Approved by Bicep Deployment Script" } } ''' retentionInterval: 'P1D' cleanupPreference: 'Always' timeout: 'PT30M' forceUpdateTag: forceUpdateTag } dependsOn: [ sqlJob scriptRoleAssignment ] } Deployment Details Below is an overview of the resources created by the automation Conclusion With this article, I’ve wanted to share how to automate the creation of an Elastic Job Agent Private Endpoint without requiring manual approval. Kindly note that the products and options presented in this article are subject to change, this article reflects for Azure SQL Database in September 2025. Please also make sure to review the code carefully. It is provided as-is, with no guarantees, warranties, or support. Any suggestions or improvements are welcome. I hope you find this guide useful and that it inspires you to experiment freely with your deployments. Thank you. Best Regards, VittorioDealing with deployment blockers with Bicep
Hello Folks! Have you ever had a deployment blocked because the behavior you get from the portal is different than the deployment you get from an Azure Resource Manager (ARM) template or a Bicep deployment? Well, I had that issue for a little internal project I was working on. Let me tell you how I resolved it using built-in Bicep functionality.
Bicep create random string with length of x
Hi Trying to create a storage account and I run into problems that StorageAccountAlredyTaken. What is best practice of doing this. // getting paramerts from json file param storageName string //pseudo code 1. get length of storageName 2. add random string behind "storageName" and add a random length of x-characters so that the storage name get 24 characters ex storageName = rg-supercode-dev-westeurope-<random>(total of 24 characters length)
