How to fix outbound IPs for App Service
App Service is one of Azure Compute services categorized as PaaS Platform and has been offered since early days of Microsoft Azure. App Service is continuously improved since it had been released. Now, App Service offers container orchestrator for both Windows and Linux, VNET integration, and many other useful features to leverage Azure platform. On the other hand, PaaS services are offered as multi-tenant shared services, which might pose challenges for integration with external services or legacy applications. As one of such challenges, external services sometimes require to fix IP address of an application. In this article, we will configure to fix outbound IPs of your App Service.Develop Custom Engine Agent to Microsoft 365 Copilot Chat with pro-code
There are some great articles that explain how to integrate an MCP server built on Azure with a declarative agent created using Microsoft Copilot Studio. These approaches aim to extend the agent’s capabilities by supplying it with tools, rather than defining a fixed role. Here were some of the challenges w encountered: The agent's behavior can only be tested through the Copilot Studio web interface, which isn't ideal for iterative development. You don’t have control over which LLM is used as the orchestrator—for example, there's no way to specify GPT-4o. The agent's responses don’t always behave the same as they would if you were prompting the LLM directly. These limitations got me thinking: why not build the entire agent myself? At the same time, I still wanted to take advantage of the familiar Microsoft 365 Copilot interface on the frontend. As I explored further, I discovered that the Microsoft 365 Copilot SDK makes it possible to bring in your own custom-built agent.Calculating Chargebacks for Business Units/Projects Utilizing a Shared Azure OpenAI Instance
Azure OpenAI Service is at the forefront of technological innovation, offering REST API access to OpenAI's suite of revolutionary language models, including GPT-4, GPT-35-Turbo, and the Embeddings model series. Enhancing Throughput for Scale As enterprises seek to deploy OpenAI's powerful language models across various business units, they often require granular control over configuration and performance metrics. To address this need, Azure OpenAI Service is introducing dedicated throughput, a feature that provides a dedicated connection to OpenAI models with guaranteed performance levels. Throughput is quantified in terms of tokens per second (tokens/sec), allowing organizations to precisely measure and optimize the performance for both prompts and completions. The model of provisioned throughput provides enhanced management and adaptability for varying workloads, guaranteeing system readiness for spikes in demand. This capability also ensures a uniform user experience and steady performance for applications that require real-time responses. Resource Sharing and Chargeback Mechanisms Large organizations frequently provision a singular instance of Azure OpenAI Service that is shared across multiple internal departments. This shared use necessitates an efficient mechanism for allocating costs to each business unit or consumer, based on the number of tokens consumed. This article delves into how chargeback is calculated for each business unit based on their token usage. Leveraging Azure API Management Policies for Token Tracking Azure API Management Policies offer a powerful solution for monitoring and logging the token consumption for each internal application. The process can be summarized in the following steps: ** Sample Code: Refer to this GitHub repository to get a step-by-step instruction on how to build the solution outlined below : private-openai-with-apim-for-chargeback 1. Client Applications Authorizes to API Management To make sure only legitimate clients can call the Azure OpenAI APIs, each client must first authenticate against Azure Active Directory and call APIM endpoint. In this scenario, the API Management service acts on behalf of the backend API, and the calling application requests access to the API Management instance. The scope of the access token is between the calling application and the API Management gateway. In API Management, configure a policy (validate-jwt or validate-azure-ad-token) to validate the token before the gateway passes the request to the backend. 2. APIM redirects the request to OpenAI service via private endpoint. Upon successful verification of the token, Azure API Management (APIM) routes the request to Azure OpenAI service to fetch response for completions endpoint, which also includes prompt and completion token counts. 3. Capture and log API response to Event Hub Leveraging the log-to-eventhub policy to capture outgoing responses for logging or analytics purposes. To use this policy, a logger needs to be configured in the API Management: # API Management service-specific details $apimServiceName = "apim-hello-world" $resourceGroupName = "myResourceGroup" # Create logger $context = New-AzApiManagementContext -ResourceGroupName $resourceGroupName -ServiceName $apimServiceName New-AzApiManagementLogger -Context $context -LoggerId "OpenAiChargeBackLogger" -Name "ApimEventHub" -ConnectionString "Endpoint=sb://<EventHubsNamespace>.servicebus.windows.net/;SharedAccessKeyName=<KeyName>;SharedAccessKey=<key>" -Description "Event hub logger with connection string" Within outbound policies section, pull specific data from the body of the response and send this information to the previously configured EventHub instance. This is not just a simple logging exercise; it is an entry point into a whole ecosystem of real-time analytics and monitoring capabilities: <outbound> <choose> <when condition="@(context.Response.StatusCode == 200)"> <log-to-eventhub logger-id="TokenUsageLogger">@{ var responseBody = context.Response.Body?.As<JObject>(true); return new JObject( new JProperty("Timestamp", DateTime.UtcNow.ToString()), new JProperty("ApiOperation", responseBody["object"].ToString()), new JProperty("AppKey", context.Request.Headers.GetValueOrDefault("Ocp-Apim-Subscription-Key",string.Empty)), new JProperty("PromptTokens", responseBody["usage"]["prompt_tokens"].ToString()), new JProperty("CompletionTokens", responseBody["usage"]["completion_tokens"].ToString()), new JProperty("TotalTokens", responseBody["usage"]["total_tokens"].ToString()) ).ToString(); }</log-to-eventhub> </when> </choose> <base /> </outbound> EventHub serves as a powerful fulcrum, offering seamless integration with a wide array of Azure and Microsoft services. For example, the logged data can be directly streamed to Azure Stream Analytics for real-time analytics or to Power BI for real-time dashboards With Azure Event Grid, the same data can also be used to trigger workflows or automate tasks based on specific conditions met in the incoming responses. Moreover, the architecture is extensible to non-Microsoft services as well. Event Hubs can interact smoothly with external platforms like Apache Spark, allowing you to perform data transformations or feed machine learning models. 4: Data Processing with Azure Functions An Azure Function is invoked when data is sent to the EventHub instance, allowing for bespoke data processing in line with your organization’s unique requirements. For instance, this could range from dispatching the data to Azure Monitor, streaming it to Power BI dashboards, or even sending detailed consumption reports via Azure Communication Service. [Function("TokenUsageFunction")] public async Task Run([EventHubTrigger("%EventHubName%", Connection = "EventHubConnection")] string[] openAiTokenResponse) { //Eventhub Messages arrive as an array foreach (var tokenData in openAiTokenResponse) { try { _logger.LogInformation($"Azure OpenAI Tokens Data Received: {tokenData}"); var OpenAiToken = JsonSerializer.Deserialize<OpenAiToken>(tokenData); if (OpenAiToken == null) { _logger.LogError($"Invalid OpenAi Api Token Response Received. Skipping."); continue; } _telemetryClient.TrackEvent("Azure OpenAI Tokens", OpenAiToken.ToDictionary()); } catch (Exception e) { _logger.LogError($"Error occured when processing TokenData: {tokenData}", e.Message); } } } In the example above, Azure function processes the tokens response data in Event Hub and sends them to Application Insights telemetry, and a basic Dashboard is configured in Azure, displaying the token consumption for each client application. This information can conveniently be used to compute chargeback costs. A sample query used in dashboard above that fetches tokens consumed by a specific client: customEvents | where name contains "Azure OpenAI Tokens" | extend tokenData = parse_json(customDimensions) | where tokenData.AppKey contains "your-client-key" | project Timestamp = tokenData.Timestamp, Stream = tokenData.Stream, ApiOperation = tokenData.ApiOperation, PromptTokens = tokenData.PromptTokens, CompletionTokens = tokenData.CompletionTokens, TotalTokens = tokenData.TotalTokens Azure OpenAI Landing Zone reference architecture A crucial detail to ensure the effectiveness of this approach is to secure the Azure OpenAI service by implementing Private Endpoints and using Managed Identities for App Service to authorize access to Azure AI services. This will limit access so that only the App Service can communicate with the Azure OpenAI service. Failing to do this would render the solution ineffective, as individuals could bypass the APIM/App Service and directly access the OpenAI Service if they get hold of the access key for OpenAI. Refer to Azure OpenAI Landing Zone reference architecture to build a secure and scalable AI environment. Additional Considerations If the client application is external, consider using an Application Gateway in front of the Azure APIM If "streaming" is set to true, tokens count is not returned in response. In that that case libraries like tiktoken (Python), orgpt-3-encoder(javascript) for most GPT-3 models can be used to programmatically calculate tokens count for the user prompt and completion response. A useful guideline to remember is that in typical English text, one token is approximately equal to around 4 characters. This equates to about three-quarters of a word, meaning that 100 tokens are roughly equivalent to 75 words. (P.S. Microsoft does not endorse or guarantee any third-party libraries.) A subscription key or a custom header like app-key can also be used to uniquely identify the client as appId in OAuth token is not very intuitive. Rate-limiting can be implemented for incoming requests using OAuth tokens or Subscription Keys, adding another layer of security and resource management. The solution can also be extended to redirect different clients to different Azure OpenAI instances. For example., some clients utilize an Azure OpenAI instance with default quotas, whereas premium clients get to consume Azure Open AI instance with dedicated throughput. Conclusion Azure OpenAI Service stands as an indispensable tool for organizations seeking to harness the immense power of language models. With the feature of provisioned throughput, clients can define their usage limits in throughput units and freely allocate these to the OpenAI model of their choice. However, the financial commitment can be significant and is dependent on factors like the chosen model's type, size, and utilization. An effective chargeback system offers several advantages, such as heightened accountability, transparent costing, and judicious use of resources within the organization.
App Service Linux container amid conversations.
What's communicating with what? What are the various IP addresses involved when App Service Linux container talks to a remote resource (e.g., Databases)? From the remote end of the connection, would I see a private IP or Public IP as the Linux Container source address?Announcing the General Availability of WordPress on Azure App Service
We are thrilled to announce that WordPress on Azure App Service, which was running on Public Preview since 15 February 2022, has been made Generally Available on 8 August 2022. To read the Public Preview Announcement read the blog post on The new and better ‘WordPress on App Service’ - Microsoft Tech Community.
App Service to Storage Account Connection Condition Summary
Currently, there are 4 main conditions the Azure App Service can connect to Azure Storage Account. Condition 1: App Service (Public ) --> Storage Account (Public, Same region) Condition 2: App Service (Public ) --> Storage Account (Public, Different region) Condition 3: App Service (Regional Vnet Integration) --> Storage Account (Private, Service Endpoint ) Condition 4: App Service (Regional Vnet Integration) --> Storage Account (Private, Private Endpoint) Before going deeper, here is a brief summary for your to choose the suitable design for your system. Requirements: If your security require the Firewall on the Storage Account. And the Storage Account and Azure App Service are in the same region. --> Use the above Condition 3 for 4 for your design. If your security require the Firewall on the Storage Account. And the Storage Account and Azure App Service are in the different region. --> Use the Condition 1 or Condition 4 for your design. When you would like to make the connection private, use the Condition 4. For the deeper analysis for the above 4 conditions, please see following: Background knowledge: ============ Azure Storage Account Network restricting logic is different from the Azure App Service. Even when we configured the Network restricting from the Azure Storage Account side, the tcpping will still working well like this: And the "List" request to the Azure Storage Account will not be locked. In order to verify if the Network Restricting is working or not, you can use the script to upload a file to Azure Storage Account to test. I am using the code: <?php $accesskey = "xxxx"; $storageAccount = 'xxxx'; $filetoUpload = realpath('xxxx'); $containerName = 'xxxx'; $blobName = 'xxxx'; $destinationURL = "https://$storageAccount.blob.core.windows.net/$containerName/$blobName"; function uploadBlob($filetoUpload, $storageAccount, $containerName, $blobName, $destinationURL, $accesskey) { $currentDate = gmdate("D, d M Y H:i:s T", time()); $handle = fopen($filetoUpload, "r"); $fileLen = filesize($filetoUpload); $headerResource = "x-ms-blob-cache-control:max-age=3600\nx-ms-blob-type:BlockBlob\nx-ms-date:$currentDate\nx-ms-version:2015-12-11"; $urlResource = "/$storageAccount/$containerName/$blobName"; $arraysign = array(); $arraysign[] = 'PUT'; /*HTTP Verb*/ $arraysign[] = ''; /*Content-Encoding*/ $arraysign[] = ''; /*Content-Language*/ $arraysign[] = $fileLen; /*Content-Length (include value when zero)*/ $arraysign[] = ''; /*Content-MD5*/ $arraysign[] = 'image/png'; /*Content-Type*/ $arraysign[] = ''; /*Date*/ $arraysign[] = ''; /*If-Modified-Since */ $arraysign[] = ''; /*If-Match*/ $arraysign[] = ''; /*If-None-Match*/ $arraysign[] = ''; /*If-Unmodified-Since*/ $arraysign[] = ''; /*Range*/ $arraysign[] = $headerResource; /*CanonicalizedHeaders*/ $arraysign[] = $urlResource; /*CanonicalizedResource*/ $str2sign = implode("\n", $arraysign); $sig = base64_encode(hash_hmac('sha256', urldecode(utf8_encode($str2sign)), base64_decode($accesskey), true)); $authHeader = "SharedKey $storageAccount:$sig"; $headers = [ 'Authorization: ' . $authHeader, 'x-ms-blob-cache-control: max-age=3600', 'x-ms-blob-type: BlockBlob', 'x-ms-date: ' . $currentDate, 'x-ms-version: 2015-12-11', 'Content-Type: image/png', 'Content-Length: ' . $fileLen ]; $ch = curl_init($destinationURL); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false); curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "PUT"); curl_setopt($ch, CURLOPT_INFILE, $handle); curl_setopt($ch, CURLOPT_INFILESIZE, $fileLen); curl_setopt($ch, CURLOPT_UPLOAD, true); $result = curl_exec($ch); echo ('Result<br/>'); print_r($result); echo ('Error<br/>'); print_r(curl_error($ch)); curl_close($ch); } uploadBlob($filetoUpload, $storageAccount, $containerName, $blobName, $destinationURL, $accesskey); Preparation: ============ Write code to upload the file to Azure Storage Account for testing as mentioned in above. Create xxx.php file under the wwwroot folder for testing. Enable the "Diagnostic settings" from Azure Portal Test: ============ Condition 1: App Service (Public ) --> Storage Account (Public, Same region) By Default, Azure App Service can access the Storage Account and upload the files. But if we enable the Firewall settings under the Networking blade of Storage Account, and add the Azure App Service Outbound IP to the whitelist like following screenshot, the Azure App Service will still not able to access Azure Storage Account. If we check the Azure Storage Account log, we will see the Azure App Service was trying to use an internal IP (100.x.x.x) to access the Storage Account. Not using the Azure App Service public outbound IP: Why? I discussed this behavior with the Azure Storage Account and Azure App Service Product Group, and the result showed if the Azure App Service and Azure Storage Account are in the same datacenter, they did some optimization, so the Azure App Service will always reach the Storage Account via the fastest route, so the resource IP (Azure App Service) observed from Azure Storage Account could be an un-predicted IP (could start with 100.x.x.x or 10.x.x.x or other others) and this IP could change at any time. Question: Considering the resource IP from Azure App Service looks like start with 100.x.x.x for now, if I whitelist the 100.0.0.0/8 in the Azure Storage Account, will the Azure App Service can upload the file on it? Answer: Yes it will make the Azure App Service can access the Storage Account for now. But cannot promise it will always work, it is a not officially support scenery. Because the IP could change to 10.x.x.x and some other un-predicted IPs. Another thing is, the x.0.0.0/8 is a huge range, so it is not a good design. If you would like to enable the Firewall in the Azure Storage Account and would like to make sure the Azure App Service in the same region still could access this Storage Account, please see the details in the Condition 3 and Condition 4. Condition 2: App Service (Public ) --> Storage Account (Public, Different region) As we can see in the Azure Storage Account log, the Azure App Service is using the public IP accessing the Azure Storage Account (13.x.x.x is one of the public outbound IP of my test Azure App Service. ) And you can enable the Firewall on Azure Storage Account and whitelist all the Azure App Service outbound IP (Inbound/Outbound IP addresses - Azure App Service | Microsoft Docs). That will make sure the Azure App Service can always access the Storage Account. Condition 3: App Service (Regional Vnet Integration) --> Storage Account (Private, Service Endpoint ) Create Vnet Integration from Azure App Service to Azure Vnet Subnet. Make sure the "Microsoft.Storage" is enabled as Service Endpoint for the Subnet that App Service is integrated with. Make sure the "Route All" is enabled for the Vnet Integration: Configure Azure Storage firewalls and virtual networks | Microsoft Docs After doing above, if we check the Azure App Service resource IP, it will be one of the IP under the Vnet Subnet: Condition 4: App Service (Regional Vnet Integration) --> Storage Account (Private, Private Endpoint) Similar configuration as the Condition 3, but please make sure: Disable the Service Endpoint on the Vnet Subnet. Create Private Endpoint on the Storage Account side. Make sure the Storage Account FQDN could be resolved to private endpoint IP from Azure App Service: We can see the similar private IP records as we saw when using the Service Endpoint. But the different between the condition 3 is the Private Endpoint support cross region traffic , and the Service Endpoint only support the connection from the Azure App Service in the same region.Announcing the General Availability of New Availability Zone Features for Azure App Service
What are Availability Zones? Availability Zones, or zone redundancy, refers to the deployment of applications across multiple availability zones within an Azure region. Each availability zone consists of one or more data centers with independent power, cooling, and networking. By leveraging zone redundancy, you can protect your applications and data from data center failures, ensuring uninterrupted service. Key Updates The minimum instance requirement for enabling Availability Zones has been reduced from three instances to two, while still maintaining a 99.99% SLA. Many existing App Service plans with two or more instances will automatically support Availability Zones without additional setup. The zone redundant setting for App Service plans and App Service Environment v3 is now mutable throughout the life of the resources. Enhanced visibility into Availability Zone information, including physical zone placement and zone counts, is now provided. For App Service Environment v3, the minimum instance fee for enabling Availability Zones has been removed, aligning the pricing model with the multi-tenant App Service offering. The minimum instance requirement for enabling Availability Zones has been reduced from three instances to two. You can now enjoy the benefits of Availability Zones with just two instances since we continue to uphold a 99.99% SLA even with the two-instance configuration. Many existing App Service plans with two or more instances will automatically support Availability Zones without necessitating additional setup. Over the past few years, efforts have been made to ensure that the App Service footprint supports Availability Zones wherever possible, and we’ve made significant gains in doing so. Therefore, many existing customers can enable Availability Zones on their current deployments without needing to redeploy. Along with supporting 2-instance Availability Zone configuration, we have enabled Availability Zones on the App Service footprint in regions where only two zones may be available. Previously, enabling Availability Zones required a region to have three zones with sufficient capacity. To account for the growing demand, we now support Availability Zone deployments in regions with just two zones. This allows us to provide you with Availability Zone features across more regions. And with that, we are upholding the 99.99% SLA even with the 2-zone configuration. Additionally, we are pleased to announce that the zone redundant setting (zoneRedundant property) for App Service plans and App Service Environment v3 is now mutable throughout the life of these resources. This enhancement allows customers on Premium V2, Premium V3, or Isolated V2 plans to toggle zone redundancy on or off as required. With this capability, you can reduce costs and scale to a single instance when multiple instances are not necessary. Conversely, you can scale out and enable zone redundancy at any time to meet your requirements. This ability has been requested for a while now and we are excited to finally make it available. For App Service Environment v3 users, this also means that your individual App Service plan zone redundancy status is now independent of other plans in your App Service Environment. This means that you can have a mix of zone redundant and non-zone redundant plans in an App Service Environment, something that was previously not supported. In addition to these new features, we also have a couple of other exciting things to share. We are now providing enhanced visibility into Availability Zone information, including the physical zone placement of your instances and zone counts. For our App Service Environment v3 customers, we have removed the minimum instance fee for enabling Availability Zones. This means that you now only pay for the Isolated V2 instances you consume. This aligns the pricing model with the multi-tenant App Service offering. For more information as well as guidance on how to use these features, see the docs - Reliability in Azure App Service. Azure Portal support for these new features will be available by mid-June 2025. In the meantime, see the documentation to use these new features with ARM/Bicep or the Azure CLI. Also check out BRK200 breakout session at Microsoft Build 2025 live on May 20th or anytime after via the recording where my team and I will be discussing these new features and many more exciting announcements for Azure App Service. If you’re in the Seattle area and attending Microsoft Build 2025 in person, come meet my team and me at our Expert Meetup Booth. FAQ Q: What are availability zones? Availability zones are physically separate locations within an Azure region, each consisting of one or more data centers with independent power, cooling, and networking. Deploying applications across multiple availability zones ensures high availability and business continuity. Q: How do I enable Availability Zones for my existing App Service plan or App Service Environment v3? There is a new toggle in the Azure portal that will be enabled if your App Service plan or App Service Environment v3 supports Availability Zones. Your deployment must be on the App Service footprint that supports zones in order to have this capability. There is a new property called “MaximumNumberOfZones”, which indicates the number of zones your deployment supports. If this value is greater than one, you are on the footprint that supports zones and can enable Availability Zones as long as you have two or more instances. If this value is equal to one, you need to redeploy. Note that we are continually working to expand the zone footprint across more App Service deployments. Q: Is there an additional charge for Availability Zones? There is no additional charge, you only pay for the instances you use. The only requirement is that you use two or more instances. Q: Can I change the zone redundant property after creating my App Service plan? Yes, the zone redundant property is now mutable, meaning you can toggle it on or off at any time. Q: How can I verify the zone redundancy status of my App Service Plans? We now display the physical zone for each instance, helping you verify zone redundancy status for audits and compliance reviews. Q: How do I use these new features? You can use ARM/Bicep or the Azure CLI at this time. Starting in mid-June, Azure Portal support should be available. The documentation currently shows how to use ARM/Bicep and the Azure CLI to enable these features. The documentation as well as this blog post will be updated once Azure Portal support is available. Q: Are Availability Zones supported on Premium V4? Yes! See the documentation for more details on how to get started with Premium V4 today.
