Disciplined Guardrail Development in enterprise application with GitHub Copilot
What Is Disciplined Guardrail-Based Development? In AI-assisted software development, approaches like Vibe Coding—which prioritize momentum and intuition—often fail to ensure code quality and maintainability. To address this, Disciplined Guardrail-Based Development introduces structured rules ("guardrails") that guide AI systems during coding and maintenance tasks, ensuring consistent quality and reliability. To get AI (LLMs) to generate appropriate code, developers must provide clear and specific instructions. Two key elements are essential: What to build – Clarifying requirements and breaking down tasks How to build it – Defining the application architecture The way these two elements are handled depends on the development methodology or process being used. Here are examples as follows. How to Set Up Disciplined Guardrails in GitHub Copilot To implement disciplined guardrail-based development with GitHub Copilot, two key configuration features are used: 1. Custom Instructions (.github/copilot-instructions.md): This file allows you to define persistent instructions that GitHub Copilot will always refer to when generating code. Purpose: Establish coding standards, architectural rules, naming conventions, and other quality guidelines. Best Practice: Instead of placing all instructions in a single file, split them into multiple modular files and reference them accordingly. This improves maintainability and clarity. Example Use: You might define rules like using camelCase for variables, enforcing error boundaries in React, or requiring TypeScript for all new code. https://docs.github.com/en/copilot/how-tos/configure-custom-instructions/add-repository-instructions 2. Chat Modes (.github/chatmodes/*.chatmode.md): These files define specialized chat modes tailored to specific tasks or workflows. Purpose: Customize Copilot’s behavior for different development contexts (e.g., debugging, writing tests, refactoring). Structure: Each .chatmode.md file includes metadata and instructions that guide Copilot’s responses in that mode. Example Use: A debug.chatmode.md might instruct Copilot to focus on identifying and resolving runtime errors, while a test.chatmode.md could prioritize generating unit tests with specific frameworks. https://code.visualstudio.com/docs/copilot/customization/custom-chat-modes The files to be created and their relationships are as follows. Next, there are introductions for the specific creation method. #1: Custom Instructions With custom instructions, you can define commands that are always provided to GitHub Copilot. The prepared files are always referenced during chat sessions and passed to the LLM (this can also be confirmed from the chat history). An important note is to split the content into several files and include links to those files within the .github/copilot-instructions.md file. Because it can become too long if everything is written in a single file. There are mainly two types of content that should be described in custom instructions: A: Development Process (≒ outcome + Creation Method) What documents or code will be created: requirements specification, design documents, task breakdown tables, implementation code, etc. In what order and by whom they will be created: for example, proceed in the order of requirements definition → design → task breakdown → coding. B: Application Architecture How will the outcome be defined in A be created? What technology stack and component structure will be used? A concrete example of copilot-instructions.md is shown below. # Development Rules ## Architecture - When performing design and coding tasks, always refer to the following architecture documents and strictly follow them as rules. ### Product Overview - Document the product overview in `.github/architecture/product.md` ### Technology Stack - Document the technologies used in `.github/architecture/techstack.md` ### Coding Standards - Document coding standards in `.github/architecture/codingrule.md` ### Project Structure - Document the project directory structure in `.github/architecture/structure.md` ### Glossary (Japanese-English) - Document the list of terms used in the project in `.github/architecture/dictionary.md` ## Development Flow - Follow a disciplined development flow and execute the following four stages in order (proceed to the next stage only after completing the current one): 1. Requirement Definition 2. Design 3. Task Breakdown 4. Coding ### 1. Requirement Definition - Document requirements in `docs/[subsystem_name]/[business_name]/requirement.md` - Use `requirement.chatmode.md` to define requirements - Focus on clarifying objectives, understanding the current situation, and setting success criteria - Once requirements are defined, obtain user confirmation before proceeding to the next stage ### 2. Design - Document design in `docs/[subsystem_name]/[business_name]/design.md` - Use `design.chatmode.md` to define the design - Define UI, module structure, and interface design - Once the design is complete, obtain user confirmation before proceeding to the next stage ### 3. Task Breakdown - Document tasks in `docs/[subsystem_name]/[business_name]/tasks.md` - Use `tasks.chatmode.md` to define tasks - Break down tasks into executable units and set priorities - Once task breakdown is complete, obtain user confirmation before proceeding to the next stage ### 4. Coding - Implement code under `src/[subsystem_name]/[business_name]/` - Perform coding task by task - Update progress in `docs/[subsystem_name]/[business_name]/tasks.md` - Report to the user upon completion of each task Note: The only file that is always sent to the LLM is `copilot-instructions.md`. Documents linked from there (such as `product.md` or `techstack.md`) are not guaranteed to be read by the LLM. That said, a reasonably capable LLM will usually review these files before proceeding with the work. If the LLM does not properly reference each file, you may explicitly add these architecture documents to the context. Another approach is to instruct the LLM to review these files in the **chat mode settings**, which will be described later. There are various “schools of thought” regarding application architecture, and it is still an ongoing challenge to determine exactly what should be defined and what documents should be created. The choice of architecture depends on factors such as the business context, development scale, and team structure, so it is difficult to prescribe a one-size-fits-all approach. That said, as a general guideline, it is desirable to summarize the following: Product Overview: Overview of the product, service, or business, including its overall characteristics Technology Stack: What technologies will be used to develop the application? Project Structure: How will folders and directories be organized during development? Module Structure: How will the application be divided into modules? Coding Rules: Rules for handling exceptions, naming conventions, and other coding practices Writing all of this from scratch can be challenging. A practical approach is to create template information with the help of Copilot and then refine it. Specifically, you can: Use tools like M365 Copilot Researcher to create content based on general principles Analyze a prototype application and have the architecture information summarized (using Ask mode or Edit mode, feed the solution files to a capable LLM for analysis) However, in most cases, the output cannot be used as-is. The structure may not be analyzed correctly (hallucinations may occur) Project-specific practices and rules may not be captured Use the generated content as a starting point, and then refine it to create architecture documentation tailored to your own project. When creating architecture documents for enterprise-scale application development, a useful approach is to distinguish between the foundational parts and the individual application parts. Discipline-based guardrail development is particularly effective when building multiple applications in a “cookie-cutter” style on top of a common foundation. A cler example of this is Data-Oriented Architecture (DOA). In DOA, individual business applications are built on top of a shared database that serves as the overall common foundation. In this case, the foundational parts (the database layer) should not be modified arbitrarily by individual developers. Instead, focus on how to standardize the development of the individual application parts (the blue-framed sections) while ensuring consistency. Architecture documentation should be organized with this distinction in mind, emphasizing the uniformity of application-level development built upon the stable foundation. #2 Chat Mode By default, GitHub Copilot provides three chat modes: Ask, Edit, and Agent. However, by creating files under .github/chatmodes/*.chatmode.md, you can customize the Agent mode to create chat modes tailored for specific tasks. Specifically, you can configure the following three aspects. Functionally, this allows you to perform a specific task without having to manually change the model or tools, or write detailed instructions each time: model: Specify the default LLM to use (Note: The user can still manually switch to another LLM if desired) tools: Restrict which tools can be used (Note: The user can still manually select other tools if desired) custom instructions: Provide custom instructions specific to this chat mode A concrete example of .github/chatmodes/*.chatmode.md is shown below. description: This mode is used for requirement definition tasks. model: Claude Sonnet 4 tools: ['changes', 'codebase', 'editFiles', 'fetch', 'findTestFiles', 'githubRepo', 'new', 'openSimpleBrowser', 'runCommands', 'search', 'searchResults', 'terminalLastCommand', 'terminalSelection', 'usages', 'vscodeAPI', 'mssql_connect', 'mssql_disconnect', 'mssql_list_servers', 'mssql_show_schema'] --- # Requirement Definition Mode In this mode, requirement definition tasks are performed. Specifically, the project requirements are clarified, and necessary functions and specifications are defined. Based on instructions or interviews with the user, document the requirements according to the format below. If any specifications are ambiguous or unclear, Copilot should ask the user questions to clarify them. ## File Storage Location Save the requirement definition file in the following location: - Save as `requirement.md` under the directory `docs/[subsystem_name]/[business_name]/` ## Requirement Definition Format While interviewing the user, document the following items in the Markdown file: - **Subsystem Name**: The name of the subsystem to which this business belongs - **Business Name**: The name of the business - **Overview**: A summary of the business - **Use Cases**: Clarify who uses this business, when/under what circumstances, and for what purpose, using the following structure: - **Who (Persona)**: User or system roles - **When/Under What Circumstances (Scenario)**: Timing when the business is executed - **Purpose (Goal)**: Objectives or expected outcomes of the business - **Importance**: The importance of the business (e.g., High, Medium, Low) - **Acceptance Criteria**: Conditions that must be satisfied for the requirement to be considered met - **Status**: Current state of the requirement (e.g., In Progress, Completed) ## After Completion - Once requirement definition is complete, obtain user confirmation and proceed to the next stage (Design). Tips for Creating Chat Modes Here are some tips for creating custom chat modes: Align with the development process: Create chat modes based on the workflow and the deliverables. Instruct the LLM to ask the user when unsure: Direct the LLM to request clarification from the user if any information is missing. Clarify what deliverables to create and where to save them: Make it explicit which outputs are expected and their storage locations. The second point is particularly important. Many AI (LLMs) tend to respond to user prompts in a sycophantic manner (known as sycophancy). As a result, they may fill in unspecified requirements or perform tasks that were not requested, often with the intention of being helpful. The key difference between Ask/Edit modes and Agent mode is that Agent mode allows the LLM to proactively ask questions and engage in dialogue with the user. However, unless the user explicitly includes a prompt such as “ask if you don’t know,” the AI rarely initiates questions on its own. By creating a custom chat mode and instructing the LLM to “ask the user when unsure,” you can fully leverage the benefits of Agent mode. About Tools You can easily check tool names from the list of available tools in the command palette. Alternatively, as shown in the diagram below, it can be convenient to open the custom chat mode file and specify the tool configuration. You can specify not only the MCP server functionality but also built-in tools and Copilot Extensions. Example of Actual Operation An example interaction when using this chat mode is as follows: The LLM behaves according to the custom instructions defined in the chat mode. When you answer questions from GHC, the LLM uses that information to reason and proceed with the task. However, the output is not guaranteed to be correct (hallucinations may occur) → A human should review the output and make any necessary corrections before committing. The basic approach to disciplined guardrail-based development has been covered above. In actual business application development, it is also helpful to understand the following two points: Referencing the database schema Integrated management of design documents and implementation code (Important) Reading the Database Schema In business application development, requirements definition and functional design are often based on the schema information of entities. There are two main ways to allow the system to read schema information: Dynamically read the schema from a development/test DB server using MCP or similar tools. Include a file containing schema information within the project and read from it. A development/test database can be prepared, and schema information can be read via the MCP server or Copilot Extensions. For SQL Server or Azure SQL Database, an MCP Server is available, but its setup can be cumbersome. Therefore, using Copilot Extensions is often easier and recommended. This approach is often seen online, but it is not recommended for the following reasons: Setting up MCP Server or Copilot Extensions can be cumbersome (installation, connection string management, etc.) It is time-consuming (the LLM needs schema information → reads the schema → writes code based on it) Connecting to a DB server via MCP or similar tools is useful for scenarios such as “querying a database in natural language” for non-engineers performing data analysis. However, if the goal is simply to obtain the schema information of entities needed for business application development, the method described below is much simpler. Storing Schema Information Within the Project Place a file containing the schema information inside the project. Any of the following formats is recommended. Write custom instructions so that development refers to this file: DDL (full CREATE DATABASE scripts) O/R mapper files (e.g., Entity Framework context files) Text files documenting schema information, etc. DDL files are difficult for humans to read, but AI (LLMs) can easily read and accurately understand them. In .NET + SQL development, it is recommended to include both the DDL and EF O/R mapper files. Additionally, if you include links to these files in your architecture documents and chat mode instructions, the LLM can generate code while understanding the schema with high accuracy. Integrated Management of Design Documents and Implementation Code Disciplined guardrail-based development with LLMs has made it practical to synchronize and manage design documents and implementation code together—something that was traditionally very difficult. In long-standing systems, it is common for old design documents to become largely useless. During maintenance, code changes are often prioritized. As a result, updating and maintaining design documents tends to be neglected, leading to a significant divergence between design documents and the actual code. For these reasons, the following have been considered best practices (though often not followed in reality): Limit requirements and external design documents to the minimum necessary. Do not create internal design documents; instead, document within the code itself. Always update design documents before making changes to the implementation code. When using LLMs, guardrail-based development makes it easier to enforce a “write the documentation first” workflow. Following the flow of defining specifications, updating the documents, and then writing code also helps the LLM generate appropriate code more reliably. Even if code is written first, LLM-assisted code analysis can significantly reduce the effort required to update the documentation afterward. However, the following points should be noted when doing this: Create and manage design documents as text files, not Word, Excel, or PowerPoint. Use text-based technologies like Mermaid for diagrams. Clearly define how design documents correspond to the code. The last point is especially important. It is crucial to align the structure of requirements and design documents with the structure of the implementation code. For example: Place design documents directly alongside the implementation code. Align folder structures, e.g., /doc and /src. Information about grouping methods and folder mapping should be explicitly included in the custom instructions. Conclusion of Disciplined Guardrail-Based Development with GHC Formalizing and Applying Guardrails Define the development flow and architecture documents in .github/copilot-instructions.md using split references. Prepare .github/chatmodes/* for each development phase, enforcing “ask the AI if anything is unclear.” Synchronization of Documents and Implementation Code Update docs first → use the diff as the basis for implementation (Doc-first). Keep docs in text format (Markdown/Mermaid). Fix folder correspondence between /docs and /src. Handling Schemas Store DDL/O-R mapper files (e.g., EF) in the repository and have the LLM reference them. Minimize dynamic DB connections, prioritizing speed, reproducibility, and security. This disciplined guardrail-based development technique is an AI-assisted approach that significantly improves the quality, maintainability, and team efficiency of enterprise business application development. Adapt it appropriately to each project to maximize productivity in application development.Generating Classes with Custom Naming Conventions Using GitHub Copilot and a Custom MCP Server
GitHub Spark and GitHub Copilot are powerful development tools that can significantly boost productivity even when used out of the box. However, in enterprise settings, a common request is for development support that aligns with specific compliances or regulations. While GitHub Copilot allows you to choose models like GPT-4o or others, it does not currently support the use of custom fine-tuned models. Additionally, many users might find it unclear how to integrate Copilot with external services, which can be a source of frustration. To address such needs, one possible approach is to build a custom MCP server and connect it to GitHub Copilot. For a basic “Hello World” style guide on how to set this up, please refer to the articles below. https://devblogs.microsoft.com/dotnet/build-a-model-context-protocol-mcp-server-in-csharp/ https://learn.microsoft.com/en-us/dotnet/ai/quickstarts/build-mcp-server By building an MCP server as an ASP.NET Core application and integrating it with GitHub Copilot, you can introduce custom rules and functionality tailored to your organization’s needs. The architecture for this post would look like the following: While the MCP server can technically be hosted anywhere as long as HTTP communication is possible, for enterprise use cases, it’s often recommended to deploy it within a private endpoint inside a closed Virtual Network. In production environments, this setup can be securely accessed from client machines via ExpressRoute, ensuring both compliance and network isolation. Building an MCP Server Using ASP.NET Core Start by creating a new ASP.NET Core Web API project in Visual Studio. Then, install the required libraries via NuGet. Note: Make sure to enable the option to include preview versions—otherwise, some of the necessary packages may not appear in the list. ModelContextProtocol ModelContextProtocol.AspNetCore Next, update the Program.cs file as shown below to enable the MCP server functionality. We’ll create the NamingConventionManagerTool class later, but as you can see, it’s being registered via Dependency Injection during application startup. This allows it to be integrated as part of the MCP server’s capabilities. using MCPServerLab01.Tools; var builder = WebApplication.CreateBuilder(args); builder.WebHost.ConfigureKestrel(servreOptions => { servreOptions.ListenAnyIP(8888); // you can change any port to adopt your environment }); builder.Logging.AddConsole(consoleLogOptions => { consoleLogOptions.LogToStandardErrorThreshold = LogLevel.Trace; }); builder.Services.AddMcpServer() .WithHttpTransport() .WithTools<NamingConventionManagerTool>(); var app = builder.Build(); app.MapMcp(); app.Run(); Next, create the NamingConventionManagerTool.cs file. By decorating the class with the [McpServerToolType] and [McpServerTool] attributes, you can expose it as a feature accessible via the MCP server. In this example, we’ll add a tool that assigns class names based on business logic IDs—a common pattern in system integration projects. The class will include the following methods: GetNamingRules: Provides an overview of the naming conventions to follow. GenerateClassNamingConvention: Generates a class name based on a given business logic ID. DetermineBusinessCategory: Extracts the business logic ID from a given class name. As noted in the prompt, we’ll assume this is part of a fictional project called Normalian Project. using ModelContextProtocol.Server; using System.ComponentModel; using System.Text.Json.Serialization; namespace MCPServerLab01.Tools; [McpServerToolType] [Description()] public class NamingConventionManagerTool { // Counter for sequential management // This is just a trial, so here is a static variable. For production, consider saving to a DB or other persistent storage. static int _counter = 0; [McpServerTool, Description(""" Provides Normalian Project rules that must be followed when adding or modifying programs. Be sure to refer to these rules as they are mandatory. """)] public string GetNamingRules() { return """ In this Normalian project, to facilitate management, class names must follow naming conventions based on business categories. Class names according to business categories are provided by the `GenerateClassNamingConvention` tool. Please define classes using the names provided here. Do not define classes with any other names. If you are unsure about the business category from the class name, use the `DetermineBusinessCategory` tool to obtain the business category. """; } [McpServerTool, Description(""" Retrieves a set of classes and namespaces that should be created for the specified business category in Normalian project. You must create classes using the names suggested here. """)] public ClassNamingConvention GenerateClassNamingConvention( [Description("Business category for the class to be created")] BusinessCategory businessCategory) { var number = _counter++; var prefix = businessCategory switch { BusinessCategory.NormalianOrder => "A", BusinessCategory.NormalianProduct => "B", BusinessCategory.NormalianCustomer => "C", BusinessCategory.NormalianSupplier => "D", BusinessCategory.NormalianEmployee => "E", _ => throw new ArgumentException("Unknown category."), }; var name = $"{prefix}{number:D4}"; return new ClassNamingConvention( ServiceNamespace: "{YourRootNamespace}.Services", ServiceClassName: $"{name}Service", UsecaseNamespace: "{YourRootNamespace}.Usecases", UsecaseClassName: $"{name}Usecase", DtoNamespace: "{YourRootNamespace}.Dtos", DtoClassName: $"{name}Dto"); } [McpServerTool, Description("If you do not know the business category in Normalian project from the class name, check the naming convention to obtain the business category to which the class belongs.")] public BusinessCategory DetermineBusinessCategory( [Description("Class name")] string className) { ArgumentException.ThrowIfNullOrEmpty(className); var prefix = className[0]; return prefix switch { 'A' => BusinessCategory.NormalianOrder, 'B' => BusinessCategory.NormalianProduct, 'C' => BusinessCategory.NormalianCustomer, 'D' => BusinessCategory.NormalianSupplier, 'E' => BusinessCategory.NormalianEmployee, _ => throw new ArgumentException("Unknown class name."), }; } } [Description("Class name to use in Normalian project")] public record ClassNamingConvention( [Description("Service namespace")] string ServiceNamespace, [Description("Class name to use for the service layer")] string ServiceClassName, [Description("Usecase namespace")] string UsecaseNamespace, [Description("Class name to use for the usecase layer")] string UsecaseClassName, [Description("DTO namespace")] string DtoNamespace, [Description("Class name to use for DTOs")] string DtoClassName); [JsonConverter(typeof(JsonStringEnumConverter))] public enum BusinessCategory { NormalianOrder, NormalianProduct, NormalianCustomer, NormalianSupplier, NormalianEmployee, } Next, run the project in Visual Studio to launch the MCP server as an ASP.NET Core application. Once the application starts, take note of the HTTP endpoint displayed in the console or output window as follows —this will be used to interact with the MCP server. Connecting GitHub Copilot Agent to the MCP Server Next, connect the GitHub Copilot Agent to the MCP server. You can easily connect the GitHub Copilot Agent by simply specifying the MCP server's endpoint. To add the server, select Agent mode and click the wrench icon as shown below. From Add MCP Server, select HTTP and specify http://localhost:<<PORT>>/sse endpoint. Give it an appropriate name, then choose where to save the MCP server settings—either User Settings or Workspace Settings. If you will use it just for yourself, User Settings should be fine. However, selecting Workspace Settings is useful when you want to share the configuration with your team. Since this is just a trial and we only want to use it within this workspace, we chose Workspace Settings. This will create a .vscode/mcp.json file with the following content: { "servers": { "mine-mcp-server": { "type": "http", "url": "http://localhost:8888/" } }, "inputs": [] } You'll see a Start icon on top of the JSON file—click it to launch the MCP server. Once started, the MCP server will run as shown below. You can also start it from the GitHub Copilot Chat window. After launching, if you click the wrench icon in the Chat window, you'll see a list of tools available on the connected MCP server. Using the Created Features via GitHub Copilot Agent Now, let’s try it out. Open the folder of your .NET console app project in VS Code, and make a request to the Agent like the one shown below. It’s doing a great job trying to check the rules. Next, it continues by trying to figure out the class name and other details needed for implementation. Then, following the naming conventions, it looks up the appropriate namespace and even starts creating folders. Once the folder is created, the class is properly generated with the specified class name. Even when you ask about the business category, it uses the tool correctly to look it up. Impressive! Conclusion In this article, we introduced how to build your own MCP server and use it via the GitHub Copilot Agent. By implementing an MCP server and adding tools tailored to your business needs, you can gain a certain level of control over the Agent’s behavior and build your own ecosystem. For this trial, we used a local machine to test the behavior, but for production use, you'll need to consider deploying the MCP server to Azure, adding authentication features, and other enhancements.Develop Custom Engine Agent to Microsoft 365 Copilot Chat with pro-code
There are some great articles that explain how to integrate an MCP server built on Azure with a declarative agent created using Microsoft Copilot Studio. These approaches aim to extend the agent’s capabilities by supplying it with tools, rather than defining a fixed role. Here were some of the challenges w encountered: The agent's behavior can only be tested through the Copilot Studio web interface, which isn't ideal for iterative development. You don’t have control over which LLM is used as the orchestrator—for example, there's no way to specify GPT-4o. The agent's responses don’t always behave the same as they would if you were prompting the LLM directly. These limitations got me thinking: why not build the entire agent myself? At the same time, I still wanted to take advantage of the familiar Microsoft 365 Copilot interface on the frontend. As I explored further, I discovered that the Microsoft 365 Copilot SDK makes it possible to bring in your own custom-built agent.How to Secure your pro-code Custom Engine Agent of Microsoft 365 Copilot?
Prerequisite This article assumes that you’ve already gone through a following post. Please make sure to read it before proceeding: Developing a Custom Engine Agent for Microsoft 365 Copilot Chat Using Pro-Code With the article, we have found how to publish a Custom Engine Agent using pro-code approaches such as C#. In this post, I’d like to shift the focus to security, specifically how to protect the endpoint of our custom Microsoft 365 Copilot. Through several architectural explorations, we found an approach that seems to work well. However, I strongly encourage you to review and evaluate it carefully for your production environment. Which Endpoints Can Be Controlled? In the current architecture, there are three key endpoints to consider from a security perspective: Teams Endpoint This is the entry point where users interact with the Custom Engine Agent through Microsoft Teams. Azure Bot Service Endpoint This is the publicly accessible endpoint provided by Azure Bot Service that relays messages between Teams and your bot backend. ASP.NET Core Endpoint In the previous article, we used a local devtunnel for development purposes. In a production environment, however, this would likely be hosted on Azure App Service or others. Each of these endpoints may require different protection strategies, which we’ll explore in the following sections. 1. Controlling the Teams Endpoint When it comes to the Teams endpoint, control ultimately comes down to Teams app management within your Microsoft 365 tenant. Specifically, the manifest file for your custom Teams app (i.e., the Custom Agent) needs to be uploaded in your tenant, and access is governed via the Teams Admin Center. This isn’t about controlling the endpoint, but rather about limiting who can access the app. You can restrict access on a per-user or per-group basis, effectively preventing malicious users inside your organization from using the app. However, you cannot restrict access at the endpoint level, nor could you prevent a malicious external organization from copying the app package. This limitation may pose a concern, especially when thinking about endpoint-level security outside your tenant’s control. 2. Controlling the Azure Bot Service Endpoint The Azure Bot Service endpoint acts as a bridge between the Teams Channel and your pro-code backend. Here, the only available security configuration is to specify the Service Principal that the agent uses. There isn’t much room for granular control here—it’s essentially a relay point managed by Azure Bot Service, and protection depends largely on how you secure the endpoints it connects to. 3. Controlling the ASP.NET Core Endpoint This is where endpoint protection becomes critical. When you configure your bot in Azure Bot Service, you must expose your pro-code endpoint to the public internet. In our earlier article, we used a local devtunnel for development. But in production, you’ll likely use Azure App Service or others, which results in a publicly accessible endpoint. While Microsoft provides documentation on network isolation options for Azure Bot Service, these are currently only supported when using the Direct Line channel - not the Teams channel. This means that when using Teams as the entry point, you cannot isolate the backend endpoint via a private network, making it critical to implement other security measures at the app level (e.g., token validation, IP restrictions, mutual TLS, etc.). https://learn.microsoft.com/en-us/azure/bot-service/dl-network-isolation-concept?view=azure-bot-service-4.0 Let’s Review Other Articles on this Topic There are several valuable resources that describe this topic. Since Microsoft Teams is a SaaS application, the bot endpoint (e.g., https://my-webapp-endpoint.net/api/messages) must be publicly accessible when integrated through the Teams channel. Is it possible to integrate Azure Bot with Teams without public access? How to create Azure Bot Service in a private network? In particular, this article provides an excellent deep dive into the traffic flow between Teams and Azure Bot Service: Azure Bot Service, Microsoft Teams architecture, and message flow In the section titled “Challenge 2: Network isolation vs. Teams connectivity,” the article clearly explains why network-level isolation is fundamentally incompatible with the Teams channel. The article also outlines a practical security approach using Azure Firewall, NSG (Network Security Groups), and JWT token validation at the application level. If you're using the Teams channel, complete network isolation is not feasible—which makes sense, given that Teams itself is a SaaS platform and cannot be brought into your private network. As a result, protecting the backend bot (e.g., the ASP.NET Core endpoint) will require application-level controls, particularly JWT token validation to ensure that only trusted sources can invoke the bot. Let’s now take a closer look at how to implement that in C#. Controlling Endpoints in the ASP.NET Core Application So, what does endpoint control look like at the application level? Let’s return to the ASP.NET Core side of things and take a closer look at the default project structure. If you recall, the Program.cs in the template project contains a specific line worth revisiting. This configuration plays an important role in how the application handles and secures incoming requests. Let’s take a look at that setup. // Register the WeatherForecastAgent builder.Services.AddTransient<WeatherForecastAgent>(); // Add AspNet token validation - ** HERE ** builder.Services.AddBotAspNetAuthentication(builder.Configuration); // Register IStorage. For development, MemoryStorage is suitable. // For production Agents, persisted storage should be used so // that state survives Agent restarts, and operate correctly // in a cluster of Agent instances. builder.Services.AddSingleton<IStorage, MemoryStorage>(); As it turns out, the AddBotAspNetAuthentication method referenced earlier in Program.cs is actually defined in the same project, within a file named AspNetExtensions.cs. This method is where access token validation is implemented and enforced. Let’s take a closer look at a key portion of the AddBotAspNetAuthentication method from AspNetExtensions.cs: public static void AddBotAspNetAuthentication(this IServiceCollection services, IConfiguration configuration, string tokenValidationSectionName = "TokenValidation", ILogger logger = null) { IConfigurationSection tokenValidationSection = configuration.GetSection(tokenValidationSectionName); List<string> validTokenIssuers = tokenValidationSection.GetSection("ValidIssuers").Get<List<string>>(); List<string> audiences = tokenValidationSection.GetSection("Audiences").Get<List<string>>(); if (!tokenValidationSection.Exists()) { logger?.LogError("Missing configuration section '{tokenValidationSectionName}'. This section is required to be present in appsettings.json",tokenValidationSectionName); throw new InvalidOperationException($"Missing configuration section '{tokenValidationSectionName}'. This section is required to be present in appsettings.json"); } // If ValidIssuers is empty, default for ABS Public Cloud if (validTokenIssuers == null || validTokenIssuers.Count == 0) { validTokenIssuers = [ "https://api.botframework.com", "https://sts.windows.net/d6d49420-f39b-4df7-a1dc-d59a935871db/", "https://login.microsoftonline.com/d6d49420-f39b-4df7-a1dc-d59a935871db/v2.0", "https://sts.windows.net/f8cdef31-a31e-4b4a-93e4-5f571e91255a/", "https://login.microsoftonline.com/f8cdef31-a31e-4b4a-93e4-5f571e91255a/v2.0", "https://sts.windows.net/69e9b82d-4842-4902-8d1e-abc5b98a55e8/", "https://login.microsoftonline.com/69e9b82d-4842-4902-8d1e-abc5b98a55e8/v2.0", ]; string tenantId = tokenValidationSection["TenantId"]; if (!string.IsNullOrEmpty(tenantId)) { validTokenIssuers.Add(string.Format(CultureInfo.InvariantCulture, AuthenticationConstants.ValidTokenIssuerUrlTemplateV1, tenantId)); validTokenIssuers.Add(string.Format(CultureInfo.InvariantCulture, AuthenticationConstants.ValidTokenIssuerUrlTemplateV2, tenantId)); } } if (audiences == null || audiences.Count == 0) { throw new ArgumentException($"{tokenValidationSectionName}:Audiences requires at least one value"); } bool isGov = tokenValidationSection.GetValue("IsGov", false); bool azureBotServiceTokenHandling = tokenValidationSection.GetValue("AzureBotServiceTokenHandling", true); // If the `AzureBotServiceOpenIdMetadataUrl` setting is not specified, use the default based on `IsGov`. This is what is used to authenticate ABS tokens. string azureBotServiceOpenIdMetadataUrl = tokenValidationSection["AzureBotServiceOpenIdMetadataUrl"]; if (string.IsNullOrEmpty(azureBotServiceOpenIdMetadataUrl)) { azureBotServiceOpenIdMetadataUrl = isGov ? AuthenticationConstants.GovAzureBotServiceOpenIdMetadataUrl : AuthenticationConstants.PublicAzureBotServiceOpenIdMetadataUrl; } // If the `OpenIdMetadataUrl` setting is not specified, use the default based on `IsGov`. This is what is used to authenticate Entra ID tokens. string openIdMetadataUrl = tokenValidationSection["OpenIdMetadataUrl"]; if (string.IsNullOrEmpty(openIdMetadataUrl)) { openIdMetadataUrl = isGov ? AuthenticationConstants.GovOpenIdMetadataUrl : AuthenticationConstants.PublicOpenIdMetadataUrl; } TimeSpan openIdRefreshInterval = tokenValidationSection.GetValue("OpenIdMetadataRefresh", BaseConfigurationManager.DefaultAutomaticRefreshInterval); _ = services.AddAuthentication(options => { options.DefaultAuthenticateScheme = JwtBearerDefaults.AuthenticationScheme; options.DefaultChallengeScheme = JwtBearerDefaults.AuthenticationScheme; }) .AddJwtBearer(options => { options.SaveToken = true; options.TokenValidationParameters = new TokenValidationParameters { ValidateIssuer = true, ValidateAudience = true, ValidateLifetime = true, ClockSkew = TimeSpan.FromMinutes(5), ValidIssuers = validTokenIssuers, ValidAudiences = audiences, ValidateIssuerSigningKey = true, RequireSignedTokens = true, }; // Using Microsoft.IdentityModel.Validators options.TokenValidationParameters.EnableAadSigningKeyIssuerValidation(); options.Events = new JwtBearerEvents { // Create a ConfigurationManager based on the requestor. This is to handle ABS non-Entra tokens. OnMessageReceived = async context => { string authorizationHeader = context.Request.Headers.Authorization.ToString(); if (string.IsNullOrEmpty(authorizationHeader)) { // Default to AadTokenValidation handling context.Options.TokenValidationParameters.ConfigurationManager ??= options.ConfigurationManager as BaseConfigurationManager; await Task.CompletedTask.ConfigureAwait(false); return; } string[] parts = authorizationHeader?.Split(' '); if (parts.Length != 2 || parts[0] != "Bearer") { // Default to AadTokenValidation handling context.Options.TokenValidationParameters.ConfigurationManager ??= options.ConfigurationManager as BaseConfigurationManager; await Task.CompletedTask.ConfigureAwait(false); return; } JwtSecurityToken token = new(parts[1]); string issuer = token.Claims.FirstOrDefault(claim => claim.Type == AuthenticationConstants.IssuerClaim)?.Value; if (azureBotServiceTokenHandling && AuthenticationConstants.BotFrameworkTokenIssuer.Equals(issuer)) { // Use the Bot Framework authority for this configuration manager context.Options.TokenValidationParameters.ConfigurationManager = _openIdMetadataCache.GetOrAdd(azureBotServiceOpenIdMetadataUrl, key => { return new ConfigurationManager<OpenIdConnectConfiguration>(azureBotServiceOpenIdMetadataUrl, new OpenIdConnectConfigurationRetriever(), new HttpClient()) { AutomaticRefreshInterval = openIdRefreshInterval }; }); } else { context.Options.TokenValidationParameters.ConfigurationManager = _openIdMetadataCache.GetOrAdd(openIdMetadataUrl, key => { return new ConfigurationManager<OpenIdConnectConfiguration>(openIdMetadataUrl, new OpenIdConnectConfigurationRetriever(), new HttpClient()) { AutomaticRefreshInterval = openIdRefreshInterval }; }); } await Task.CompletedTask.ConfigureAwait(false); }, OnTokenValidated = context => { logger?.LogDebug("TOKEN Validated"); return Task.CompletedTask; }, OnForbidden = context => { logger?.LogWarning("Forbidden: {m}", context.Result.ToString()); return Task.CompletedTask; }, OnAuthenticationFailed = context => { logger?.LogWarning("Auth Failed {m}", context.Exception.ToString()); return Task.CompletedTask; } }; }); } From examining the code, we can see that it reads configuration settings from the appsettings.{your-env}.json file and uses them during token validation. In particular, the following line stands out: TokenValidationParameters.ValidAudiences = audiences; This ensures that only tokens issued for the configured audience (i.e., your Azure Bot Service's Service Principal) will be accepted. Any requests carrying tokens with mismatched audiences will be rejected during validation. One critical observation is that if no access token is provided at all, the code effectively lets the request through without enforcing validation. This means that if the Service Principal is misconfigured or lacks proper permissions, and therefore no token is issued with the request, the bot may still continue processing it without rejecting the request. This could potentially create a security loophole, especially if the backend API is publicly accessible. OnMessageReceived = async context => { string authorizationHeader = context.Request.Headers.Authorization.ToString(); if (string.IsNullOrEmpty(authorizationHeader)) { // Default to AadTokenValidation handling context.Options.TokenValidationParameters.ConfigurationManager ??= options.ConfigurationManager as BaseConfigurationManager; await Task.CompletedTask.ConfigureAwait(false); return; } Additional Security Concerns and Improvements Another point worth noting about the current code is that the Custom Engine Agent app can be copied and uploaded to a different Entra ID tenant, and it would still work. (Admittedly, this might be intentional since the architecture assumes providing Custom Engine Agent services to multiple organizations.) The project template and Teams settings raise two key security concerns that we should address: Reject requests when the token is missing - token should not be empty. Block access from unknown or unauthorized Entra ID tenants. To enforce the above, you will need to update the Service Principal configuration accordingly. Specifically, open the Service Principal's API permissions tab and add the following permission: User.Read.All Without this permission, access tokens will not be issued, making token validation impossible. After updating the Service Principal permissions, run your ASP.NET Core app and set a breakpoint around the following code to inspect the contents of the token included in the Authorization header. This will help you verify whether the token is correctly issued and contains the expected claims. string authorizationHeader = context.Request.Headers.Authorization.ToString(); The token is Base64 encoded, so let’s decode it to inspect its contents. I asked Copilot to help us decode the token so we can better understand the claims and data included inside. Let's inspect the token contents. After decoding the token (some parts are redacted for privacy), we can see that: The aud (audience) claim contains the Service Principal’s client ID. The serviceurl claim includes the Entra ID tenant ID. I attempted to configure the authorization settings to include the Tenant ID directly in the access token claims, but was not successful this time. Below is a sample code snippet that implements of the following requirements: Reject requests with an empty or missing token. Deny access from unknown Entra ID tenants. This is the sample code for "1. Reject requests with an empty or missing token". I’ve added comments in the code to clearly indicate what was changed. public static void AddBotAspNetAuthentication(this IServiceCollection services, IConfiguration configuration, string tokenValidationSectionName = "TokenValidation", ILogger logger = null) { IConfigurationSection tokenValidationSection = configuration.GetSection(tokenValidationSectionName); List<string> validTokenIssuers = tokenValidationSection.GetSection("ValidIssuers").Get<List<string>>(); List<string> audiences = tokenValidationSection.GetSection("Audiences").Get<List<string>>(); if (!tokenValidationSection.Exists()) { logger?.LogError("Missing configuration section '{tokenValidationSectionName}'. This section is required to be present in appsettings.json",tokenValidationSectionName); throw new InvalidOperationException($"Missing configuration section '{tokenValidationSectionName}'. This section is required to be present in appsettings.json"); } // If ValidIssuers is empty, default for ABS Public Cloud if (validTokenIssuers == null || validTokenIssuers.Count == 0) { validTokenIssuers = [ "https://api.botframework.com", "https://sts.windows.net/d6d49420-f39b-4df7-a1dc-d59a935871db/", "https://login.microsoftonline.com/d6d49420-f39b-4df7-a1dc-d59a935871db/v2.0", "https://sts.windows.net/f8cdef31-a31e-4b4a-93e4-5f571e91255a/", "https://login.microsoftonline.com/f8cdef31-a31e-4b4a-93e4-5f571e91255a/v2.0", "https://sts.windows.net/69e9b82d-4842-4902-8d1e-abc5b98a55e8/", "https://login.microsoftonline.com/69e9b82d-4842-4902-8d1e-abc5b98a55e8/v2.0", ]; string tenantId = tokenValidationSection["TenantId"]; if (!string.IsNullOrEmpty(tenantId)) { validTokenIssuers.Add(string.Format(CultureInfo.InvariantCulture, AuthenticationConstants.ValidTokenIssuerUrlTemplateV1, tenantId)); validTokenIssuers.Add(string.Format(CultureInfo.InvariantCulture, AuthenticationConstants.ValidTokenIssuerUrlTemplateV2, tenantId)); } } if (audiences == null || audiences.Count == 0) { throw new ArgumentException($"{tokenValidationSectionName}:Audiences requires at least one value"); } bool isGov = tokenValidationSection.GetValue("IsGov", false); bool azureBotServiceTokenHandling = tokenValidationSection.GetValue("AzureBotServiceTokenHandling", true); // If the `AzureBotServiceOpenIdMetadataUrl` setting is not specified, use the default based on `IsGov`. This is what is used to authenticate ABS tokens. string azureBotServiceOpenIdMetadataUrl = tokenValidationSection["AzureBotServiceOpenIdMetadataUrl"]; if (string.IsNullOrEmpty(azureBotServiceOpenIdMetadataUrl)) { azureBotServiceOpenIdMetadataUrl = isGov ? AuthenticationConstants.GovAzureBotServiceOpenIdMetadataUrl : AuthenticationConstants.PublicAzureBotServiceOpenIdMetadataUrl; } // If the `OpenIdMetadataUrl` setting is not specified, use the default based on `IsGov`. This is what is used to authenticate Entra ID tokens. string openIdMetadataUrl = tokenValidationSection["OpenIdMetadataUrl"]; if (string.IsNullOrEmpty(openIdMetadataUrl)) { openIdMetadataUrl = isGov ? AuthenticationConstants.GovOpenIdMetadataUrl : AuthenticationConstants.PublicOpenIdMetadataUrl; } TimeSpan openIdRefreshInterval = tokenValidationSection.GetValue("OpenIdMetadataRefresh", BaseConfigurationManager.DefaultAutomaticRefreshInterval); _ = services.AddAuthentication(options => { options.DefaultAuthenticateScheme = JwtBearerDefaults.AuthenticationScheme; options.DefaultChallengeScheme = JwtBearerDefaults.AuthenticationScheme; }) .AddJwtBearer(options => { options.SaveToken = true; options.TokenValidationParameters = new TokenValidationParameters { ValidateIssuer = true, ValidateAudience = true, // this option enables to validate the audience claim with audiences values ValidateLifetime = true, ClockSkew = TimeSpan.FromMinutes(5), ValidIssuers = validTokenIssuers, ValidAudiences = audiences, ValidateIssuerSigningKey = true, RequireSignedTokens = true, }; // Using Microsoft.IdentityModel.Validators options.TokenValidationParameters.EnableAadSigningKeyIssuerValidation(); options.Events = new JwtBearerEvents { // Create a ConfigurationManager based on the requestor. This is to handle ABS non-Entra tokens. OnMessageReceived = async context => { string authorizationHeader = context.Request.Headers.Authorization.ToString(); if (string.IsNullOrEmpty(authorizationHeader)) { // Default to AadTokenValidation handling // context.Options.TokenValidationParameters.ConfigurationManager ??= options.ConfigurationManager as BaseConfigurationManager; // await Task.CompletedTask.ConfigureAwait(false); // return; // // Fail the request when the token is empty context.Fail("Authorization header is missing."); logger?.LogWarning("Authorization header is missing."); return; } string[] parts = authorizationHeader?.Split(' '); if (parts.Length != 2 || parts[0] != "Bearer") { // Default to AadTokenValidation handling context.Options.TokenValidationParameters.ConfigurationManager ??= options.ConfigurationManager as BaseConfigurationManager; await Task.CompletedTask.ConfigureAwait(false); return; } Next, we should implement about "2. Deny access from unknown Entra ID tenants" We can retrieve the Tenant ID inside the MessageActivityAsync method of Bot/WeatherAgentBot.cs. Let’s extend the logic by referring to the following sample code to capture and utilize the Tenant ID within that method. https://github.com/OfficeDev/microsoft-teams-apps-company-communicator/blob/dcf3b169084d3fff7c1e4c5b68718fb33c3391dd/Source/CompanyCommunicator/Bot/CompanyCommunicatorBotFilterMiddleware.cs#L44 Here is how you can extend the logic to retrieve and use the Tenant ID within the MessageActivityAsync method: using MyM365Agent1.Bot.Agents; using Microsoft.Agents.Builder; using Microsoft.Agents.Builder.App; using Microsoft.Agents.Builder.State; using Microsoft.Agents.Core.Models; using Microsoft.SemanticKernel; using Microsoft.SemanticKernel.ChatCompletion; using Microsoft.Extensions.DependencyInjection.Extensions; namespace MyM365Agent1.Bot; public class WeatherAgentBot : AgentApplication { private WeatherForecastAgent _weatherAgent; private Kernel _kernel; private readonly string _tenantId; private readonly ILogger<WeatherAgentBot> _logger; public WeatherAgentBot(AgentApplicationOptions options, Kernel kernel, IConfiguration configuration, ILogger<WeatherAgentBot> logger) : base(options) { _kernel = kernel ?? throw new ArgumentNullException(nameof(kernel)); _logger = logger ?? throw new ArgumentNullException(nameof(logger)); OnConversationUpdate(ConversationUpdateEvents.MembersAdded, WelcomeMessageAsync); OnActivity(ActivityTypes.Message, MessageActivityAsync, rank: RouteRank.Last); // Get TenantId from TokenValidation section var tokenValidationSection = configuration.GetSection("TokenValidation"); _tenantId = tokenValidationSection["TenantId"]; } protected async Task MessageActivityAsync(ITurnContext turnContext, ITurnState turnState, CancellationToken cancellationToken) { // add validation of tenant ID var activity = turnContext.Activity; // Log: Received activity _logger.LogInformation("Received message activity from: {FromId}, AadObjectId:{AadObjectId}, TenantId: {TenantId}, ChannelId: {ChannelId}, ConversationType: {ConversationType}", activity?.From?.Id, activity?.From?.AadObjectId, activity?.Conversation?.TenantId, activity?.ChannelId, activity?.Conversation?.ConversationType); if (activity.ChannelId != "msteams" // Ignore messages not from Teams || activity.Conversation?.ConversationType?.ToLowerInvariant() != "personal" // Ignore messages from team channels or group chats || string.IsNullOrEmpty(activity.From?.AadObjectId) // Ignore if not an AAD user (e.g., bots, guest users) || (!string.IsNullOrEmpty(_tenantId) && !string.Equals(activity.Conversation?.TenantId, _tenantId, StringComparison.OrdinalIgnoreCase))) // Ignore if tenant ID does not match { _logger.LogWarning("Unauthorized serviceUrl detected: {ServiceUrl}. Expected to contain TenantId: {TenantId}", activity?.ServiceUrl, _tenantId); await turnContext.SendActivityAsync("Unauthorized service URL.", cancellationToken: cancellationToken); return; } // Setup local service connection ServiceCollection serviceCollection = [ new ServiceDescriptor(typeof(ITurnState), turnState), new ServiceDescriptor(typeof(ITurnContext), turnContext), new ServiceDescriptor(typeof(Kernel), _kernel), ]; // Start a Streaming Process await turnContext.StreamingResponse.QueueInformativeUpdateAsync("Working on a response for you"); ChatHistory chatHistory = turnState.GetValue("conversation.chatHistory", () => new ChatHistory()); _weatherAgent = new WeatherForecastAgent(_kernel, serviceCollection.BuildServiceProvider()); // Invoke the WeatherForecastAgent to process the message WeatherForecastAgentResponse forecastResponse = await _weatherAgent.InvokeAgentAsync(turnContext.Activity.Text, chatHistory); if (forecastResponse == null) { turnContext.StreamingResponse.QueueTextChunk("Sorry, I couldn't get the weather forecast at the moment."); await turnContext.StreamingResponse.EndStreamAsync(cancellationToken); return; } // Create a response message based on the response content type from the WeatherForecastAgent // Send the response message back to the user. switch (forecastResponse.ContentType) { case WeatherForecastAgentResponseContentType.Text: turnContext.StreamingResponse.QueueTextChunk(forecastResponse.Content); break; case WeatherForecastAgentResponseContentType.AdaptiveCard: turnContext.StreamingResponse.FinalMessage = MessageFactory.Attachment(new Attachment() { ContentType = "application/vnd.microsoft.card.adaptive", Content = forecastResponse.Content, }); break; default: break; } await turnContext.StreamingResponse.EndStreamAsync(cancellationToken); // End the streaming response } protected async Task WelcomeMessageAsync(ITurnContext turnContext, ITurnState turnState, CancellationToken cancellationToken) { foreach (ChannelAccount member in turnContext.Activity.MembersAdded) { if (member.Id != turnContext.Activity.Recipient.Id) { await turnContext.SendActivityAsync(MessageFactory.Text("Hello and Welcome! I'm here to help with all your weather forecast needs!"), cancellationToken); } } } } Since we’re at it, I’ve added various validations as well. I hope this will be helpful as a reference for everyone.Calculating Chargebacks for Business Units/Projects Utilizing a Shared Azure OpenAI Instance
Azure OpenAI Service is at the forefront of technological innovation, offering REST API access to OpenAI's suite of revolutionary language models, including GPT-4, GPT-35-Turbo, and the Embeddings model series. Enhancing Throughput for Scale As enterprises seek to deploy OpenAI's powerful language models across various business units, they often require granular control over configuration and performance metrics. To address this need, Azure OpenAI Service is introducing dedicated throughput, a feature that provides a dedicated connection to OpenAI models with guaranteed performance levels. Throughput is quantified in terms of tokens per second (tokens/sec), allowing organizations to precisely measure and optimize the performance for both prompts and completions. The model of provisioned throughput provides enhanced management and adaptability for varying workloads, guaranteeing system readiness for spikes in demand. This capability also ensures a uniform user experience and steady performance for applications that require real-time responses. Resource Sharing and Chargeback Mechanisms Large organizations frequently provision a singular instance of Azure OpenAI Service that is shared across multiple internal departments. This shared use necessitates an efficient mechanism for allocating costs to each business unit or consumer, based on the number of tokens consumed. This article delves into how chargeback is calculated for each business unit based on their token usage. Leveraging Azure API Management Policies for Token Tracking Azure API Management Policies offer a powerful solution for monitoring and logging the token consumption for each internal application. The process can be summarized in the following steps: ** Sample Code: Refer to this GitHub repository to get a step-by-step instruction on how to build the solution outlined below : private-openai-with-apim-for-chargeback 1. Client Applications Authorizes to API Management To make sure only legitimate clients can call the Azure OpenAI APIs, each client must first authenticate against Azure Active Directory and call APIM endpoint. In this scenario, the API Management service acts on behalf of the backend API, and the calling application requests access to the API Management instance. The scope of the access token is between the calling application and the API Management gateway. In API Management, configure a policy (validate-jwt or validate-azure-ad-token) to validate the token before the gateway passes the request to the backend. 2. APIM redirects the request to OpenAI service via private endpoint. Upon successful verification of the token, Azure API Management (APIM) routes the request to Azure OpenAI service to fetch response for completions endpoint, which also includes prompt and completion token counts. 3. Capture and log API response to Event Hub Leveraging the log-to-eventhub policy to capture outgoing responses for logging or analytics purposes. To use this policy, a logger needs to be configured in the API Management: # API Management service-specific details $apimServiceName = "apim-hello-world" $resourceGroupName = "myResourceGroup" # Create logger $context = New-AzApiManagementContext -ResourceGroupName $resourceGroupName -ServiceName $apimServiceName New-AzApiManagementLogger -Context $context -LoggerId "OpenAiChargeBackLogger" -Name "ApimEventHub" -ConnectionString "Endpoint=sb://<EventHubsNamespace>.servicebus.windows.net/;SharedAccessKeyName=<KeyName>;SharedAccessKey=<key>" -Description "Event hub logger with connection string" Within outbound policies section, pull specific data from the body of the response and send this information to the previously configured EventHub instance. This is not just a simple logging exercise; it is an entry point into a whole ecosystem of real-time analytics and monitoring capabilities: <outbound> <choose> <when condition="@(context.Response.StatusCode == 200)"> <log-to-eventhub logger-id="TokenUsageLogger">@{ var responseBody = context.Response.Body?.As<JObject>(true); return new JObject( new JProperty("Timestamp", DateTime.UtcNow.ToString()), new JProperty("ApiOperation", responseBody["object"].ToString()), new JProperty("AppKey", context.Request.Headers.GetValueOrDefault("Ocp-Apim-Subscription-Key",string.Empty)), new JProperty("PromptTokens", responseBody["usage"]["prompt_tokens"].ToString()), new JProperty("CompletionTokens", responseBody["usage"]["completion_tokens"].ToString()), new JProperty("TotalTokens", responseBody["usage"]["total_tokens"].ToString()) ).ToString(); }</log-to-eventhub> </when> </choose> <base /> </outbound> EventHub serves as a powerful fulcrum, offering seamless integration with a wide array of Azure and Microsoft services. For example, the logged data can be directly streamed to Azure Stream Analytics for real-time analytics or to Power BI for real-time dashboards With Azure Event Grid, the same data can also be used to trigger workflows or automate tasks based on specific conditions met in the incoming responses. Moreover, the architecture is extensible to non-Microsoft services as well. Event Hubs can interact smoothly with external platforms like Apache Spark, allowing you to perform data transformations or feed machine learning models. 4: Data Processing with Azure Functions An Azure Function is invoked when data is sent to the EventHub instance, allowing for bespoke data processing in line with your organization’s unique requirements. For instance, this could range from dispatching the data to Azure Monitor, streaming it to Power BI dashboards, or even sending detailed consumption reports via Azure Communication Service. [Function("TokenUsageFunction")] public async Task Run([EventHubTrigger("%EventHubName%", Connection = "EventHubConnection")] string[] openAiTokenResponse) { //Eventhub Messages arrive as an array foreach (var tokenData in openAiTokenResponse) { try { _logger.LogInformation($"Azure OpenAI Tokens Data Received: {tokenData}"); var OpenAiToken = JsonSerializer.Deserialize<OpenAiToken>(tokenData); if (OpenAiToken == null) { _logger.LogError($"Invalid OpenAi Api Token Response Received. Skipping."); continue; } _telemetryClient.TrackEvent("Azure OpenAI Tokens", OpenAiToken.ToDictionary()); } catch (Exception e) { _logger.LogError($"Error occured when processing TokenData: {tokenData}", e.Message); } } } In the example above, Azure function processes the tokens response data in Event Hub and sends them to Application Insights telemetry, and a basic Dashboard is configured in Azure, displaying the token consumption for each client application. This information can conveniently be used to compute chargeback costs. A sample query used in dashboard above that fetches tokens consumed by a specific client: customEvents | where name contains "Azure OpenAI Tokens" | extend tokenData = parse_json(customDimensions) | where tokenData.AppKey contains "your-client-key" | project Timestamp = tokenData.Timestamp, Stream = tokenData.Stream, ApiOperation = tokenData.ApiOperation, PromptTokens = tokenData.PromptTokens, CompletionTokens = tokenData.CompletionTokens, TotalTokens = tokenData.TotalTokens Azure OpenAI Landing Zone reference architecture A crucial detail to ensure the effectiveness of this approach is to secure the Azure OpenAI service by implementing Private Endpoints and using Managed Identities for App Service to authorize access to Azure AI services. This will limit access so that only the App Service can communicate with the Azure OpenAI service. Failing to do this would render the solution ineffective, as individuals could bypass the APIM/App Service and directly access the OpenAI Service if they get hold of the access key for OpenAI. Refer to Azure OpenAI Landing Zone reference architecture to build a secure and scalable AI environment. Additional Considerations If the client application is external, consider using an Application Gateway in front of the Azure APIM If "streaming" is set to true, tokens count is not returned in response. In that that case libraries like tiktoken (Python), orgpt-3-encoder(javascript) for most GPT-3 models can be used to programmatically calculate tokens count for the user prompt and completion response. A useful guideline to remember is that in typical English text, one token is approximately equal to around 4 characters. This equates to about three-quarters of a word, meaning that 100 tokens are roughly equivalent to 75 words. (P.S. Microsoft does not endorse or guarantee any third-party libraries.) A subscription key or a custom header like app-key can also be used to uniquely identify the client as appId in OAuth token is not very intuitive. Rate-limiting can be implemented for incoming requests using OAuth tokens or Subscription Keys, adding another layer of security and resource management. The solution can also be extended to redirect different clients to different Azure OpenAI instances. For example., some clients utilize an Azure OpenAI instance with default quotas, whereas premium clients get to consume Azure Open AI instance with dedicated throughput. Conclusion Azure OpenAI Service stands as an indispensable tool for organizations seeking to harness the immense power of language models. With the feature of provisioned throughput, clients can define their usage limits in throughput units and freely allocate these to the OpenAI model of their choice. However, the financial commitment can be significant and is dependent on factors like the chosen model's type, size, and utilization. An effective chargeback system offers several advantages, such as heightened accountability, transparent costing, and judicious use of resources within the organization.
What's New in Azure App Service at #MSBuild 2025
New App Service Premium v4 plan The new App Service Premium v4 (Pv4) plan has entered public preview at Microsoft Build 2025 for both Windows and Linux! This new plan is designed to support today's highly demanding application performance, scale, and budgets. Built on the latest "v6" general-purpose virtual machines and memory-optimized x64 Azure hardware with faster processors and NVMe temporary storage, it provides a noticeable performance uplift over prior generations of App Service Premium plans (over 25% in early testing). The Premium v4 offering includes nine new sizes ranging from P0v4 with a single virtual CPU and 4GB RAM all the way up through P5mv4, with 32 virtual CPUs and 256GB RAM, providing CPU and memory options to meet any business need. App Service Premium v4 plans provide attractive price-performance across the entire performance curve for both Windows and Linux customers. Premium v4 customers using pay-as-you-go (PAYG) on Azure App Service for Windows can expect to save up to 24% compared with prior Premium plans. We plan to provide deeper commitment-based discounts such as reserved instances and savings plan at GA. For more detailed pricing on the various CPU and memory options, see the pricing pages for Windows and Linux as well as the Azure Pricing Calculator. App Service currently has Pv4 deployed in a few regions with more regions being regularly added. For more details on how to configure app service plans with Premium v4 as well as a regularly updated list of regional availability, see the product documentation and start taking advantage of faster performance today! 2-zone Availability Zone support is now generally available With a recently completed platform update in May, customers now enjoy the 99.99% Availability Zone (AZ) SLA when running on only two instances (instead of three)! As part of this update more parts of the App Service footprint have enabled AZ support “in place”, which means many existing app service plans can now also use Availability Zones. Availability Zone configuration for app service plans is also now mutable. This means if an app service plan is running on an AZ-enabled part of the App Service footprint, customers can choose to enable and disable Availability Zone support at any time. Read more about the new Availability Zone options in the announcement article! ARM/CLI surface area for Availability Zone support has also been updated to provide increased visibility into AZ configuration details. The same enhanced visibility is also coming to the Azure Portal in June. With these changes customers can determine if an App Service plan is on an AZ-enabled scale unit, as well as how many zones are available for zone spanning. This allows customers to deploy with either two zones, or three zones (where available), of zone spanning for their App Service plans. For App Service plans that are AZ-enabled, customers will also be able to see the physical zone placement of each AZ enabled App Service plan. Availability Zone support is available on the new Premium v4 plan, and also supported with Premium v2, Premium v3, and the dedicated App Service Environment v3 (Isolated V2 plan). Check out the Availability Zone options for your App Service plans and start getting the benefits of zone resiliency today! .NET Aspire on Azure App Service .NET Aspire support is now available in public preview for App Service on Linux! .NET Aspire developers creating applications have an additional deployment option with App Service as a deployment target. Developers can create multi-app/multi-service .NET Aspire applications locally and deploy them into Azure using the new App Service deployment provider. The App Service and .NET Aspire teams worked together to create an App Service “provider” using .NET Aspire’s new “provider model”. The build provider translates the code-centric view of a .NET Aspire application topology into an Azure deployment mapped onto App Service constructs. The App Service provider supports securely deploying multiple .NET Aspire applications, with observability via the familiar .NET Aspire dashboard coming in the near future. The Getting Started with .NET Aspire on Azure App Service blog has instructions on how to create a .NET Aspire project for deployment onto App Service, as well a link for providing feedback. If you happen to be at Build 2025, drop by our booth or the theatre session “DEM548: How .NET Aspire on App Service enhances modern app development” to see live demonstrations of the App Service support for .NET Aspire! Using App Service to build agentic AI apps The last few months of intelligent app development have seen a frenetic pace of change with the rapid evolution of agents on Azure AI Foundry Agent Service and new agent extensibility options like Model Context Protocol (MCP) opening avenues for integrating existing data sources and APIs into agentic architectures. Here's a quick run-down of useful resources published recently: This article demonstrates hosting a remote MCP server on Azure App Service. The sample is an adaptation of the weather service example from the MCP site. The App Service variation also includes an azd template for easy experimentation via a CLI deployment to App Service! This article walks through integrating a .NET Core implementation of a “To-Do” list API running on App Service with an agent created on Azure AI Foundry Agent Service. It’s a straightforward example demonstrating how developers can bring together the power of AI agents with existing web API investments. Quick start guides for using App Service with Azure Open AI in your language of choice -- Python, Node, .NET, and Java. Using Microsoft Research’s latest 1-bit “super-small” language model, BitNet on App Service. Enhance search queries on text data stored in Azure SQL DB using natural language vector functions and Azure App Service. Includes an accompanying azd example. How to use Azure AI Search hybrid search capabilities from App Service with .NET (Blazor), Java (Spring Boot), Node (Express), or Python (FastAPI). Use GitHub Copilot to compare your application’s bicep against a representative “best practices” bicep definition and then generate the necessary bicep diff. In addition, using Sidecar for App Service on Linux, developers can easily connect Phi SLMs to their applications. Examples using the chat completion endpoint in the SLM sidecar extensions are available in this GitHub repo with code examples for .NET, Node, Python and Java. There are also accompanying docs for .NET, Node, Python (FastAPI) and Java (Spring Boot) which go into more details on using the SLM sidecar extensions. The sidecar extensions capability is also now enabled in the Azure Portal. AI Labs at Microsoft Build For those of you attending Microsoft Build in person, we will have labs for additional hands-on experience using AI with Azure App Service. LAB347: Add AI experiences to existing .NET apps using Sidecar in App Service This lab (first lab occurrence and second lab occurrence - see Exercise 4) covers an e-commerce inventory API (written in .NET) integrated with an agent running on Azure AI Foundry Agent Service. When a customer interacts with the AI agent it automatically invokes the appropriate web APIs to fetch real-time inventory information, add/remove products in a shopping cart, and increment/decrement product inventory. This is a great example of an AI powered agent grounded in a company’s ever changing transactional data. As a fun sidenote, GitHub Copilot was used extensively to build >95% of the sample application as well as to generate the OpenAPI specification that integrates the inventory web API with the AI agent! The same AI-on-App Service lab (Exercise 1) walks developers through integrating a basic Azure OpenAI chat interface into a web application. The lab also demonstrates using a background WebJob on Linux with Azure OpenAI (Exercise 2) to categorize user sentiment for product reviews. The lab also shows (Exercise 3) how to use a small language model (SLM) like Microsoft’s Phi-4 model in a WebJob to perform similar categorization, without the need to call out to an LLM. Although SLMs are not as powerful as LLMs, SLMs are an interesting alternative for integrating AI functionality where either cost, or control over AI data flows, are considerations. Azure SRE Agent for App Service One of the big announcements at Build this year was the Agentic DevOps announcement, which includes the new Azure SRE Agent. Designed to empower Site Reliability Engineers (SRE), the SRE Agent is a new agentic service that can manage Azure application platform services. including App Service, Functions, and Azure Container Apps to name just a few. It provides automatic incident response and mitigation, faster root cause analysis (RCA) of production issues, and continuous monitoring of application health and performance. With SRE Agent, you can use a natural language interface for managing your web applications on Azure App Service. To be an early adopter of the Agentic DevOps revolution, check out the announcement blog and sign-up to join the SRE Agent preview as it starts rolling out! WebJobs for App Service on Linux (GA) WebJobs for App Service on Linux just recently GA’d earlier this month. With this functionality developers can implement the same “infra-glue” style of background jobs that they have enjoyed with App service on Windows. Take a look at the documentationdemonstrating WebJobs support for shell scripts, Python, Java, .NET and Node on Linux! As mentioned earlier, the AI-on-App Service lab at this year’s Build conference has two code examples (see Exercise 2 and Exercise 3) demonstrating Linux WebJobs with Azure OpenAI, as well as a locally connected Phi-4 Small Language Model (SLM) sidecar, to categorize user sentiment for submitted product reviews. These are great examples of creatively using WebJobs to perform background batch-style work with your AI resources. Also keep an eye out for the upcoming WebJobs for Windows Containers GA which is planned “soon” this summer! Language and Framework Updates In addition to the release of .NET Aspire support for App Service, the App Service team has kept busy updating myriad Node, Python, Java/JBoss, .NET and PHP versions. To give an idea of the scope of effort keeping language and framework versions up to date across both Windows and Linux, App Service released more than two dozen language/framework specific updates in the last few weeks prior to Build. That represents the ongoing platform commitment to keeping languages regularly updated without the need for developers to explicitly invest time and effort doing so themselves. Just last month, Strapi support was introduced for App Service on Linux! Strapi is an open source headless Javascript based content management system that provides developers a robust platform for developing and delivering content across a variety of formats. The Azure Marketplace Strapi offering provides customization control, global availability and pre-built integration to essential Azure services like Azure Database for MySQL or PostgreSQL and Azure Email Communication Services. Deep dive on the details of hosting Strapi on App Service in this article. The custom error pages feature for App Service has also been updated just prior to Build. Custom error pages enable developers to customize the response rendered for common HTTP errors (403, 502 and 503) which are returned by the platform. This release includes a new option to always render custom errors regardless of whether the HTTP error was platform generated, or application generated. There will also be an Azure Portal update coming in June with support for the new custom error page features! Looking ahead to summer, stay tuned for the impending arrival of .NET 10 preview bits on App Service across both Windows and Linux! Networking and ASE Updates App Service support for public inbound IPv6 traffic is availablein most regions in public preview, with the service working towards a planned GA of inbound IPv6 support during the summer. Inbound IPv6 is supported for both IPv6-only upstream clients, as well dual-stack scenarios where a web application is reachable over either an IPv4 address or an IPv6 address. As part of an upcoming summer release, App Service will be delivering a public preview of *outbound* IPv6 traffic. For details on using IPv6 on App Service, as well to track all of the upcoming updates, consult this article: Announcing inbound IPv6 support in public preview - Azure App Service. For App Service Environment (ASE) customers, App Service will soon be releasing new support for adding custom Certificate Authorities (CAs) to an ASE. This new support will enable securing inbound TLS traffic using certificates issued by a custom Certificate Authority. Hybrid Connections customers will be happy to see that a new version of the App Service Hybrid Connection Manager (HCM) was just released just a few weeks ago. The new HCM delivers updated UX support for both Linux and Windows customers, enhanced logging and connection testing, and a brand new CLI for scripting and command-line management of Hybrid Connections! You might have missed it, but there was a recent addition to the troubleshooting options on App Service with the new Network Troubleshooter! The Network Troubleshooter offers comprehensive analysis and actionable insights to resolve connectivity failures for both Linux and Windows web apps. It tests connectivity to Azure resources like Storage, Redis, SQL Server, MySQL server, and other apps running on App Service. It diagnoses connectivity problems with Private endpoints, Service endpoints, and Internet-based endpoints, detects NAT gateways, and investigates DNS failures with custom DNS servers. Additionally, it provides actionable recommendations and surfaces any network rules it finds that are blocking connectivity. If you regularly wrestle with connectivity challenges, give the Network Troubleshooter a try! Next Steps Developers can learn more about Azure App Service at Getting Started with Azure App Service. Stay up to date on new features and innovations on Azure App Service via Azure Updates as well as the Azure App Service (@AzAppService) X feed. There is always a steady stream of great deep-dive technical articles about App Service as well as the breadth of developer focused Azure services over on the Apps on Azure blog. And lastly take a look at Azure App Service Community Standups hosted on the Microsoft Azure Developers YouTube channel. The Azure App Service Community Standup series regularly features walkthroughs of new and upcoming features from folks that work directly on the product! Build 2025 Session Reference (Note: all times below are listed in Seattle time - Pacific Daylight Time) (Note: some labs have more than one timeslot spanning multiple days) Innovate, deploy, & optimize your apps without infrastructure hassles https://build.microsoft.com/en-US/sessions/BRK201 Monday, May 19 th 11:15 AM – 12:15 PM Pacific Daylight Time Arch, 705 Pike, Level 6, Room 606 Breakout, Streaming Online and Recorded Session (BRK201) Quickly build, deploy, and scale web apps and APIs globally with App Service https://build.microsoft.com/en-US/sessions/BRK200 Tuesday, May 20 th 11:45 AM – 12:45 PM Pacific Daylight Time Arch, 705 Pike, Level 6, Room 608 Breakout, Streaming Online and Recorded Session (BRK200) Simplifying .NET upgrades with GitHub Copilot https://build.microsoft.com/en-US/sessions/DEM549 Monday, May 19 th 5:05 PM - 5:20 PM Pacific Daylight Time Arch, 705 Pike, Level 4, Hub, Theater B Demo Session – Also Recorded (DEM549) Use Azure SRE Agent to automate tasks and increase site reliability https://build.microsoft.com/en-US/sessions/DEM550 Tuesday, May 20 th 5:10 PM - 5:25 PM Pacific Daylight Time Arch, 705 Pike, Level 4, Hub, Theater A Demo Session – Also Recorded (DEM550) How .NET Aspire on App Service enhances modern app development https://build.microsoft.com/en-US/sessions/DEM548 Wednesday, May 21 st 2:00 PM - 2:15 PM Pacific Daylight Time Arch, 705 Pike, Level 4, Hub, Theater B Demo Session – Also Recorded (DEM548) Add AI experiences to existing .NET apps using Sidecars in App Service [Note: Lab participants will be able to try Phi-4 and Azure AI Foundry Agent service scenarios in this lab.] https://build.microsoft.com/en-US/sessions/LAB347 Monday, May 19 th 4:45 PM - 6:00 PM Pacific Daylight Time Arch, 800 Pike, Level 1, Yakima 1 Hands on Lab – In-Person Only (LAB347) You can also work through the lab with your own Azure subscription! Code is available at https://github.com/Azure-Samples/Build2025-LAB347. Deploy the lab resources using the included resource provisioning template (https://github.com/Azure-Samples/Build2025-LAB347/blob/main/resources/lab347.json). You can deploy the template by searching on “Deploy a custom template” in the Azure Portal, and copying and pasting the template into the “Build your own template in the editor option”! Add AI experiences to existing .NET apps using Sidecars in App Service [Note: Lab participants will be able to try Phi-4 and Azure AI Foundry Agent service scenarios in this lab.] https://build.microsoft.com/en-US/sessions/LAB347-R1 Wednesday, May 21 st 4:30 PM - 5:45 PM Pacific Daylight Time Arch, 800 Pike, Lower Level, Skagit 5 Hands on Lab – In-Person Only (LAB347-R1) You can also work through the lab with your own Azure subscription! Code is available at https://github.com/Azure-Samples/Build2025-LAB347. Deploy the lab resources using the included resource provisioning template (https://github.com/Azure-Samples/Build2025-LAB347/blob/main/resources/lab347.json). You can deploy the template by searching on “Deploy a custom template” in the Azure Portal, and copying and pasting the template into the “Build your own template in the editor option”! Modernizing .NET Applications using Azure Migrate and GitHub Copilot https://build.microsoft.com/en-US/sessions/LAB343 Tuesday, May 20 th 5:15 PM - 6:30 PM Pacific Daylight Time Arch, 800 Pike, Level 1, Yakima 1 Hands on Lab – In-Person Only (LAB343) Modernizing .NET Applications using Azure Migrate and GitHub Copilot https://build.microsoft.com/en-US/sessions/LAB343-R1 Thursday, May 22 nd 10:15 AM – 11:30 AM Pacific Daylight Time Arch, 800 Pike, Level 2, Chelan 2 Hands on Lab – In-Person Only (LAB343-R1)Building the Agentic Future
As a business built by developers, for developers, Microsoft has spent decades making it faster, easier and more exciting to create great software. And developers everywhere have turned everything from BASIC and the .NET Framework, to Azure, VS Code, GitHub and more into the digital world we all live in today. But nothing compares to what’s on the horizon as agentic AI redefines both how we build and the apps we’re building. In fact, the promise of agentic AI is so strong that market forecasts predict we’re on track to reach 1.3 billion AI Agents by 2028. Our own data, from 1,500 organizations around the world, shows agent capabilities have jumped as a driver for AI applications from near last to a top three priority when comparing deployments earlier this year to applications being defined today. Of those organizations building AI agents, 41% chose Microsoft to build and run their solutions, significantly more than any other vendor. But within software development the opportunity is even greater, with approximately 50% of businesses intending to incorporate agentic AI into software engineering this year alone. Developers face a fascinating yet challenging world of complex agent workflows, a constant pipeline of new models, new security and governance requirements, and the continued pressure to deliver value from AI, fast, all while contending with decades of legacy applications and technical debt. This week at Microsoft Build, you can see how we’re making this future a reality with new AI-native developer practices and experiences, by extending the value of AI across the entire software lifecycle, and by bringing critical AI, data, and toolchain services directly to the hands of developers, in the most popular developer tools in the world. Agentic DevOps AI has already transformed the way we code, with 15 million developers using GitHub Copilot today to build faster. But coding is only a fraction of the developer’s time. Extending agents across the entire software lifecycle, means developers can move faster from idea to production, boost code quality, and strengthen security, while removing the burden of low value, routine, time consuming tasks. We can even address decades of technical debt and keep apps running smoothly in production. This is the foundation of agentic DevOps—the next evolution of DevOps, reimagined for a world where intelligent agents collaborate with developer teams and with each other. Agents introduced today across GitHub Copilot and Azure operate like a member of your development team, automating and optimizing every stage of the software lifecycle, from performing code reviews, and writing tests to fixing defects and building entire specs. Copilot can even collaborate with other agents to complete complex tasks like resolving production issues. Developers stay at the center of innovation, orchestrating agents for the mundane while focusing their energy on the work that matters most. Customers like EY are already seeing the impact: “The coding agent in GitHub Copilot is opening up doors for each developer to have their own team, all working in parallel to amplify their work. Now we're able to assign tasks that would typically detract from deeper, more complex work, freeing up several hours for focus time." - James Zabinski, DevEx Lead at EY You can learn more about agentic DevOps and the new capabilities announced today from Amanda Silver, Corporate Vice President of Product, Microsoft Developer Division, and Mario Rodriguez, Chief Product Office at GitHub. And be sure to read more from GitHub CEO Thomas Dohmke about the latest with GitHub Copilot. At Microsoft Build, see agentic DevOps in action in the following sessions, available both in-person May 19 - 22 in Seattle and on-demand: BRK100: Reimagining Software Development and DevOps with Agentic AI BRK 113: The Agent Awakens: Collaborative Development with GitHub Copilot BRK118: Accelerate Azure Development with GitHub Copilot, VS Code & AI BRK131: Java App Modernization Simplified with AI BRK102: Agent Mode in Action: AI Coding with Vibe and Spec-Driven Flows BRK101: The Future of .NET App Modernization Streamlined with AI New AI Toolchain Integrations Beyond these new agentic capabilities, we’re also releasing new integrations that bring key services directly to the tools developers are already using. From the 150 million GitHub users to the 50 million monthly users of the VS Code family, we’re making it easier for developers everywhere to build AI apps. If GitHub Copilot changed how we write code, Azure AI Foundry is changing what we can build. And the combination of the two is incredibly powerful. Now we’re bringing leading models from Azure AI Foundry directly into your GitHub experience and workflow, with a new native integration. GitHub models lets you experiment with leading models from OpenAI, Meta, Cohere, Microsoft, Mistral and more. Test and compare performance while building models directly into your codebase all within in GitHub. You can easily select the best model performance and price side by side and swap models with a simple, unified API. And keeping with our enterprise commitment, teams can set guardrails so model selection is secure, responsible, and in line with your team’s policies. Meanwhile, new Azure Native Integrations gives developers seamless access to a curated set of 20 software services from DataDog, New Relic, Pinecone, Pure Storage Cloud and more, directly through Azure portal, SDK, and CLI. With Azure Native Integrations, developers get the flexibility to work with their preferred vendors across the AI toolchain with simplified single sign-on and management, while staying in Azure. Today, we are pleased to announce the addition of even more developer services: Arize AI: Arize’s platform provides essential tooling for AI and agent evaluation, experimentation, and observability at scale. With Arize, developers can easily optimize AI applications through tools for tracing, prompt engineering, dataset curation, and automated evaluations. Learn more. LambdaTest HyperExecute: LambdaTest HyperExecute is an AI-native test execution platform designed to accelerate software testing. It enables developers and testers to run tests up to 70% faster than traditional cloud grids by optimizing test orchestration, observability and streamlining TestOps to expedite release cycles. Learn more. Mistral: Mistral and Microsoft announced a partnership today, which includes integrating Mistral La Plateforme as part of Azure Native Integrations. Mistral La Plateforme provides pay-as-you-go API access to Mistral AI's latest large language models for text generation, embeddings, and function calling. Developers can use this AI platform to build AI-powered applications with retrieval-augmented generation (RAG), fine-tune models for domain-specific tasks, and integrate AI agents into enterprise workflows. MongoDB (Public Preview): MongoDB Atlas is a fully managed cloud database that provides scalability, security, and multi-cloud support for modern applications. Developers can use it to store and search vector embeddings, implement retrieval-augmented generation (RAG), and build AI-powered search and recommendation systems. Learn more. Neon: Neon Serverless Postgres is a fully managed, autoscaling PostgreSQL database designed for instant provisioning, cost efficiency, and AI-native workloads. Developers can use it to rapidly spin up databases for AI agents, store vector embeddings with pgvector, and scale AI applications seamlessly. Learn more. Java and .Net App Modernization Shipping to production isn’t the finish line—and maintaining legacy code shouldn’t slow you down. Today we’re announcing comprehensive resources to help you successfully plan and execute app modernization initiatives, along with new agents in GitHub Copilot to help you modernize at scale, in a fraction of the time. In fact, customers like Ford China are seeing breakthrough results, reducing up to 70% of their Java migration efforts by using GitHub Copilot to automate middleware code migration tasks. Microsoft’s App Modernization Guidance applies decades of enterprise apps experience to help you analyze production apps and prioritize modernization efforts, while applying best practices and technical patterns to ensure success. And now GitHub Copilot transforms the modernization process, handling code assessments, dependency updates, and remediation across your production Java and .NET apps (support for mainframe environments is coming soon!). It generates and executes update plans automatically, while giving you full visibility, control, and a clear summary of changes. You can even raise modernization tasks in GitHub Issues from our proven service Azure Migrate to assign to developer teams. Your apps are more secure, maintainable, and cost-efficient, faster than ever. Learn how we’re reimagining app modernization for the era of AI with the new App Modernization Guidance and the modernization agent in GitHub Copilot to help you modernize your complete app estate. Scaling AI Apps and Agents Sophisticated apps and agents need an equally powerful runtime. And today we’re advancing our complete portfolio, from serverless with Azure Functions and Azure Container Apps, to the control and scale of Azure Kubernetes Service. At Build we’re simplifying how you deploy, test, and operate open-source and custom models on Kubernetes through Kubernetes AI Toolchain Operator (KAITO), making it easy to inference AI models with the flexibility, auto-scaling, pay-per-second pricing, and governance of Azure Container Apps serverless GPU, helping you create real-time, event-driven workflows for AI agents by integrating Azure Functions with Azure AI Foundry Agent Service, and much, much more. The platform you choose to scale your apps has never been more important. With new integrations with Azure AI Foundry, advanced automation that reduces developer overhead, and simplified operations, security and governance, Azure’s app platform can help you deliver the sophisticated, secure AI apps your business demands. To see the full slate of innovations across the app platform, check out: Powering the Next Generation of AI Apps and Agents on the Azure Application Platform Tools that keep pace with how you need to build This week we’re also introducing new enhancements to our tooling to help you build as fast as possible and explore what’s next with AI, all directly from your editor. GitHub Copilot for Azure brings Azure-specific tools into agent mode in VS Code, keeping you in the flow as you create, manage, and troubleshoot cloud apps. Meanwhile the Azure Tools for VS Code extension pack brings everything you need to build apps on Azure using GitHub Copilot to VS Code, making it easy to discover and interact with cloud services that power your applications. Microsoft’s gallery of AI App Templates continues to expand, helping you rapidly move from concept to production app, deployed on Azure. Each template includes fully working applications, complete with app code, AI features, infrastructure as code (IaC), configurable CI/CD pipelines with GitHub Actions, along with an application architecture, ready to deploy to Azure. These templates reflect the most common patterns and use cases we see across our AI customers, from getting started with AI agents to building GenAI chat experiences with your enterprise data and helping you learn how to use best practices such as keyless authentication. Learn more by reading the latest on Build Apps and Agents with Visual Studio Code and Azure Building the agentic future The emergence of agentic DevOps, the new wave of development powered by GitHub Copilot and new services launching across Microsoft Build will be transformative. But just as we’ve seen over the first 50 years of Microsoft’s history, the real impact will come from the global community of developers. You all have the power to turn these tools and platforms into advanced AI apps and agents that make every business move faster, operate more intelligently and innovate in ways that were previously impossible. Learn more and get started with GitHub CopilotReimagining App Modernization for the Era of AI
This blog highlights the key announcements and innovations from Microsoft Build 2025. It focuses on how AI is transforming the software development lifecycle, particularly in app modernization. Key topics include the use of GitHub Copilot for accelerating development and modernization, the introduction of Azure SRE agent for managing production systems, and the launch of the App Modernization Guidance to help organizations modernize their applications with AI-first design. The blog emphasizes the strategic approach to modernization, aiming to reduce complexity, improve agility, and deliver measurable business outcomes
Streamline & Modernise ASP.NET Auth: Moving enterprise apps from IIS to App Service with Easy Auth
Introduction When modernising your enterprise ASP.NET (.NET Framework) or ASP.NET Core applications and moving them from IIS over to Azure App Service, one of the aspects you will have to take into consideration is how you will manage authentication (AuthN) and authorisation (AuthZ). Specifically, for applications that leverage on-premises auth mechanisms such as Integrated Windows Authentication, you will need to start considering more modern auth protocols such as OpenID Connect/OAuth which are more suited to the cloud. Fortunately, App Service includes built-in authentication and authorisation support also known as 'Easy Auth', which requires minimal to zero code changes. This feature is integrated into the platform, includes a built-in token store, and operates as a middleware running the AuthN logic outside of your code logic, as illustrated by the image below:- More information on how EasyAuth works can be found here. Easy Auth supports several providers as illustrated above, but in this blog we will purely focus on using Entra ID (formerly known as Azure Active Directory) as the provider. It also assumes all of the Active Directory users have been synced up to Entra ID. With a few clicks, you can enable Entra ID authentication across any of your web, mobile or API apps in App Service, restrict which tenants or even specific identities are allowed access – all without touching code. This can be quite powerful in many scenarios, such as when you do not have access to the source control to implement your own auth logic, reducing the maintenance overhead of maintaining libraries or simply want a quick path to apply auth across your apps. For more detailed scenarios and comparison on when it makes sense using Easy Auth versus other authentication methods can be found here. Setting up Easy Auth Let’s see Easy Auth in action. As you can see below I have a sample ASP.NET app hosted on App Service which is accessible without any authentication:- Now let’s demonstrate how quickly it is to setup Easy Auth for my app:- 1) I navigated to my App Service resource within the Azure Portal 2) I went to Authentication and used the below configuration o Selected Microsoft as the Identity provider o Workforce configuration (current tenant) o Create a new app registration (appropriate Entra ID roles are required) o Entra ID app name:- sot-easyauth o Client secret expiry:- 180 days (this means I must renew the secret in advance of the 180 days otherwise my app/authentication will fail to function upon expiry causing downtime). o Allow requests only from this application itself. o Current tenant – Single tenant (i.e users outside of my tenant will be denied access) o Identity requirement:- Allow requests from any identity o Restrict access:- Require authentication (this will require authentication across my whole app, whereas “Allow unauthenticated access” means it is up to my app to decide when authentication is required). o HTTP 302 Found redirect (redirects unauthenticated users to the login page rather than just a page stating 401 unauthorised for example). o Token store:- Enabled (also allows the app to have access to the token). 3) For permissions, I left the default User.Read Graph API permission selected. More information on the different permissions can be found here. Now if I go back to the app and refresh the page again, I am redirected to the login page which is surfaced by the Easy Auth middleware:- Only after successful authentication, will I be able to see the Welcome page again:- Now that is pretty impressive, but you might want to go even further and have questions such as: how will my app know who’s logged in? How can I access the claims? How do I perform more granular AuthZ? Well for starters, Easy Auth essentially creates the claims in the incoming token and exposes them to your app as request headers which your app can then leverage to interpret accordingly. The list of headers can be found here. Typically, you will be tasked with creating custom logic to decode and interpret these claims, but with the likes of ASP.NET (.NET Framework), App Service can populate the claims of the authenticated user without additional code logic. However for ASP.NET Core this does not hold true. Thus, given the approach differs between ASP.NET (.NET Framework) and ASP.NET Core (starting from .NET Core), I will split these up into two different sections after touching upon AuthZ and Entra ID app roles. AuthZ and Entra ID App Roles If your IIS ASP.NET app leverages Windows Authentication for AuthN, but your app manages AuthZ itself, perhaps by mapping the domain credentials (e.g. CONTOSO\Sam) to specific AuthZ roles stored in somewhere like a database and remains a requirement to do, you can achieve a similar outcome by using the claims provided by Easy Auth. However, it is not recommended to use fields such as domain credentials, e-mail or UPN (e.g. dan.long@contoso.com) given such attributes can change and even be re-used over time. For example, an employee called Dan Long has the UPN of dan.long@contoso.com leaves the company and another employee with the same name joins the company and is assigned the same UPN dan.long@contoso.com – potentially giving unauthorised access to resources belonging to the former employee. Instead you may consider using the oid (i.e objectId), which is a globally unique GUID that identifies the user across applications within a single tenant. You might also consider pairing oid with tid (i.e tenant ID) for sharding/routing if required. A note for multi-tenancy applications: the same user that exists in different tenants will have a different oid. More information on how to reliably identify a user and the different claim types available can be found here. Alternatively, if the built-in authorisation policies do not suffice, you can leverage Entra ID app roles to apply authorisation within your ASP.NET App, which we will cover in more depth further down below. For demonstration purposes, I have created an app role called Member in my Entra ID App Registration and assigned the Entra ID group “Contoso EA Members” to this role via the associated Entra ID Enterprise Application, which my identity is part of as shown below:- I am leveraging said role to restrict only the role Member from being able to access the Member Area page (more on this further down). More information on creating your own Entra ID app roles can be found here. ASP.NET (.NET Framework) claims and Entra ID App roles For ASP.NET 4.6 apps, App Service populates the user’s claims through ClaimsPrincipal.Current, which means you can easily reference the claims without additional logic. I have created sample code which demonstrates this here, and the output of this in App Service can be found below:- You will notice Easy Auth has picked up my Entra ID app role called Member under claim type roles. In the screenshot and sample, you will notice I have a link located on the top nav bar called Member Area which is guarded by an [Authorize] tag to restrict only members with the role Member access. Unfortunately, at this stage, if we were to access the page it will return with 401 Access Denied, regardless of my identity having the appropriate app role. The reason behind this, is because ASP.NET is looking for the claim type “http://schemas.microsoft.com/ws/2008/06/identity/claims/role” instead of “role”. Fortunately, Easy Auth can be configured to display the long claim name instead by configuring the Environment Variable WEBSITE_AUTH_USE_LEGACY_CLAIMS to False, as shown in the below screenshot:- After the change, if I logout and back in again, I will see this being reflected back into my application and the Member Area page will grant me access as shown in the screenshots below:- Voila, we now have claims and Entra ID app roles working within our ASP.NET application. ASP.NET Core claims and Entra ID App roles Out-of-the-box, Easy Auth does not support populating ASP.NET Core with the current user’s authentication context like it does for ASP.NET (.NET Framework) with ClaimsPrincipal. However, this can be achieved by using the nuget package Microsoft.Identity.Web which has built-in capability to achieve this. What I did was as follows:- 1) Installed the nuget package Microsoft.Identity.Web into my solution. 2) In my Program.cs file, I loaded in the library:- builder.Services.AddMicrosoftIdentityWebAppAuthentication(builder.Configuration); 3) I also added app.UseAuthorization() after app.UseAuthentication() app.UseAuthentication(); After these changes, User.Identities will now be populated with the claims, and the [Authorize] tag will work permitting only the role Member when visiting the Member Area page. The full sample code can be found here. Unlike with ASP.NET (.NET Framework), the downside with this approach is the added responsibility of managing an additional library (Microsoft.Identity.Web). Conclusion App Service Easy Auth can provide a streamlined and efficient way to manage authentication and authorisation for your ASP.NET applications. By leveraging Easy Auth, you can apply modern auth protocols with minimal to zero code changes. The built-in support for various identity providers, including Entra ID, can help developers implement flexible and robust auth mechanisms. As demonstrated, Easy Auth simplifies the process of integrating authentication and authorisation into your applications, making it an valuable tool for modernising enterprise apps. Good to know and additional resources Limitations of Entra ID app roles. For example “A user, group, or service principal can have a maximum of 1,500 app role assignment”:- Service limits and restrictions - Microsoft Entra ID | Microsoft Learn. You can leverage Azure Policy to audit across the organisation when App Service does not have Easy Auth enabled by turning on “App Service apps should have authentication enabled”:- Built-in policy definitions for Azure App Service - Azure App Service | Microsoft Learn. Work with User Identities in AuthN/AuthZ - Azure App Service | Microsoft Learn Configure Microsoft Entra Authentication - Azure App Service | Microsoft Learn Work with OAuth Tokens in AuthN/AuthZ - Azure App Service | Microsoft LearnKeep Your Azure Functions Up to Date: Identify Apps Running on Retired Versions
Running Azure Functions on retired language versions can lead to security risks, performance issues, and potential service disruptions. While Azure Functions Team notifies users about upcoming retirements through the portal, emails, and warnings, identifying affected Function Apps across multiple subscriptions can be challenging. To simplify this, we’ve provided Azure CLI scripts to help you: ✅ Identify all Function Apps using a specific runtime version ✅ Find apps running on unsupported or soon-to-be-retired versions ✅ Take proactive steps to upgrade and maintain a secure, supported environment Read on for the full set of Azure CLI scripts and instructions on how to upgrade your apps today! Why Upgrading Your Azure Functions Matters Azure Functions supports six different programming languages, with new stack versions being introduced and older ones retired regularly. Staying on a supported language version is critical to ensure: Continued access to support and security updates Avoidance of performance degradation and unexpected failures Compliance with best practices for cloud reliability Failure to upgrade can lead to security vulnerabilities, performance issues, and unsupported workloads that may eventually break. Azure's language support policy follows a structured deprecation timeline, which you can review here. How Will You Know When a Version Is Nearing its End-of-Life? The Azure Functions team communicates retirements well in advance through multiple channels: Azure Portal notifications Emails to subscription owners Warnings in client tools and Azure Portal UI when an app is running on a version that is either retired, or about to be retired in the next 6 months Official Azure Functions Supported Languages document here To help you track these changes, we recommend reviewing the language version support timelines in the Azure Functions Supported Languages document. However, identifying all affected apps across multiple subscriptions can be challenging. To simplify this process, I've built some Azure CLI scripts below that can help you list all impacted Function Apps in your environment. Linux* Function Apps with their language stack versions: az functionapp list --query "[?siteConfig.linuxFxVersion!=null && siteConfig.linuxFxVersion!=''].{Name:name, ResourceGroup:resourceGroup, OS:'Linux', LinuxFxVersion:siteConfig.linuxFxVersion}" --output table *Running on Elastic Premium and App Service Plans Linux* Function Apps on a specific language stack version: Ex: Node.js 18 az functionapp list --query "[?siteConfig.linuxFxVersion=='Node|18'].{Name:name, ResourceGroup:resourceGroup, OS: 'Linux', LinuxFxVersion:siteConfig.linuxFxVersion}" --output table *Running on Elastic Premium and App Service Plans Windows Function Apps only: az functionapp list --query "[?!contains(kind, 'linux')].{Name:name, ResourceGroup:resourceGroup, OS:'Windows'}" --output table Windows Function Apps with their language stack versions: az functionapp list --query "[?!contains(kind, 'linux')].{name: name, resourceGroup: resourceGroup}" -o json | ConvertFrom-Json | ForEach-Object { $appSettings = az functionapp config appsettings list -n $_.name -g $_.resourceGroup --query "[?name=='FUNCTIONS_WORKER_RUNTIME' || name=='WEBSITE_NODE_DEFAULT_VERSION']" -o json | ConvertFrom-Json $siteConfig = az functionapp config show -n $_.name -g $_.resourceGroup --query "{powerShellVersion: powerShellVersion, netFrameworkVersion: netFrameworkVersion, javaVersion: javaVersion}" -o json | ConvertFrom-Json $runtime = ($appSettings | Where-Object { $_.name -eq 'FUNCTIONS_WORKER_RUNTIME' }).value $version = switch($runtime) { 'node' { ($appSettings | Where-Object { $_.name -eq 'WEBSITE_NODE_DEFAULT_VERSION' }).value } 'powershell' { $siteConfig.powerShellVersion } 'dotnet' { $siteConfig.netFrameworkVersion } 'java' { $siteConfig.javaVersion } default { 'Unknown' } } [PSCustomObject]@{ Name = $_.name ResourceGroup = $_.resourceGroup OS = 'Windows' Runtime = $runtime Version = $version } } | Format-Table -AutoSize Windows Function Apps running on Node.js runtime: az functionapp list --query "[?!contains(kind, 'linux')].{name: name, resourceGroup: resourceGroup}" -o json | ConvertFrom-Json | ForEach-Object { $appSettings = az functionapp config appsettings list -n $_.name -g $_.resourceGroup --query "[?name=='FUNCTIONS_WORKER_RUNTIME' || name=='WEBSITE_NODE_DEFAULT_VERSION']" -o json | ConvertFrom-Json $runtime = ($appSettings | Where-Object { $_.name -eq 'FUNCTIONS_WORKER_RUNTIME' }).value if ($runtime -eq 'node') { $version = ($appSettings | Where-Object { $_.name -eq 'WEBSITE_NODE_DEFAULT_VERSION' }).value [PSCustomObject]@{ Name = $_.name ResourceGroup = $_.resourceGroup OS = 'Windows' Runtime = $runtime Version = $version } } } | Format-Table -AutoSize Windows Function Apps running on a specific language version: Ex: Node.js 18 az functionapp list --query "[?!contains(kind, 'linux')].{name: name, resourceGroup: resourceGroup}" -o json | ConvertFrom-Json | ForEach-Object { $appSettings = az functionapp config appsettings list -n $_.name -g $_.resourceGroup --query "[?name=='FUNCTIONS_WORKER_RUNTIME' || name=='WEBSITE_NODE_DEFAULT_VERSION']" -o json | ConvertFrom-Json $runtime = ($appSettings | Where-Object { $_.name -eq 'FUNCTIONS_WORKER_RUNTIME' }).value $nodeVersion = ($appSettings | Where-Object { $_.name -eq 'WEBSITE_NODE_DEFAULT_VERSION' }).value if ($runtime -eq 'node' -and $nodeVersion -eq '~18') { [PSCustomObject]@{ Name = $_.name ResourceGroup = $_.resourceGroup OS = 'Windows' Runtime = $runtime Version = $nodeVersion } } } | Format-Table -AutoSize All windows Apps running on unsupported language runtimes: (as of March 2025) az functionapp list --query "[?!contains(kind, 'linux')].{name: name, resourceGroup: resourceGroup}" -o json | ConvertFrom-Json | ForEach-Object { $appSettings = az functionapp config appsettings list -n $_.name -g $_.resourceGroup --query "[?name=='FUNCTIONS_WORKER_RUNTIME' || name=='WEBSITE_NODE_DEFAULT_VERSION']" -o json | ConvertFrom-Json $siteConfig = az functionapp config show -n $_.name -g $_.resourceGroup --query "{powerShellVersion: powerShellVersion, netFrameworkVersion: netFrameworkVersion}" -o json | ConvertFrom-Json $runtime = ($appSettings | Where-Object { $_.name -eq 'FUNCTIONS_WORKER_RUNTIME' }).value $version = switch($runtime) { 'node' { $nodeVer = ($appSettings | Where-Object { $_.name -eq 'WEBSITE_NODE_DEFAULT_VERSION' }).value if ([string]::IsNullOrEmpty($nodeVer)) { 'Unknown' } else { $nodeVer } } 'powershell' { $siteConfig.powerShellVersion } 'dotnet' { $siteConfig.netFrameworkVersion } default { 'Unknown' } } # Check if runtime version is unsupported $isUnsupported = switch($runtime) { 'node' { $ver = $version -replace '~','' [double]$ver -le 16 } 'powershell' { $ver = $version -replace '~','' [double]$ver -le 7.2 } 'dotnet' { $ver = $siteConfig.netFrameworkVersion $ver -notlike 'v7*' -and $ver -notlike 'v8*' } default { $false } } if ($isUnsupported) { [PSCustomObject]@{ Name = $_.name ResourceGroup = $_.resourceGroup OS = 'Windows' Runtime = $runtime Version = $version } } } | Format-Table -AutoSize Take Action Now By using these scripts, you can proactively identify and update Function Apps before they reach end-of-support status. Stay ahead of runtime retirements and ensure the reliability of your Function Apps. For step-by-step instructions to upgrade your Function Apps, check out the Azure Functions Language version upgrade guide. For more details on Azure Functions' language support lifecycle, visit the official documentation. Have any questions? Let us know in the comments below!
