Agentic Integration with SAP, ServiceNow, and Salesforce
Copilot/Copilot Studio Integration with SAP (No Code) By integrating SAP Cloud Identity Services with Microsoft Entra ID, organizations can establish secure, federated identity management across platforms. This configuration enables Microsoft Copilot and Teams to seamlessly connect with SAP’s Joule digital assistant, supporting natural language interactions and automating business processes efficiently. Key Resources as given in SAP docs (Image courtesy SAP): Configuring SAP Cloud Identity Services and Microsoft Entra ID for Joule Enable Microsoft Copilot and Teams to Pass Requests to Joule Copilot Studio Integration with ServiceNow and Salesforce (No Code) Integration with ServiceNow and Salesforce, has two main approaches: Copilot Agents using Copilot Studio: Custom agents can be built in Copilot Studio to interact directly with Salesforce CRM data or ServiceNow knowledge bases and helpdesk tickets. This enables organizations to automate sales and support processes using conversational AI. Create a custom sales agent using your Salesforce CRM data (YouTube) ServiceNow Connect Knowledge Base + Helpdesk Tickets (YouTube) 3rd Party Agents using Copilot for Service Agent: Microsoft Copilot can be embedded within Salesforce and ServiceNow interfaces, providing users with contextual assistance and workflow automation directly inside these platforms. Set up the embedded experience in Salesforce Set up the embedded experience in ServiceNow MCP or Agent-to-Agent (A2A) Interoperability (Pro Code) - (Image courtesy SAP) If you choose a pro-code approach, you can either implement the Model Context Protocol (MCP) in a client/server setup for SAP, ServiceNow, and Salesforce, or leverage existing agents for these third-party services using Agent-to-Agent (A2A) integration. Depending on your requirements, you may use either method individually or combine them. The recently released Azure Agent Framework offers practical examples for both MCP and A2A implementations. Below is the detailed SAP reference architecture, illustrating how A2A solutions can be layered on top of SAP systems to enable modular, scalable automation and data exchange. Agent2Agent Interoperability | SAP Architecture Center Logic Apps as Integration Actions Logic Apps is the key component of Azure Integration platform. Just like so many other connectors it has connectors for all this three platforms (SAP, ServiceNow, Salesforce). Logic Apps can be invoked from custom Agent (built in action in Foundry) or Copilot Agent. Same can be said for Power Platform/Automate as well. Conclusion This article provides a comprehensive overview of how Microsoft Copilot, Copilot Studio, Foundry by A2A/MCP, and Azure Logic Apps can be combined to deliver robust, agentic integrations with SAP, ServiceNow, and Salesforce. The narrative highlights the importance of secure identity federation, modular agent orchestration, and low-code/pro-code automation in building next-generation enterprise solutions.Selecting the Right Agentic Solution on Azure - Part 1
Recently, we have seen a surge in requests from customers and Microsoft partners seeking guidance on building and deploying agentic solutions at various scales. With the rise of Generative AI, replacing traditional APIs with agents has become increasingly popular. There are several approaches to building, deploying, running, and orchestrating agents on Azure. In this discussion, I will focus exclusively on Azure-specific tools, services, and methodologies, setting aside Copilot and Copilot Studio for now. This article describes the options available as of today. 1. Azure OpenAI Assistants API: This feature within Azure OpenAI Service enables developers to create conversational agents (“assistants”) based on OpenAI models (such as GPT-3.5 and GPT-4). It supports capabilities like memory, tool/function calls, and retrieval (e.g., document search). However, Microsoft has already deprecated version 1 of the Azure OpenAI Assistants API, and version 2 remains in preview. Microsoft strongly recommends migrating all existing Assistants API-based agents to the Agent Service. Additionally, OpenAI is retiring the Assistants API and advises developers to use the modern “Response” API instead (see migration detail). Given these developments, it is not advisable to use the Assistants API for building agents. Instead, you should use the Azure AI Agent Service, which is part of Azure AI Foundry. 2. Workflows with AI agents and models in Azure Logic Apps (Preview) – As the name suggests, this feature is currently in public preview and is only available with Logic Apps Standard, not with the consumption plan. You can enhance your workflow by integrating agentic capabilities. For example, in a visa processing workflow, decisions can be made based on priority, application type, nationality, and background checks using a knowledge base. The workflow can then route cases to the appropriate queue and prepare messages accordingly. Workflows can be implemented either as chat assistant or APIs. If your project is workflow-dependent and you are ready to implement agents in a declarative way, this is a great option. However, there are currently limited choices for models and regional availability. For CI/CD, there is an Azure Logic Apps Standard template available for VS Code you can use. 3. Azure AI Agent Service – Part of Azure AI Foundry, the Azure AI Agent Service allows you to provision agents declaratively from the UI. You can consume various OpenAI models (with support for non-OpenAI models coming soon) and leverage important tools or knowledge bases such as files, Azure AI Search, SharePoint, and Fabric. You can connect agents together and create hierarchical agent dependencies. SDKs are available for building agents within agent services using Python, C#, or Java. Microsoft manages the infrastructure to host and run these agents in isolated containers. The service offers role-based access control, MS Entra ID integration, and options to bring your own storage for agent states and Azure Key Vault keys. You can also incorporate different actions including invoking a Logic App instance from your agent. There is also option to trigger an agent using Logic Apps (preview). Microsoft recommends using Agent Service/Azure Foundry as the destination for agents, as further enhancements and investments are focused here. 4. Agent Orchestrators – There are several excellent orchestrators available, such as LlamaIndex, LangGraph, LangChain, and two from Microsoft—Semantic Kernel and AutoGen. These options are ideal if you need full control over agent creation, hosting, and orchestration. They are developer-only solutions and do not offer a UI (barring AutoGen Studio having some UI assistance). You can create complex, multi-layered agent connections. You can then host and run these agents in you choice of Azure services like AKS or Apps Service. Additionally, you have the option to create agents using Agent Service and then orchestrate them with one of these orchestrators. Choosing the Right Solution The choice of agentic solution depends on several factors, including whether you prefer code or no-code approaches, control over the hosting platform, customer needs, scalability, maintenance, orchestration complexity, security, and cost. Customer Need: If agents need to be part of a workflow, use AI Agents in Logic Apps; otherwise, consider other options. No-Code: For workflow-based agents, Logic Apps is suitable; for other scenarios, Azure AI Agent Service is recommended. Hosting and Maintenance: If Logic Apps is not an option and you prefer not to maintain your own environment, use Azure AI Agent Service. Otherwise, consider custom agent orchestrators like Semantic Kernel or AutoGen to build the agent and services like AKS or Apps Service to host those. Orchestration Complexity: For simple hierarchical agent connections, Azure AI Agent Service is good choice. For complex orchestration, use an agent orchestrator. Versioning - If you are concerned about versioning to ensure solid CI/CD regime then you may have to chose Agent Orchestrators. Agent Service still miss this feature clarity. We have some work-around but it is not robust implementation. Hopefully we will catch up soon with a better versioning solution. Summary: When selecting the right agentic solution on Azure, consider the latest recommendations and platform developments. For most scenarios, Microsoft advises using the Azure AI Agent Service within Azure Foundry, as it is the focus of ongoing enhancements and support. For workflow-driven projects, Azure Logic Apps with agentic capabilities may be suitable, while advanced users can leverage orchestrators for custom agent architectures. In Selecting the Right Agentic Solution on Azure – Part 2 (Security) blog we will examine the security aspects of each option, one by one.How Great Engineers Make Architectural Decisions — ADRs, Trade-offs, and an ATAM-Lite Checklist
Why Decision-Making Matters Without a shared framework, context fades and teams' re-debate old choices. ADRs solve that by recording the why behind design decisions — what problem we solved, what options we considered, and what trade-offs we accepted. A good ADR: Lives next to the code in your repo. Explains reasoning in plain language. Survives personnel changes and version history. Think of it as your team’s engineering memory. The Five Pillars of Trade-offs At Microsoft, we frame every major design discussion using the Azure Well-Architected pillars: Reliability – Will the system recover gracefully from failures? Performance Efficiency – Can it meet latency and throughput targets? Cost Optimization – Are we using resources efficiently? Security – Are we minimizing blast radius and exposure? Operational Excellence – Can we deploy, monitor, and fix quickly? No decision optimizes all five. Great engineers make conscious trade-offs — and document them. A Practical Decision Flow Step What to Do Output 1. Frame It Clarify the problem, constraints, and quality goals (SLOs, cost caps). Problem statement 2. List Options Identify 2-4 realistic approaches. Options list 3. Score Trade-offs Use a Decision Matrix to rate options (1–5) against pillars. Table of scores 4. ATAM-Lite Review List scenarios, identify sensitivity points (small changes with big impact) and risks. Risk notes 5. Record It as an ADR Capture everything in one markdown doc beside the code. ADR file Example: Adding a Read-Through Cache Decision: Add a Redis cache in front of Cosmos DB to reduce read latency. Context: Average P95 latency from DB is 80 ms; target is < 15 ms. Options: A) Query DB directly B) Add read-through cache using Redis Trade-offs Performance: + Massive improvement in read speed. Cost: + Fewer RU/s on Cosmos DB. Reliability: − Risk of stale data if cache invalidation fails. Operational: + Added complexity for monitoring and TTLs. Templates You Can Re-use ADR Template # ADR-001: Add Read-through Cache in Front of Cosmos DB Status: Accepted Date: 2025-10-21 Context: High read latency; P95 = 80ms, target <15ms Options: A) Direct DB reads B) Redis cache for hot keys ✅ Decision: Adopt Redis cache for performance and cost optimization. Consequences: - Improved read latency and reduced RU/s cost - Risk of data staleness during cache invalidation - Added operational complexity Links: PR#3421, Design Doc #204, Azure Monitor dashboard Decision Matrix Example Pillar Weight Option A Option B Notes Reliability 5 3 4 Redis clustering handles failover Performance 4 2 5 In-memory reads Cost 3 4 5 Reduced RU/s Security 4 4 4 Same auth posture Operational Excellence 3 4 3 More moving parts Weighted total = Σ(weight × score) → best overall score wins. Team Guidelines Create a /docs/adr folder in each repo. One ADR per significant change; supersede old ones instead of editing history. Link ADRs in design reviews and PRs. Revisit when constraints change (incidents, new SLOs, cost shifts). Publish insights as follow-up blogs to grow shared knowledge. Why It Works This practice connects the theory of trade-offs with Microsoft’s engineering culture of reliability and transparency. It improves onboarding, enables faster design reviews, and builds a traceable record of engineering evolution. Join the Conversation Have you tried ADRs or other decision frameworks in your projects? Share your experience in the comments or link to your own public templates — let’s make architectural reasoning part of our shared language.
Accelerating Enterprise AI Adoption with Azure AI Landing Zone
Introduction As organizations across industries race to integrate Artificial Intelligence (AI) into their business processes and realize tangible value, one question consistently arises — where should we begin? Customers often wonder: What should the first steps in AI adoption look like? Should we build a unified, enterprise-grade platform for all AI initiatives? Who should guide us through this journey — Microsoft, our partners, or both? This blog aims to demystify these questions by providing a foundational understanding of the Azure AI Landing Zone (AI ALZ) — a unified, scalable, and secure framework for enterprise AI adoption. It explains how AI ALZ builds on two key architectural foundations — the Cloud Adoption Framework (CAF) and the Well-Architected Framework (WAF) — and outlines an approach to setting up an AI Landing Zone in your Azure environment. Foundational Frameworks Behind the AI Landing Zone 1.1 Cloud Adoption Framework (CAF) The Azure Cloud Adoption Framework is Microsoft’s proven methodology for guiding customers through their cloud transformation journey. It encompasses the complete lifecycle of cloud enablement across stages such as Strategy, Plan, Ready, Adopt, Govern, Secure, and Manage. The Landing Zone concept sits within the Ready stage — providing a secure, scalable, and compliant foundation for workload deployment. CAF also defines multiple adoption scenarios, one of which focuses specifically on AI adoption, ensuring that AI workloads align with enterprise cloud governance and best practices. 1.2 Well-Architected Framework (WAF) The Azure Well-Architected Framework complements CAF by providing detailed design guidance across five key pillars: Reliability Security Cost Optimization Operational Excellence Performance Efficiency AI Landing Zones integrate these design principles to ensure that AI workloads are not only functional but also resilient, cost-effective, and secure at enterprise scale. Understanding Azure Landing Zones To understand an AI Landing Zone, it’s important to first understand Azure Landing Zones in general. An Azure Landing Zone acts as a blueprint or foundation for deploying workloads in a cloud environment — much like a strong foundation is essential for constructing a building or bridge. Each workload type (SAP, Oracle, CRM, AI, etc.) may require a different foundation, but all share the same goal: to provide a consistent, secure, and repeatable environment built on best practices. Azure Landing Zones provide: A governed, scalable foundation aligned with enterprise standards Repeatable, automated deployment patterns using Infrastructure as Code (IaC) Integrated security and management controls baked into the architecture To have more insightful understanding of Azure Landing zone architecture pls visit the official link here and refer diagram below: The Role of Azure AI Foundry in AI Landing Zones Azure AI Foundry is emerging as Microsoft’s unified environment for enterprise AI development and deployment. It acts as a one-stop platform for building, deploying, and managing AI solutions at scale. Key components include: Foundry Model Catalog: A collection of foundation and fine-tuned models Agent Service: Enables model selection, tool and knowledge integration, and control over data and security Search and Machine Learning Services: Integrated capabilities for knowledge retrieval and ML lifecycle management Content Safety and Observability: Ensures responsible AI use and operational visibility Compute Options: Customers can choose from various Azure compute services based on control and scalability needs: Azure Kubernetes Service (AKS) — full control App Service and Azure Container Apps — simplified management Azure Functions — fully serverless option What Is Azure AI Landing Zone (AI ALZ)? The Azure AI Landing Zone is a workload-specific landing zone designed to help enterprises deploy AI workloads securely and efficiently in production environments. Key Objectives of AI ALZ Accelerate deployment of production-grade AI solutions Embed security, compliance, and resilience from the start Enable cost and operational optimization through standardized architecture Support repeatable patterns for multiple AI use cases using Azure AI Foundry Empower customer-centric enablement with extensibility and modularity By adopting the AI ALZ, organizations can move faster from proof-of-concept (POC) to production, addressing common challenges such as inconsistent architectures, lack of governance, and operational inefficiencies. Core Components of AI Landing Zone The AI ALZ is structured around three major components: Design Framework – Based on the Cloud Adoption Framework (CAF) and Well-Architected Framework (WAF). Reference Architectures – Blueprint architectures for common AI workloads. Extensible Implementations – Deployable through Terraform, Bicep, or (soon) Azure Portal templates using Azure Verified Modules (AVM). Together, these elements allow customers to quickly deploy a secure, standardized, and production-ready AI environment. Customer Readiness and Discovery A common question during early customer engagements is: “Can our existing enterprise-scale landing zone support AI workloads, or do we need a new setup?” To answer this, organizations should start with a discovery and readiness assessment, reviewing their existing enterprise-scale landing zone across key areas such as: Identity and Access Management Networking and Connectivity Data Security and Compliance Governance and Policy Controls Compute and Deployment Readiness Based on this assessment, customers can either: Extend their existing enterprise-scale foundation, or Deploy a dedicated AI workload spoke designed specifically for Azure AI Foundry and enterprise-wide AI enablement. Attached excel contains the discovery question to enquire about customer current setup and propose a adoption plan to reflect architecture changes if any. The Journey Toward AI Adoption The AI Landing Zone represents the first critical step in an organization’s AI adoption journey. It establishes the foundation for: Consistent governance and policy enforcement Security and networking standardization Rapid experimentation and deployment of AI workloads Scalable, production-grade AI environments By aligning with CAF and WAF, customers can be confident that their AI adoption strategy is architecturally sound, secure, and sustainable. Conclusion The Azure AI Landing Zone provides enterprises with a structured, secure, and scalable foundation for AI adoption at scale. It bridges the gap between innovation and governance, enabling organizations to deploy AI workloads faster while maintaining compliance, performance, and operational excellence. By leveraging Microsoft’s proven frameworks — CAF and WAF — and adopting Azure AI Foundry as the unified development platform, enterprises can confidently build the next generation of responsible, production-grade AI solutions on Azure. Get Started Ready to start your AI Landing Zone journey? Microsoft can help assess your readiness and accelerate deployment through validated reference implementations and expert-led guidance. To help organizations accelerate deployment, Microsoft has published open-source Azure AI Landing Zone templates and automation scripts in Terraform and Bicep that can be directly used to implement the architecture described in this blog. 👉 Explore and deploy the Azure AI Landing Zone(Preview) on GitHub: https://github.com/Azure/AI-Landing-ZonesHow Azure NetApp Files Object REST API powers Azure and ISV Data and AI services – on YOUR data
This article introduces the Azure NetApp Files Object REST API, a transformative solution for enterprises seeking seamless, real-time integration between their data and Azure's advanced analytics and AI services. By enabling direct, secure access to enterprise data—without costly transfers or duplication—the Object REST API accelerates innovation, streamlines workflows, and enhances operational efficiency. With S3-compatible object storage support, it empowers organizations to make faster, data-driven decisions while maintaining compliance and data security. Discover how this new capability unlocks business potential and drives a new era of productivity in the cloud.Validating Scalable EDA Storage Performance: Azure NetApp Files and SPECstorage Solution 2020
Electronic Design Automation (EDA) workloads drive innovation across the semiconductor industry, demanding robust, scalable, and high-performance cloud solutions to accelerate time-to-market and maximize business outcomes. Azure NetApp Files empowers engineering teams to run complex simulations, manage vast datasets, and optimize workflows by delivering industry-leading performance, flexibility, and simplified deployment—eliminating the need for costly infrastructure overprovisioning or disruptive workflow changes. This leads to faster product development cycles, reduced risk of project delays, and the ability to capitalize on new opportunities in a highly competitive market. In a historic milestone, Microsoft has been independently validated Azure NetApp Files for EDA workloads through the publication of the SPECstorage® Solution 2020 EDA_BLENDED benchmark, providing objective proof of its readiness to meet the most demanding enterprise requirements, now and in the future.
AI Azure Landing Zone: Shared Capabilities and Models to Enable AI as a Platform
This architecture diagram illustrates a Microsoft Azure AI Landing Zone Pattern — a scalable, secure, and well-governed framework for deploying AI workloads across multiple subscriptions in an enterprise environment. Let's walk through it end-to-end, breaking down each section, the flow, and key Azure services involved. 🧭 Overview: The architecture is split into 4 major landing zones: Connectivity Subscription AI Apps Landing Zone Subscription AI Hub Landing Zone Subscription AI Services Landing Zone Subscription 🔁 Step-by-Step Breakdown 🔹 1. Users → Application Gateway (WAF) Users (e.g., enterprise employees or external users) access the system via the Application Gateway with Web Application Firewall (WAF). This is part of the Connectivity Subscription and provides: Centralized ingress control Zone redundancy Protection against common exploits 🔹 2. Route to AI Apps Landing Zone Subscription Traffic is routed to the AI Apps Landing Zone Subscription via the Application Gateway. This subscription hosts applications that use AI services, typically in a containerized or App Service-based architecture. 🔹 3. AI Apps Workload Components This section includes: App Hosting: Azure App Services Container Apps (with Container Registry) Networking: Private Endpoints Subnets Network Security Groups Monitoring: Log Analytics Workspaces Diagnostic Settings App Agents: Represent container/app service instances (Agent 1, 2, 3) 🔹 4. Integration with AI Services & Secrets Management These apps securely connect to: Azure Key Vault (secrets, credentials) Azure AI Search Azure Cosmos DB Azure Storage Azure OpenAI App Insights is used for application performance monitoring. Logic Apps & Functions handle: Knowledge Management Processing LLM Integration Workflows 🔹 5 & 6. Connectivity to Centralized Services Virtual Network Peering connects AI Apps Landing Zone with: Connectivity Subscription Hub Virtual Network in the Platform Landing Zone Subscription These provide access to shared infrastructure: Azure Firewall Azure Bastion VPN Gateway / ExpressRoute Azure DNS / Private Resolver Azure DDoS Protection 🔹 7. AI Hub Landing Zone Subscription This acts as a centralized workload processing zone with components like: Event Hubs Azure Key Vault App Insights Power BI Cosmos DB API Management (OpenAI Endpoints) Used for: Observability Usage processing API integration 🔹 8 & 9. FTU Usage Processing & Reporting Function Apps & Logic Apps: Process usage data (e.g., for chargebacks, monitoring) FTU = "Fair Tenant Usage" Reporting is done using Power BI and stored in Cosmos DB 🔹 10 & 11. Network Peering to Platform Zone AI Hub connects back to Platform Landing Zone via Virtual Network Peering Provides access to shared DNS zones and network services 🔹 12. AI Services Landing Zone Subscription This is where core AI capabilities live, such as: Azure OpenAI Azure AI Services: Speech Vision Language Machine Learning Foundry Project: OpenAI Agents Agent Service Dependencies Models hosted in Azure (e.g., GPT) This zone is accessed securely via: Private Endpoints Azure Key Vault Network rules 📦 Subscription Vending (All Zones) Each subscription includes a Subscription Vending Framework for: Spoke VNet placement Route configurations Policy/role assignments Defender for Cloud & cost management This ensures a consistent and compliant environment across the enterprise. 📌 Key Architectural Benefits Feature Purpose 🔐 Zero Trust Network Controlled access via WAF, private endpoints 📡 Scalable AI Apps Container Apps & App Services 🧠 Central AI Services Managed in isolated subscriptions 🔍 Monitoring Deep insights via App Insights, Log Analytics 🧾 Governance Role-based access, policy enforcement 🔌 Secure Integration VNet Peering, Azure Key Vault, API Management 🔚 End-to-End Data Flow Summary Users access app through Application Gateway (WAF) Apps in AI Apps Landing Zone process input Apps call AI services (OpenAI, Cognitive) via private endpoints Data usage and insights flow to AI Hub for logging and analysis FTU and usage metrics processed and stored Platform services support routing, DNS, security 🎯 Goal of the User Journey The user interacts with an AI-powered application (e.g., chatbot, document summarizer, recommendation engine) deployed on Azure. The app is secure, scalable, and integrated with advanced Azure AI services (like OpenAI). 👣 User Journey: Step-by-Step Breakdown ✅ 1. User Access (Public Entry Point) The user (browser or mobile app) sends a request (e.g., opens an AI web app or sends a prompt to a chatbot). The request hits the Azure Application Gateway with Web Application Firewall (WAF). ✅ Filters and protects against malicious traffic. ✅ Ensures high availability with zone redundancy. 🧠 Think of it as the front door to the AI platform. ✅ 2. Routing to AI Application The Application Gateway securely routes the request to the AI Apps Landing Zone Subscription. The user request reaches the App Service or Container App hosting the AI-based application logic. Example: A user submits a product question via a chatbot UI hosted here. ✅ 3. Processing the Request (App Logic) The app receives the input and begins processing: App uses App Insights for performance telemetry. Secrets or config (API keys, connection strings) are securely pulled from Azure Key Vault. Based on the business logic, the app needs to call an AI model (e.g., OpenAI). ✅ 4. Calling AI Services (via Private Endpoints) The app securely connects (using private endpoints) to the AI Services Landing Zone to: 🔹 Call Azure OpenAI (e.g., ChatGPT, DALL·E, embeddings) 🔹 Use Azure Cognitive Services (e.g., speech, vision, search) These services are isolated in their own subscription for security, scalability, and cost governance. 🧠 Here’s where the “AI magic” happens. ✅ 5. Retrieval-Augmented Generation (Optional) If the AI needs additional knowledge (RAG pattern), the app can: Query Azure AI Search for documents. Pull knowledge from Azure Cosmos DB or Azure Storage. AI results are processed via Logic Apps / Functions (e.g., post-processing, formatting). ✅ 6. Return the Response to the User The application receives the AI-generated output. It formats the result (e.g., chatbot message, visual, PDF, etc.) and returns it to the user via the original secure path. ✅ 7. Observability & Usage Logging App, AI service usage, and telemetry are logged in: Log Analytics / App Insights Event Hub → Streamed to AI Hub Landing Zone This enables centralized monitoring and analytics (Power BI dashboards, anomaly detection, etc.) ✅ 8. Usage Reporting & Governance Function App & Logic App in the AI Hub Landing Zone process usage logs. Usage is stored in Azure Cosmos DB. FTU (Fair Tenant Usage) policies are enforced and reported via Power BI dashboards. ✅ 9. Admin/Platform Layer All resources and subscriptions are governed via the Platform Landing Zone: Shared services like DNS, security policies, firewalls Cost controls, Defender for Cloud, DDoS protection Subscription vending and network segmentation 🗺️ Visual Recap: User Journey Flow User → App Gateway (WAF) → App in AI Apps Landing Zone → Call to Azure OpenAI / AI Services → (Optional: Knowledge retrieval) → AI Response →Returned to User → Usage logged & monitored → Usage reporting in AI Hub User Workflow 🔐 Security Throughout the Journey Step Security Feature App Gateway Web Application Firewall App Hosting Private Endpoints, Managed Identity Secrets Azure Key Vault Network Virtual Network Peering, NSGs Governance Role-based access, Policy Assignments 🧠 Example: Real-World Use Case Scenario: A doctor uses a medical AI assistant to analyze patient notes. Logs in via secure portal (WAF gateway) Submits patient notes (App Service) App calls OpenAI with prompt: "Summarize this diagnosis." App also queries internal document store (RAG) OpenAI returns result → displayed in UI Usage tracked for audit and reporting 🧭 User Journey Flow Users End users initiate a request (e.g., accessing an AI-powered app). Application Gateway + WAF (Connectivity Subscription) Request is routed through the Application Gateway with Web Application Firewall for security and traffic filtering. AI Apps Landing Zone Subscription Request enters the AI Apps subscription. Workloads run on App Services or Container Apps (Agents 1, 2, 3). Secure Access Application services authenticate and securely retrieve data from Azure Key Vault, Cosmos DB, Azure Storage, and Azure AI Search. Knowledge Management Processing Logic Apps / Function Apps process the request, enabling workflows, integrations, and knowledge enrichment. AI Hub Gateway Application Requests requiring AI services are routed to the AI Hub for centralized management. API Management (OpenAI Endpoints) APIs handle communication with downstream AI services. Event Hub + App Insights Telemetry and logs are captured for monitoring and troubleshooting. Power BI + Cosmos DB Usage data is aggregated and analyzed for reporting (FTU usage tracking). AI Services Subscription API calls are directed to the AI Services subscription. Azure AI Models Execution Requests hit Azure OpenAI, Azure AI Foundry, Cognitive Services (Speech, Vision, Search, etc.). Foundry/Agent services provide additional AI processing. Response back to User Processed AI output is routed back through the pipeline → API → Hub → Apps → Application Gateway → returned to the user. High Level Architecture Diagram Security & Governance Overview AI Landing Zone Lifecycle Workflow URL Reference Architectures: Baseline Azure AI Foundry Chat Reference Architecture in an Azure Landing Zone - Azure Architecture Center | Microsoft Learn Repo Link for AI Landing Zone: https://github.com/Azure/AI-Landing-Zones
Building a Secure and Compliant Azure AI Landing Zone: Policy Framework & Best Practices
As organizations accelerate their AI adoption on Microsoft Azure, governance, compliance, and security become critical pillars for success. Deploying AI workloads without a structured compliance framework can expose enterprises to data privacy issues, misconfigurations, and regulatory risks. To address this challenge, the Azure AI Landing Zone provides a scalable and secure foundation — bringing together Azure Policy, Blueprints, and Infrastructure-as-Code (IaC) to ensure every resource aligns with organizational and regulatory standards. The Azure Policy & Compliance Framework acts as the governance backbone of this landing zone. It enforces consistency across environments by applying policy definitions, initiatives, and assignments that monitor and remediate non-compliant resources automatically. This blog will guide you through: 🧭 The architecture and layers of an AI Landing Zone 🧩 How Azure Policy as Code enables automated governance ⚙️ Steps to implement and deploy policies using IaC pipelines 📈 Visualizing compliance flows for AI-specific resources What is Azure AI Landing Zone (AI ALZ)? AI ALZ is a foundational architecture that integrates core Azure services (ML, OpenAI, Cognitive Services) with best practices in identity, networking, governance, and operations. To ensure consistency, security, and responsibility, a robust policy framework is essential. Policy & Compliance in AI ALZ Azure Policy helps enforce standards across subscriptions and resource groups. You define policies (single rules), group them into initiatives (policy sets), and assign them with certain scopes & exemptions. Compliance reporting helps surface noncompliant resources for mitigation. In AI workloads, some unique considerations: Sensitive data (PII, models) Model accountability, logging, audit trails Cost & performance from heavy compute usage Preview features and frequent updates Scope This framework covers: Azure Machine Learning (AML) Azure API Management Azure AI Foundry Azure App Service Azure Cognitive Services Azure OpenAI Azure Storage Accounts Azure Databases (SQL, Cosmos DB, MySQL, PostgreSQL) Azure Key Vault Azure Kubernetes Service Core Policy Categories 1. Networking & Access Control Restrict resource deployment to approved regions (e.g., Europe only). Enforce private link and private endpoint usage for all critical resources. Disable public network access for workspaces, storage, search, and key vaults. 2. Identity & Authentication Require user-assigned managed identities for resource access. Disable local authentication; enforce Microsoft Entra ID (Azure AD) authentication. 3. Data Protection Enforce encryption at rest with customer-managed keys (CMK). Restrict public access to storage accounts and databases. 4. Monitoring & Logging Deploy diagnostic settings to Log Analytics for all key resources. Ensure activity/resource logs are enabled and retained for at least one year. 5. Resource-Specific Guardrails Apply built-in and custom policy initiatives for OpenAI, Kubernetes, App Services, Databases, etc. A detailed list of all policies is bundled and attached at the end of this blog. Be sure to check it out for a ready-to-use Excel file—perfect for customer workshops—which includes policy type (Standalone/Initiative), origin (Built-in/Custom), and more. Implementation: Policy-as-Code using EPAC To turn policies from Excel/JSON into operational governance, Enterprise Policy as Code (EPAC) is a powerful tool. EPAC transforms policy artifacts into a desired state repository and handles deployment, lifecycle, versioning, and CI/CD automation. What is EPAC & Why Use It? EPAC is a set of PowerShell scripts / modules to deploy policy definitions, initiatives, assignments, role assignments, exemptions. Enterprise Policy As Code (EPAC) It supports CI/CD integration (GitHub Actions, Azure DevOps) so policy changes can be treated like code. It handles ordering, dependency resolution, and enforcement of a “desired state” — any policy resources not in your repo may be pruned (depending on configuration). It integrates with Azure Landing Zones (including governance baseline) out of the box. References & Further Reading EPAC GitHub Repository Advanced Azure Policy management - Microsoft Learn [Advanced A...Framework] How to deploy Azure policies the DevOps way [How to dep...- Rabobank]Azure OpenAI Landing Zone reference architecture
In this article, delve into the synergy of Azure Landing Zones and Azure OpenAI Service, building a secure and scalable AI environment. Unpack the Azure OpenAI Landing Zone architecture, which integrates numerous Azure services for optimal AI workloads. Explore robust security measures and the significance of monitoring for operational success. This journey of deploying Azure OpenAI evolves alongside Azure's continual innovation.
