Announcing General Availability for Azure Resource Graph (ARG) GET/LIST API
ARG GET/LIST API delivers 10X higher throttling quotas to callers compared to ARG query unlocking a more scalable, resilient way to perform resource lookups in Azure. ARG GET/LIST API is a new platform capability within Azure Resource Graph that provides a high-performance experience for both Point GET and collection GET requests. A key advantage of this capability is its ability to significantly reduce READ throttling for high volume calls efficiently. This is made possible through intelligent control plane routing based on a query parameter controlled by the caller. When a specific query parameter is included, requests are automatically directed to this optimized ARG GET/LIST backend. When the parameter is omitted, requests flow to the Resource provider —ensuring flexibility and backward compatibility. What Challenge Are We Addressing? Azure Read Throttling is a significant challenge for many customers. When services hit throttling limits, applications may experience performance degradation, elevated latency, or even failed requests—issues that can disrupt critical workloads and customer operations. The ARG GET/LIST API is designed to directly address this problem. By routing GET and LIST calls through Azure Resource Graph’s scalable indexing infrastructure and intelligent control-plane routing, it dramatically reduces the likelihood of read throttling. Best of all, it follows the ARM control plane GET APIs request response contract, allowing you to benefit from improved performance and reliability with minimal effort, appending the flag “useResourceGraph=true”. When to use Azure Resource Graph (ARG) GET/LIST API The ARG GET/LIST API is designed for scenarios where you need to retrieve a single resource by its ID or list resources of the same type within a defined scope—whether that's a subscription, resource group, or parent resource. You should consider using the ARG GET/LIST API if your service fits into one or more of the following categories: High Volume of GET Calls Within a Single Scope: Your service issues a large number of GET requests targeting resources within a single subscription or resource group, without the need for cross-subscription queries, complex filters, or joins. Risk of Throttling or Quota Competition: Your service produces a high volume of requests and may encounter issues such as:: Experience throttling during sudden traffic spikes. Quota competition, where other workloads in the same subscription consume shared quota limits, causing your service to be throttled. Bursty traffic patterns, where large volume of GET requests are issued within a short time window, increasing the chance of throttling. Need for High Availability and Faster Performance: Your service depends on consistent; low-latency GET operations for either single-resource lookups or listing resources within a specific scope Note: The ARG GET/LIST API is currently supported only for resources in the resources and computeresources tables. Using the ARG GET/LIST API To get started with the ARG GET/LIST API, begin by assessing whether your scenario aligns with the recommended calling patterns and throttling considerations described earlier. Once confirmed, simply append the parameter &useResourceGraph=true to your eligible GET/LIST API calls. This flag routes your request through the Azure Resource Graph GET/LIST API backend, allowing you to take advantage of its optimized performance and query efficiency. No calls will route to ARG GET/LIST backend automatically. The switch is entirely in the user’s control—the call will route to ARG GET/LIST API only when you explicitly include the useResourceGraph=true parameter in your request. Follow the ARG GET/LIST API contract here - Azure Resource Graph GET/LIST API Guidance - Azure Resource Graph | Microsoft Learn Let’s walk through a simple example of retrieving a Virtual Machine (VM) along with its InstanceView through ARG Query vs. ARM API vs. ARG GET/LIST API to show the difference in the calling experience. Using an ARG Query (via ARG Explorer) In ARG Explorer, you can use Kusto Query Language (KQL) to query resources. A sample query to retrieve a specific VM looks like this: Resources | where type =~ 'microsoft.compute/virtualmachines' | where id =~ '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachines/{vm}' This query filters the Resource Graph index to return the VM resource. Using the ARM (Compute RP) API The equivalent ARM API call to retrieve the VM with InstanceView is: GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachines/{vm}?api-version=2024-07-01&$expand=instanceView This hits the Compute Resource Provider, pulls the VM state, and expands the instanceView section. Using the ARG GET/LIST API ARG GET/LIST APIs that follow the same request structure as ARM—but with an additional flag that routes the call through ARG: GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/microsoft.compute/virtualmachines/{vm}?api-version=2024-07-01&$expand=instanceView&useResourceGraph=true The important distinction here is the useResourceGraph=true parameter, which routes the call through ARM to serve the response through ARG’s GET/LIST backend. Sample Response - You can find more examples in our documentation - Azure Resource Graph GET/LIST API Guidance - Azure Resource Graph | Microsoft Learn Video Walkthrough Increase Throttling Quota via Azure Resource Graph Learn More Azure Resource Graph GET/LIST API Overview Known Limitations Frequently Asked Questions Share Your Feedback For questions and feedback, you can reach us at Azure Resource Graph team Share Product feedback and ideas with us at Azure Governance · Community Happy Querying!Security Guidance Series: CAF 4.0 Building Proactive Cyber Resilience
It’s Time To Act Microsoft's Digital Defense Report 2025 clearly describes the cyber threat landscape that this guidance is situated in, one that has become more complex, more industrialized, and increasingly democratized. Each day, Microsoft processes more than 100 trillion security signals, giving unparalleled visibility into adversarial tradecraft. Identity remains the most heavily targeted attack vector, with 97% of identity-based attacks relying on password spray, while phishing and unpatched assets continue to provide easy routes for initial compromise. Financially motivated attacks, particularly ransomware and extortion, now make up over half of global incidents, and nation-state operators continue to target critical sectors, including IT, telecommunications, and Government networks. AI is accelerating both sides of the equation: enhancing attacker capability, lowering barriers to entry through open-source models, and simultaneously powering more automated, intelligence-driven defence. Alongside this, emerging risks such as quantum computing underline the urgency of preparing today for tomorrow’s threats. Cybersecurity has therefore become a strategic imperative shaping national resilience and demanding genuine cross-sector collaboration to mitigate systemic risk. It is within this environment that UK public sector organizations are rethinking their approach to cyber resilience. As an Account Executive Apprentice in the Local Public Services team here at Microsoft, I have seen how UK public sector organizations are rethinking their approach to cyber resilience, moving beyond checklists and compliance toward a culture of continuous improvement and intelligence-led defence. When we talk about the UK public sector in this series, we are referring specifically to central government departments, local government authorities, health and care organizations (including the NHS), education institutions, and public safety services such as police, fire, and ambulance. These organizations form a deeply interconnected ecosystem delivering essential services to millions of citizens every day, making cyber resilience not just a technical requirement but a foundation of public trust. Against this backdrop, the UK public sector is entering a new era of cyber resilience with the release of CAF 4.0, the latest evolution of the National Cyber Security Centre’s Cyber Assessment Framework. This guidance has been developed in consultation with national cyber security experts, including the UK’s National Cyber Security Centre (NCSC), and is an aggregation of knowledge and internationally recognized expertise. Building on the foundations of CAF 3.2, this update marks a decisive shift, like moving from a static map to a live radar. Instead of looking back at where threats once were, organizations can now better anticipate them and adjust their digital defences in real time. For the UK’s public sector, this transformation could not be timelier. The complexity of digital public services, combined with the growing threat of ransomware, insider threat, supply chain compromise, and threats from nation state actors, demands a faster, smarter, and more connected approach to resilience. Where CAF 3.2 focused on confirming the presence and effectiveness of security measures, CAF 4.0 places greater emphasis on developing organizational capability and improving resilience in a more dynamic threat environment. While the CAF remains an outcome-based framework, not a maturity model, it is structured around Objectives, Principles, and Contributing Outcomes, with each contributing outcome supported by Indicators of Good Practice. For simplicity, I refer to these contributing outcomes as “controls” throughout this blog and use that term to describe the practical expectations organizations are assessed against. CAF 4.0 challenges organizations not only to understand the threats they face but to anticipate, detect, and respond in a more informed and adaptive way. Two contributing outcomes exemplify this proactive mindset: A2.b Understanding Threat and C2 Threat Hunting. Together, they represent what it truly means to understand your adversaries and act before harm occurs. For the UK’s public sector, achieving these new objectives may seem daunting, but the path forward is clearer than ever. Many organizations are already beginning this journey, supported by technologies that help turn insight into action and coordination into resilience. At Microsoft, we’ve seen how tools like E3, E5, and Sentinel are already helping public sector teams to move from reactive to intelligence-driven security operations. Over the coming weeks, we’ll explore how these capabilities align to CAF 4.0’s core principles and share practical examples of how councils can strengthen their resilience journey through smarter visibility, automation, and collaboration. CAF 4.0 vs CAF 3.2 - What’s Changed and Why It Matters The move from CAF 3.2 to CAF 4.0 represents a fundamental shift in how the UK public sector builds cyber resilience. The focus is no longer on whether controls exist - it is on whether they work, adapt, and improve over time. CAF 4.0 puts maturity at the centre. It pushes organizations to evolve from compliance checklists to operational capability, adopting a threat-informed, intelligence-led, and proactive security posture, by design. CAF 4.0 raises the bar for cyber maturity across the public sector. It calls for departments and authorities to build on existing foundations and embrace live threat intelligence, behavioural analytics, and structured threat hunting to stay ahead of adversaries. By understanding how attackers might target essential services and adapting controls in real time, organizations can evolve from awareness to active defence. Today’s threat actors are agile, persistent, and increasingly well-resourced, which means reactive measures are no longer enough. CAF 4.0 positions resilience as a continuous process of learning, adapting, and improving, supported by data-driven insights and modern security operations. CAF 4.0 is reshaping how the UK’s public sector approaches security maturity. In the coming weeks, we’ll explore what this looks like in practice, starting with how to build a deeper understanding of threat (control A2.b) and elevate threat hunting (control C2) into an everyday capability, using the tools and insights that are available within existing Microsoft E3 and E5 licences to help support these objectives. Until then, how ready is your organization to turn insight into action?Security as the core primitive - Securing AI agents and apps
This week at Microsoft Ignite, we shared our vision for Microsoft security -- In the agentic era, security must be ambient and autonomous, like the AI it protects. It must be woven into and around everything we build—from silicon to OS, to agents, apps, data, platforms, and clouds—and throughout everything we do. In this blog, we are going to dive deeper into many of the new innovations we are introducing this week to secure AI agents and apps. As I spend time with our customers and partners, there are four consistent themes that have emerged as core security challenges to secure AI workloads. These are: preventing agent sprawl and access to resources, protecting against data oversharing and data leaks, defending against new AI threats and vulnerabilities, and adhering to evolving regulations. Addressing these challenges holistically requires a coordinated effort across IT, developers, and security leaders, not just within security teams and to enable this, we are introducing several new innovations: Microsoft Agent 365 for IT, Foundry Control Plane in Microsoft Foundry for developers, and the Security Dashboard for AI for security leaders. In addition, we are releasing several new purpose-built capabilities to protect and govern AI apps and agents across Microsoft Defender, Microsoft Entra, and Microsoft Purview. Observability at every layer of the stack To facilitate the organization-wide effort that it takes to secure and govern AI agents and apps – IT, developers, and security leaders need observability (security, management, and monitoring) at every level. IT teams need to enable the development and deployment of any agent in their environment. To ensure the responsible and secure deployment of agents into an organization, IT needs a unified agent registry, the ability to assign an identity to every agent, manage the agent’s access to data and resources, and manage the agent’s entire lifecycle. In addition, IT needs to be able to assign access to common productivity and collaboration tools, such as email and file storage, and be able to observe their entire agent estate for risks such as over-permissioned agents. Development teams need to build and test agents, apply security and compliance controls by default, and ensure AI models are evaluated for safety guardrails and security vulnerabilities. Post deployment, development teams must observe agents to ensure they are staying on task, accessing applications and data sources appropriately, and operating within their cost and performance expectations. Security & compliance teams must ensure overall security of their AI estate, including their AI infrastructure, platforms, data, apps, and agents. They need comprehensive visibility into all their security risks- including agent sprawl and resource access, data oversharing and leaks, AI threats and vulnerabilities, and complying with global regulations. They want to address these risks by extending their existing security investments that they are already invested in and familiar with, rather than using siloed or bolt-on tools. These teams can be most effective in delivering trustworthy AI to their organizations if security is natively integrated into the tools and platforms that they use every day, and if those tools and platforms share consistent security primitives such as agent identities from Entra; data security and compliance controls from Purview; and security posture, detections, and protections from Defender. With the new capabilities being released today, we are delivering observability at every layer of the AI stack, meeting IT, developers, and security teams where they are in the tools they already use to innovate with confidence. For IT Teams - Introducing Microsoft Agent 365, the control plane for agents, now in preview The best infrastructure for managing your agents is the one you already use to manage your users. With Agent 365, organizations can extend familiar tools and policies to confidently deploy and secure agents, without reinventing the wheel. By using the same trusted Microsoft 365 infrastructure, productivity apps, and protections, organizations can now apply consistent and familiar governance and security controls that are purpose-built to protect against agent-specific threats and risks. gement and governance of agents across organizations Microsoft Agent 365 delivers a unified agent Registry, Access Control, Visualization, Interoperability, and Security capabilities for your organization. These capabilities work together to help organizations manage agents and drive business value. The Registry powered by the Entra provides a complete and unified inventory of all the agents deployed and used in your organization including both Microsoft and third-party agents. Access Control allows you to limit the access privileges of your agents to only the resources that they need and protect their access to resources in real time. Visualization gives organizations the ability to see what matters most and gain insights through a unified dashboard, advanced analytics, and role-based reporting. Interop allows agents to access organizational data through Work IQ for added context, and to integrate with Microsoft 365 apps such as Outlook, Word, and Excel so they can create and collaborate alongside users. Security enables the proactive detection of vulnerabilities and misconfigurations, protects against common attacks such as prompt injections, prevents agents from processing or leaking sensitive data, and gives organizations the ability to audit agent interactions, assess compliance readiness and policy violations, and recommend controls for evolving regulatory requirements. Microsoft Agent 365 also includes the Agent 365 SDK, part of Microsoft Agent Framework, which empowers developers and ISVs to build agents on their own AI stack. The SDK enables agents to automatically inherit Microsoft's security and governance protections, such as identity controls, data security policies, and compliance capabilities, without the need for custom integration. For more details on Agent 365, read the blog here. For Developers - Introducing Microsoft Foundry Control Plane to observe, secure and manage agents, now in preview Developers are moving fast to bring agents into production, but operating them at scale introduces new challenges and responsibilities. Agents can access tools, take actions, and make decisions in real time, which means development teams must ensure that every agent behaves safely, securely, and consistently. Today, developers need to work across multiple disparate tools to get a holistic picture of the cybersecurity and safety risks that their agents may have. Once they understand the risk, they then need a unified and simplified way to monitor and manage their entire agent fleet and apply controls and guardrails as needed. Microsoft Foundry provides a unified platform for developers to build, evaluate and deploy AI apps and agents in a responsible way. Today we are excited to announce that Foundry Control Plane is available in preview. This enables developers to observe, secure, and manage their agent fleets with built-in security, and centralized governance controls. With this unified approach, developers can now identify risks and correlate disparate signals across their models, agents, and tools; enforce consistent policies and quality gates; and continuously monitor task adherence and runtime risks. Foundry Control Plane is deeply integrated with Microsoft’s security portfolio to provide a ‘secure by design’ foundation for developers. With Microsoft Entra, developers can ensure an agent identity (Agent ID) and access controls are built into every agent, mitigating the risk of unmanaged agents and over permissioned resources. With Microsoft Defender built in, developers gain contextualized alerts and posture recommendations for agents directly within the Foundry Control Plane. This integration proactively prevents configuration and access risks, while also defending agents from runtime threats in real time. Microsoft Purview’s native integration into Foundry Control Plane makes it easy to enable data security and compliance for every Foundry-built application or agent. This allows Purview to discover data security and compliance risks and apply policies to prevent user prompts and AI responses from safety and policy violations. In addition, agent interactions can be logged and searched for compliance and legal audits. This integration of the shared security capabilities, including identity and access, data security and compliance, and threat protection and posture ensures that security is not an afterthought; it’s embedded at every stage of the agent lifecycle, enabling you to start secure and stay secure. For more details, read the blog. For Security Teams - Introducing Security Dashboard for AI - unified risk visibility for CISOs and AI risk leaders, coming soon AI proliferation in the enterprise, combined with the emergence of AI governance committees and evolving AI regulations, leaves CISOs and AI risk leaders needing a clear view of their AI risks, such as data leaks, model vulnerabilities, misconfigurations, and unethical agent actions across their entire AI estate, spanning AI platforms, apps, and agents. 90% of security professionals, including CISOs, report that their responsibilities have expanded to include data governance and AI oversight within the past year. 1 At the same time, 86% of risk managers say disconnected data and systems lead to duplicated efforts and gaps in risk coverage. 2 To address these needs, we are excited to introduce the Security Dashboard for AI. This serves as a unified dashboard that aggregates posture and real-time risk signals from Microsoft Defender, Microsoft Entra, and Microsoft Purview. This unified dashboard allows CISOs and AI risk leaders to discover agents and AI apps, track AI posture and drift, and correlate risk signals to investigate and act across their entire AI ecosystem. For example, you can see your full AI inventory and get visibility into a quarantined agent, flagged for high data risk due to oversharing sensitive information in Purview. The dashboard then correlates that signal with identity insights from Entra and threat protection alerts from Defender to provide a complete picture of exposure. From there, you can delegate tasks to the appropriate teams to enforce policies and remediate issues quickly. With the Security Dashboard for AI, CISOs and risk leaders gain a clear, consolidated view of AI risks across agents, apps, and platforms—eliminating fragmented visibility, disconnected posture insights, and governance gaps as AI adoption scales. Best of all, there’s nothing new to buy. If you’re already using Microsoft security products to secure AI, you’re already a Security Dashboard for AI customer. Figure 5: Security Dashboard for AI provides CISOs and AI risk leaders with a unified view of their AI risk by bringing together their AI inventory, AI risk, and security recommendations to strengthen overall posture Together, these innovations deliver observability and security across IT, development, and security teams, powered by Microsoft’s shared security capabilities. With Microsoft Agent 365, IT teams can manage and secure agents alongside users. Foundry Control Plane gives developers unified governance and lifecycle controls for agent fleets. Security Dashboard for AI provides CISOs and AI risk leaders with a consolidated view of AI risks across platforms, apps, and agents. Added innovation to secure and govern your AI workloads In addition to the IT, developer, and security leader-focused innovations outlined above, we continue to accelerate our pace of innovation in Microsoft Entra, Microsoft Purview, and Microsoft Defender to address the most pressing needs for securing and governing your AI workloads. These needs are: Manage agent sprawl and resource access e.g. managing agent identity, access to resources, and permissions lifecycle at scale Prevent data oversharing and leaks e.g. protecting sensitive information shared in prompts, responses, and agent interactions Defend against shadow AI, new threats, and vulnerabilities e.g. managing unsanctioned applications, preventing prompt injection attacks, and detecting AI supply chain vulnerabilities Enable AI governance for regulatory compliance e.g. ensuring AI development, operations, and usage comply with evolving global regulations and frameworks Manage agent sprawl and resource access 76% of business leaders expect employees to manage agents within the next 2–3 years. 3 Widespread adoption of agents is driving the need for visibility and control, which includes the need for a unified registry, agent identities, lifecycle governance, and secure access to resources. Today, Microsoft Entra provides robust identity protection and secure access for applications and users. However, organizations lack a unified way to manage, govern, and protect agents in the same way they manage their users. Organizations need a purpose-built identity and access framework for agents. Introducing Microsoft Entra Agent ID, now in preview Microsoft Entra Agent ID offers enterprise-grade capabilities that enable organizations to prevent agent sprawl and protect agent identities and their access to resources. These new purpose-built capabilities enable organizations to: Register and manage agents: Get a complete inventory of the agent fleet and ensure all new agents are created with an identity built-in and are automatically protected by organization policies to accelerate adoption. Govern agent identities and lifecycle: Keep the agent fleet under control with lifecycle management and IT-defined guardrails for both agents and people who create and manage them. Protect agent access to resources: Reduce risk of breaches, block risky agents, and prevent agent access to malicious resources with conditional access and traffic inspection. Agents built in Microsoft Copilot Studio, Microsoft Foundry, and Security Copilot get an Entra Agent ID built-in at creation. Developers can also adopt Entra Agent ID for agents they build through Microsoft Agent Framework, Microsoft Agent 365 SDK, or Microsoft Entra Agent ID SDK. Read the Microsoft Entra blog to learn more. Prevent data oversharing and leaks Data security is more complex than ever. Information Security Media Group (ISMG) reports that 80% of leaders cite leakage of sensitive data as their top concern. 4 In addition to data security and compliance risks of generative AI (GenAI) apps, agents introduces new data risks such as unsupervised data access, highlighting the need to protect all types of corporate data, whether it is accessed by employees or agents. To mitigate these risks, we are introducing new Microsoft Purview data security and compliance capabilities for Microsoft 365 Copilot and for agents and AI apps built with Copilot Studio and Microsoft Foundry, providing unified protection, visibility, and control for users, AI Apps, and Agents. New Microsoft Purview controls safeguard Microsoft 365 Copilot with real-time protection and bulk remediation of oversharing risks Microsoft Purview and Microsoft 365 Copilot deliver a fully integrated solution for protecting sensitive data in AI workflows. Based on ongoing customer feedback, we’re introducing new capabilities to deliver real-time protection for sensitive data in M365 Copilot and accelerated remediation of oversharing risks: Data risk assessments: Previously, admins could monitor oversharing risks such as SharePoint sites with unprotected sensitive data. Now, they can perform item-level investigations and bulk remediation for overshared files in SharePoint and OneDrive to quickly reduce oversharing exposure. Data Loss Prevention (DLP) for M365 Copilot: DLP previously excluded files with sensitivity labels from Copilot processing. Now in preview, DLP also prevents prompts that include sensitive data from being processed in M365 Copilot, Copilot Chat, and Copilot agents, and prevents Copilot from using sensitive data in prompts for web grounding. Priority cleanup for M365 Copilot assets: Many organizations have org-wide policies to retain or delete data. Priority cleanup, now generally available, lets admins delete assets that are frequently processed by Copilot, such as meeting transcripts and recordings, on an independent schedule from the org-wide policies while maintaining regulatory compliance. On-demand classification for meeting transcripts: Purview can now detect sensitive information in meeting transcripts on-demand. This enables data security admins to apply DLP policies and enforce Priority cleanup based on the sensitive information detected. & bulk remediation Read the full Data Security blog to learn more. Introducing new Microsoft Purview data security capabilities for agents and apps built with Copilot Studio and Microsoft Foundry, now in preview Microsoft Purview now extends the same data security and compliance for users and Copilots to agents and apps. These new capabilities are: Enhanced Data Security Posture Management: A centralized DSPM dashboard that provides observability, risk assessment, and guided remediation across users, AI apps, and agents. Insider Risk Management (IRM) for Agents: Uniquely designed for agents, using dedicated behavioral analytics, Purview dynamically assigns risk levels to agents based on their risky handing of sensitive data and enables admins to apply conditional policies based on that risk level. Sensitive data protection with Azure AI Search: Azure AI Search enables fast, AI-driven retrieval across large document collections, essential for building AI Apps. When apps or agents use Azure AI Search to index or retrieve data, Purview sensitivity labels are preserved in the search index, ensuring that any sensitive information remains protected under the organization’s data security & compliance policies. For more information on preventing data oversharing and data leaks - Learn how Purview protects and governs agents in the Data Security and Compliance for Agents blog. Defend against shadow AI, new threats, and vulnerabilities AI workloads are subject to new AI-specific threats like prompt injections attacks, model poisoning, and data exfiltration of AI generated content. Although security admins and SOC analysts have similar tasks when securing agents, the attack methods and surfaces differ significantly. To help customers defend against these novel attacks, we are introducing new capabilities in Microsoft Defender that deliver end-to-end protection, from security posture management to runtime defense. Introducing Security Posture Management for agents, now in preview As organizations adopt AI agents to automate critical workflows, they become high-value targets and potential points of compromise, creating a critical need to ensure agents are hardened, compliant, and resilient by preventing misconfigurations and safeguarding against adversarial manipulation. Security Posture Management for agents in Microsoft Defender now provides an agent inventory for security teams across Microsoft Foundry and Copilot Studio agents. Here, analysts can assess the overall security posture of an agent, easily implement security recommendations, and identify vulnerabilities such as misconfigurations and excessive permissions, all aligned to the MITRE ATT&CK framework. Additionally, the new agent attack path analysis visualizes how an agent’s weak security posture can create broader organizational risk, so you can quickly limit exposure and prevent lateral movement. Introducing Threat Protection for agents, now in preview Attack techniques and attack surfaces for agents are fundamentally different from other assets in your environment. That’s why Defender is delivering purpose-built protections and detections to help defend against them. Defender is introducing runtime protection for Copilot Studio agents that automatically block prompt injection attacks in real time. In addition, we are announcing agent-specific threat detections for Copilot Studio and Microsoft Foundry agents coming soon. Defender automatically correlates these alerts with Microsoft’s industry-leading threat intelligence and cross-domain security signals to deliver richer, contextualized alerts and security incident views for the SOC analyst. Defender’s risk and threat signals are natively integrated into the new Microsoft Foundry Control Plane, giving development teams full observability and the ability to act directly from within their familiar environment. Finally, security analysts will be able to hunt across all agent telemetry in the Advanced Hunting experience in Defender, and the new Agent 365 SDK extends Defender’s visibility and hunting capabilities to third-party agents, starting with Genspark and Kasisto, giving security teams even more coverage across their AI landscape. To learn more about how you can harden the security posture of your agents and defend against threats, read the Microsoft Defender blog. Enable AI governance for regulatory compliance Global AI regulations like the EU AI Act and NIST AI RMF are evolving rapidly; yet, according to ISMG, 55% of leaders report lacking clarity on current and future AI regulatory requirements. 5 As enterprises adopt AI, they must ensure that their AI innovation aligns with global regulations and standards to avoid costly compliance gaps. Introducing new Microsoft Purview Compliance Manager capabilities to stay ahead of evolving AI regulations, now in preview Today, Purview Compliance Manager provides over 300 pre-built assessments for common industry, regional, and global standards and regulations. However, the pace of change for new AI regulations requires controls to be continuously re-evaluated and updated so that organizations can adapt to ongoing changes in regulations and stay compliant. To address this need, Compliance Manager now includes AI-powered regulatory templates. AI-powered regulatory templates enable real-time ingestion and analysis of global regulatory documents, allowing compliance teams to quickly adapt to changes as they happen. As regulations evolve, the updated regulatory documents can be uploaded to Compliance Manager, and the new requirements are automatically mapped to applicable recommended actions to implement controls across Microsoft Defender, Microsoft Entra, Microsoft Purview, Microsoft 365, and Microsoft Foundry. Automated actions by Compliance Manager further streamline governance, reduce manual workload, and strengthen regulatory accountability. Introducing expanded Microsoft Purview compliance capabilities for agents and AI apps now in preview Microsoft Purview now extends its compliance capabilities across agent-generated interactions, ensuring responsible use and regulatory alignment as AI becomes deeply embedded across business processes. New capabilities include expanded coverage for: Audit: Surface agent interactions, lifecycle events, and data usage with Purview Audit. Unified audit logs across user and agent activities, paired with traceability for every agent using an Entra Agent ID, support investigation, anomaly detection, and regulatory reporting. Communication Compliance: Detect prompts sent to agents and agent-generated responses containing inappropriate, unethical, or risky language, including attempts to manipulate agents into bypassing policies, generating risky content, or producing noncompliant outputs. When issues arise, data security admins get full context, including the prompt, the agent’s output, and relevant metadata, so they can investigate and take corrective action Data Lifecycle Management: Apply retention and deletion policies to agent-generated content and communication flows to automate lifecycle controls and reduce regulatory risk. Read about Microsoft Purview data security for agents to learn more. Finally, we are extending our data security, threat protection, and identity access capabilities to third-party apps and agents via the network. Advancing Microsoft Entra Internet Access Secure Web + AI Gateway - extend runtime protections to the network, now in preview Microsoft Entra Internet Access, part of the Microsoft Entra Suite, has new capabilities to secure access to and usage of GenAI at the network level, marking a transition from Secure Web Gateway to Secure Web and AI Gateway. Enterprises can accelerate GenAI adoption while maintaining compliance and reducing risk, empowering employees to experiment with new AI tools safely. The new capabilities include: Prompt injection protection which blocks malicious prompts in real time by extending Azure AI Prompt Shields to the network layer. Network file filtering which extends Microsoft Purview to inspect files in transit and prevents regulated or confidential data from being uploaded to unsanctioned AI services. Shadow AI Detection that provides visibility into unsanctioned AI applications through Cloud Application Analytics and Defender for Cloud Apps risk scoring, empowering security teams to monitor usage trends, apply Conditional Access, or block high-risk apps instantly. Unsanctioned MCP server blocking prevents access to MCP servers from unauthorized agents. With these controls, you can accelerate GenAI adoption while maintaining compliance and reducing risk, so employees can experiment with new AI tools safely. Read the Microsoft Entra blog to learn more. As AI transforms the enterprise, security must evolve to meet new challenges—spanning agent sprawl, data protection, emerging threats, and regulatory compliance. Our approach is to empower IT, developers, and security leaders with purpose-built innovations like Agent 365, Foundry Control Plane, and the Security Dashboard for AI. These solutions bring observability, governance, and protection to every layer of the AI stack, leveraging familiar tools and integrated controls across Microsoft Defender, Microsoft Entra, and Microsoft Purview. The future of security is ambient, autonomous, and deeply woven into the fabric of how we build, deploy, and govern AI systems. Explore additional resources Learn more about Security for AI solutions on our webpage Learn more about Microsoft Agent 365 Learn more about Microsoft Entra Agent ID Get started with Microsoft 365 Copilot Get started with Microsoft Copilot Studio Get started with Microsoft Foundry Get started with Microsoft Defender for Cloud Get started with Microsoft Entra Get started with Microsoft Purview Get started with Microsoft Purview Compliance Manager Sign up for a free Microsoft 365 E5 Security Trial and Microsoft Purview Trial 1 Bedrock Security, 2025 Data Security Confidence Index, published Mar 17, 2025. 2 AuditBoard & Ascend2, Connected Risk Report 2024; as cited by MIT Sloan Management Review, Spring 2025. 3 KPMG AI Quarterly Pulse Survey | Q3 2025. September 2025. n= 130 U.S.-based C-suite and business leaders representing organizations with annual revenue of $1 billion or more 4 First Annual Generative AI study: Business Rewards vs. Security Risks, , Q3 2023, ISMG, N=400 5 First Annual Generative AI study: Business Rewards vs. Security Risks, Q3 2023, ISMG, N=400
Improve your resiliency posture with new capabilities and intelligent assistance
At Microsoft Ignite 2025, Azure introduces intelligent automation and expanded capabilities to keep your business running—no matter what. From zonal protection and disaster recovery to ransomware defense, discover how the new AI innovations in Azure Copilot helps you move from reactive recovery to proactive resilience.
Optimize Your Cloud Environment Using Agentic AI
In today’s cloud-first world, optimization is no longer a luxury—it’s a strategic imperative. As IT professionals and developers navigate increasingly complex environments, the need to reduce costs, improve sustainability, and accelerate decision-making has never been more urgent. At Ignite 2025, Microsoft is introducing a new wave of agentic capabilities within Azure Copilot—one of the key capabilities includes the optimization agent, designed to help you identify, validate, and act on opportunities to streamline cloud operations. For FinOps teams, this agent becomes especially powerful, enabling cost governance, carbon insights, and actionable recommendations to maximize financial efficiency at scale. From Complexity to Clarity For users familiar with Azure’s cost and performance tools, the new operations center experience in the Azure Portal provides a unified agentic experience to monitor spend and carbon emissions side by side, surface the most critical optimization opportunities, and seamlessly trigger actions by invoking the Optimization agent—bringing governance, efficiency, and sustainability into one streamlined experience. What’s New in Optimization The optimization agent in Azure Copilot empowers teams to: Identify top actions prioritized by impact, cost savings, and ease of implementation. Evaluate cost and carbon impacts side-by-side, helping you make informed decisions that align with financial and sustainability goals. Validate recommendations with supporting evidence, current / projected utilization trends, and alternative SKU choices. Accelerate implementation with step-by-step guidance and agentic workflows that reduce toil and increase confidence. These capabilities are designed to scale FinOps impact, enabling collaboration across engineering, finance, procurement, and sustainability teams—all within a unified experience. A Day in the Life: FinOps in Action Let’s step into the shoes of a FinOps practitioner at a large enterprise navigating the complexities of cost management. It’s Monday morning. Over the weekend, a set of development VMs were left running, quietly accumulating costs. The optimization agent—a capability within Azure Copilot—surfaces a top action: resize or shut down the idle resources. With a few clicks, the practitioner reviews the supporting evidence, including usage trends, cost impact, and carbon footprint. The agent offers visibility over alternative SKUs and guides the practitioner through a step-by-step implementation—all within the same interface. But it doesn’t stop there. For teams that prefer automation or scripting, the agent also generates Azure CLI and PowerShell scripts tailored to the recommended action. This gives practitioners flexibility: they can execute changes directly in the portal or integrate scripts into their existing workflows for repeatability and scale. The experience is seamless—every recommendation is actionable, verifiable, and aligned with enterprise policy. By midweek, the practitioner has implemented multiple optimizations without leaving the console or writing custom code. Each action is logged for audit visibility, ensuring compliance and transparency across the organization. What used to take hours of manual investigation and coordination now happens in minutes, freeing the team to focus on strategic initiatives rather than firefighting cost overruns. Why It Matters These aren’t just features—they’re answers to the pain points customers have been voicing for years. Cost visibility and predictability: Azure Copilot centralizes insights across subscriptions, helping teams avoid surprise bills and understand where every dollar goes. Resource inefficiencies: The optimization agent proactively identifies underutilized resources and guide teams to act before costs escalate. Scalability and complexity: Azure Copilot’s unified experience simplifies operations for even the most complex setups. Azure Copilot isn’t just simplifying cloud operations—it’s transforming how teams collaborate, govern, and optimize. Get Started at Ignite At Ignite 2025, you’ll get hands-on with Azure Copilot’s optimization capabilities. Explore how intelligent assistance can help you: Reduce cloud costs Improve sustainability metrics Strengthen governance and compliance Drive better outcomes—faster Azure Copilot: turning cloud operations into intelligent collaboration. Sign up for the Agents in Azure Copilot Limited (Preview) and try the experience today.Step-by-Step Guide: Integrating Microsoft Purview with Azure Databricks and Microsoft Fabric
Co-Authored By: aryananmol, laurenkirkwood and mmanley This article provides practical guidance on setup, cost considerations, and integration steps for Azure Databricks and Microsoft Fabric to help organizations plan for building a strong data governance framework. It outlines how Microsoft Purview can unify governance efforts across cloud platforms, enabling consistent policy enforcement, metadata management, and lineage tracking. The content is tailored for architects and data leaders seeking to execute governance in scalable, hybrid environments. Note: this article focuses mainly on Data Governance features for Microsoft Purview. Why Microsoft Purview Microsoft Purview enables organizations to discover, catalog, and manage data across environments with clarity and control. Automated scanning and classification build a unified view of your data estate enriched with metadata, lineage, and sensitivity labels, and the Unified Catalog gives business-friendly search and governance constructs like domains, data products, glossary terms, and data quality. Note: Microsoft Purview Unified Catalog is being rolled out globally, with availability across multiple Microsoft Entra tenant regions; this page lists supported regions, availability dates, and deployment plans for the Unified Catalog service: Unified Catalog Supported Regions. Understanding Data Governance Features Cost in Purview Under the classic model: Data Map (Classic), users pay for an “always-on” Data Map capacity and scanning compute. In the new model, those infrastructure costs are subsumed into the consumption meters – meaning there are no direct charges for metadata storage or scanning jobs when using the Unified Catalog (Enterprise tier). Essentially, Microsoft stopped billing separately for the underlying data map and scan vCore-hours once you opt into the new model or start fresh with it. You only incur charges when you govern assets or run data processing tasks. This makes costs more predictable and tied to governance value: you can scan as much as needed to populate the catalog without worrying about scan fees and then pay only for the assets you actively manage (“govern”) and any data quality processes you execute. In summary, Purview Enterprise’s pricing is usage-based and divided into two primary areas: (1) Governed Assets and (2) Data Processing (DGPUs). Plan for Governance Microsoft Purview’s data governance framework is built on two core components: Data Map and Unified Catalog. The Data Map acts as the technical foundation, storing metadata about assets discovered through scans across your data estate. It inventories sources and organizes them into collections and domains for technical administration. The Unified Catalog sits on top as the business-facing layer, leveraging the Data Map’s metadata to create a curated marketplace of data products, glossary terms, and governance domains for data consumers and stewards. Before onboarding sources, align Unified Catalog (business-facing) and Data Map (technical inventory) and define roles, domains, and collections so ownership and access boundaries are clear. Here is a documentation that covers roles and permissions in Purview: Permissions in the Microsoft Purview portal | Microsoft Learn. The imageabove helps understand therelationship between the primary data governance solutions, Unified Catalog and Data Map, and the permissions granted by the roles for each solution. Considerations and Steps for Setting up Purview Steps for Setting up Purview: Step 1: Create a Purview Account. In the Azure Portal, use the search bar at the top to navigate to Microsoft Purview Accounts. Once there, click “Create”. This will take you to the following screen: Step 2: Click Next: Configuration and follow the Wizard, completing the necessary fields, including information on Networking, Configurations, and Tags. Then click Review + Create to create your Purview account. Consideration: Private networking: Use Private Endpoints to secure Unified Catalog/Data Map access and scan traffic; follow the new platform private endpoints guidance in the Microsoft Purview portal or migrate classic endpoints. Once your Purview Account is created, you’ll want to set up and manage your organization’s governance strategy to ensure that your data is classified and managed according to the specific lifecycle guidelines you set. Note: Follow the steps in this guide to set up Microsoft Purview Data Lifecycle Management: Data retention policy, labeling, and records management. Data Map Best Practices Design your collections hierarchy to align with organizational strategy—such as by geography, business function, or data domain. Register each data source only once per Purview account to avoid conflicting access controls. If multiple teams consume the same source, register it at a parent collection and create scans under subcollections for visibility. The imageaboveillustrates a recommended approach for structuring your Purview DataMap. Why Collection Structure Matters A well-structured Data Map strategy, including a clearly defined hierarchy of collections and domains, is critical because the Data Map serves as the metadata backbone for Microsoft Purview. It underpins the Unified Catalog, enabling consistent governance, role-based access control, and discoverability across the enterprise. Designing this hierarchy thoughtfully ensures scalability, simplifies permissions management, and provides a solid foundation for implementing enterprise-wide data governance. Purview Integration with Azure Databricks Databricks Workspace Structure In Azure Databricks, each region supports a single Unity Catalog metastore, which is shared across all workspaces within that region. This centralized architecture enables consistent data governance, simplifies access control, and facilitates seamless data sharing across teams. As an administrator, you can scan one workspace in the region using Microsoft Purview to discover and classify data managed by Unity Catalog, since the metastore governs all associated workspaces in a region. If your organization operates across multiple regions and utilizes cross-region data sharing, please review the consideration and workaround outlined below to ensure proper configuration and governance. Follow pre-requisite requirements here, before you register your workspace: Prerequisites to Connect and manage Azure Databricks Unity Catalog in Microsoft Purview. Steps to Register Databricks Workspace Step 1: In the Microsoft Purview portal, navigate to the Data Map section from the left-hand menu. Select Data Sources. Click on Register to begin the process of adding your Databricks workspace. Step 2: Note: There are two Databricks data sources, please review documentation here to review differences in capability: Connect to and manage Azure Databricks Unity Catalog in Microsoft Purview | Microsoft Learn. You can choose either source based on your organization’s needs. Recommended is “Azure Databricks Unity Catalog”: Step 3: Register your workspace. Here are the steps to register your data source: Steps to Register an Azure Databricks workspace in Microsoft Purview. Step 4: Initiate scan for your workspace, follow steps here: Steps to scan Azure Databricks to automatically identify assets. Once you have entered the required information test your connection and click continue to set up scheduled scan trigger. Step 5: For Scan trigger, choose whether to set up a schedule or run the scan once according to your business needs. Step 6: From the left pane, select Data Map and select your data source for your workspace. You can view a list of existing scans on that data source under Recent scans, or you can view all scans on the Scans tab. Review further options here: Manage and Review your Scans. You can review your scanned data sources, history and details here: Navigate to scan run history for a given scan. Limitation: The “Azure Databricks Unity Catalog” data source in Microsoft Purview does not currently support connection via Managed Vnet. As a workaround, the product team recommends using the “Azure Databricks Unity Catalog” source in combination with a Self-hosted Integration Runtime (SHIR) to enable scanning and metadata ingestion. You can find setup guidance here: Create and manage SHIR in Microsoft Purview Choose the right integration runtime configuration Scoped scan support for Unity Catalog is expected to enter private preview soon. You can sign up here: https://aka.ms/dbxpreview. Considerations: If you have delta-shared Databricks-to-Databricks workspaces, you may have duplication in your data assets if you are scanning both Workspaces. The workaround for this scenario is as you add tables/data assets to a Data Product for Governance in Microsoft Purview, you can identify the duplicated tables/data assets using their Fully Qualified Name (FQN). To make identification easier: Look for the keyword “sharing” in the FQN, which indicates a Delta-Shared table. You can also apply tags to these tables for quicker filtering and selection. The screenshot highlights how the FQN appears in the interface, helping you confidently identify and manage your data assets. Purview Integration with Microsoft Fabric Understanding Fabric Integration: Connect Cross-Tenant: This refers to integrating Microsoft Fabric resources across different Microsoft Entra tenants. It enables organizations to share data, reports, and workloads securely between separate tenants, often used in multi-organization collaborations or partner ecosystems. Key considerations include authentication, data governance, and compliance with cross-tenant policies. Connect In-Same-Tenant: This involves connecting Fabric resources within the same Microsoft Entra tenant. It simplifies integration by leveraging shared identity and governance models, allowing seamless access to data, reports, and pipelines across different workspaces or departments under the same organizational umbrella. Requirements: An Azure account with an active subscription. Create an account for free. An active Microsoft Purview account. Authentication is supported via: Managed Identity. Delegated Authentication and Service Principal. Steps to Register Fabric Tenant Step 1: In the Microsoft Purview portal, navigate to the Data Map section from the left-hand menu. Select Data Sources. Click on Register to begin the process of adding your Fabric Tenant (which also includes PowerBI). Step 2: Add in Data Source Name, keep Tenant ID as default (auto-populated). Microsoft Fabric and Microsoft Purview should be in the same tenant. Step 3: Enter in Scan name, enable/disable scanning for personal workspaces. You will notice under Credentials automatically created identity for authenticating Purview account. Note: If your Purview is behind Private Network, follow the guidelines here: Connect to your Microsoft Fabric tenant in same tenant as Microsoft Purview. Step 4: From your Microsoft Fabric, open Settings, Click on Tenant Settings and enable “Service Principals can access read-only admin APIs”, “Enhanced admin API responses within detailed metadata” and “Enhance Admin API responses with DAX and Mashup Expressions” within Admin API Settings section. Step 5: You will need to create a group, add the Purviews' managed identity to the group and add the group under “Service Principals can access read-only admin APIs” section of your tenant settings inside Microsoft Fabric Step 6: Test your connection and setup scope for your scan. Select the required workspaces, click continue and automate a scan trigger. Step 7: From the left pane, select Data Map and select your data source for your workspace. You can view a list of existing scans on that data source under Recent scans, or you can view all scans on the Scans tab. Review further options here: Manage and Review your Scans. You can review your scanned data sources, history and details here: Navigate to scan run history for a given scan. Why Customers Love Purview Kern County unified its approach to securing and governing data with Microsoft Purview, ensuring consistent compliance and streamlined data management across departments. EY accelerated secure AI development by leveraging the Microsoft Purview SDK, enabling robust data governance and privacy controls for advanced analytics and AI initiatives. Prince William County Public Schools created a more cyber-safe classroom environment with Microsoft Purview, protecting sensitive student information while supporting digital learning. FSA (Food Standards Agency) helps keep the UK food supply safe using Microsoft Purview Records Management, ensuring regulatory compliance and safeguarding critical data assets. Conclusion Purview’s Unified Catalog centralizes governance across Discovery, Catalog Management, and Health Management. The Governance features in Purview allow organizations to confidently answer critical questions: What data do we have? Where did it come from? Who is responsible for it? Is it secure and compliant? Can we trust its quality? Microsoft Purview, when integrated with Azure Databricks and Microsoft Fabric, provides a unified approach to cataloging, classifying, and governing data across diverse environments. By leveraging Purview’s Unified Catalog, Data Map, and advanced governance features, organizations can achieve end-to-end visibility, enforce consistent policies, and improve data quality. You might ask, why does data quality matter? Well, in today’s world, data is the new gold. References Microsoft Purview | Microsoft Learn Pricing - Microsoft Purview | Microsoft Azure Use Microsoft Purview to Govern Microsoft Fabric Connect to and manage Azure Databricks Unity Catalog in Microsoft Purview[Public Preview] Introducing Customizable Security Baseline Policies in Machine Configuration
Background: Azure Machine Configuration remains committed to enabling greater security and simplicity in at-scale server management for all Azure customers. Machine Configuration (previously known as Azure Policy Guest Configuration) enables both built-in and custom configuration as code allowing you to audit and configure OS, app, and workload level settings at scale, both for machines running in Azure and hybrid Azure Arc-enabled servers. We’re excited to announce Public Preview support for Customizable Security Baselines in Azure Policy and Machine Configuration. This feature empowers you to tailor industry security benchmarks—such as CIS benchmarks for Linux or Azure Security Baselines for Windows and Linux —to align with your organization’s unique compliance standards across both Azure and Arc-connected machines. This feature builds on top of our existing audit baseline capabilities for Windows and Linux. Now you can create, parameterize, and assign custom baselines at scale, enabling continuous compliance visibility across your entire environment. Learn more about how to get started here: Customize Security Baselines with Azure Policy and Machine Configuration. What's New? Customizable security baselines in Azure Policy and Machine Configuration bring a powerful new way to assess, monitor, and improve your security posture across both Windows and Linux servers. Built on industry benchmarks such as the Center for Internet Security (CIS) and Microsoft’s own Azure Compute Security Baselines, this capability enables you to adapt compliance frameworks to your organization’s specific needs — all while maintaining a consistent governance model across Azure and hybrid environments. By passing custom baseline parameters directly into Azure Policy, you can represent internal controls at scale, ensuring that compliance reflects your enterprise’s unique standards and regulatory requirements. This cloud-native approach embodies Microsoft’s Secure by Design and Secure by Default principles — ensuring your workloads stay compliant, wherever they run. Key Scenarios Baseline Customization Tailor your security standards through the Modify Settings wizard under Policy > Machine Configuration. You can: Enable, exclude, or adjust rules from existing benchmarks Apply organization-specific parameters Export your custom configuration as a downloadable JSON file Each baseline JSON file serves as a reusable, declarative artifact—ideal for policy-as-code workflows, version control, and CI/CD integration. Assign Audit Policies When you assign a baseline via Azure Policy, it automatically: Evaluates configurations against your defined standards Reports compliance in near real time Surfaces findings in Azure Policy, Azure Resource Graph, and the Guest Assignments view This integrated visibility helps IT administrators, security teams, and auditors track compliance status with minimal overhead. Integration and Automation Security baselines integrate seamlessly into your DevOps pipelines and configuration management workflows. Each baseline produces a declarative settings catalog (JSON) that can be versioned and deployed using: Azure CLI ARM templates Bicep CI/CD automation This ensures reproducible, traceable compliance configurations across environments. Supported Standards Standard Description CIS Linux Benchmarks Official CIS Benchmarks for Azure-endorsed Linux distributions, matching the latest CIS versions. Azure Compute Security Baseline for Windows Applies security controls for Windows Server 2022 and 2025, aligned with Azure Compute guidance. Azure Compute Security Baseline for Linux Enforces consistent controls aligned with Azure Compute recommendations. Availability Customizable security baselines are available in all public Azure regions. NOTE: Support for Azure Government and Sovereign Clouds will be added in a future release. These environments are not included in the current Public Preview. Getting Started Prerequisites Before you begin: Deploy the Azure Machine Configuration prerequisite policy initiative. (This installs the required Guest Configuration extension on supported VMs.) Ensure your Azure subscription or management group includes supported Windows or Linux VMs. Have sufficient permissions (Owner or Resource Policy Contributor) to create and assign custom policy definitions. Step-by-Step Guidance Select a baseline from the Machine Configuration tab in Azure Policy. Modify settings to enable, exclude, or parameterize rules to match your internal policies. Download JSON to export your customized baseline configuration file for programmatic and repeatable customization. Assign the policy which can be deployed through the Azure portal, CLI, or your CI/CD pipeline. Review compliance results to track outcomes in Azure Policy, Azure Resource Graph, or the Guest Assignments page. Learn More Azure Machine Configuration security baselines official documentation CIS Benchmark for Linux documentation Azure Windows Baseline and Azure Linux Baseline documentation Please note that the use of Azure Machine Configuration on Azure Arc-enabled servers will incur a charge.
Some Fabric Lakehouse tables not appearing in Microsoft Purview after scan
Hi everyone, I’m running into an issue where several tables from a Fabric Lakehouse aren’t appearing in Microsoft Purview after a workspace scan. Here’s the situation: I scanned a Fabric workspace that contains multiple Lakehouses. For most Lakehouses, the tables appear correctly in Purview after the scan. However, for one specific Lakehouse, several tables that I know exist aren’t showing up in the scanned assets — even after adding the Lakehouse as an asset to a data product in the Unified Catalog. What I’ve tried: I rescanned the workspace and the specific Lakehouses. I verified that the tables are persistent (not temporary) and appear under the Tables section in Fabric, not only as files. I confirmed permissions for the Purview connection account. Scan results and errors: After the rescan, the tables still didn’t appear. The scan logs show several ingestion errors with messages like: Failed to ingest asset with type fabric_lakehouse and qualified name [qualified name] due to invalid data payload to data map I checked the error entries to see which assets they point to, and none of them are related to the tables in the Lakehouse in question. There were four of these errors in the last run. Additional context: Some older Lakehouses that had been archived months ago in Fabric still appeared as active in Purview before the rescan, so there may be stale metadata being retained. Notes: I’m aware Fabric scanning in Purview currently has sub-item scanning limitations where item-level metadata is prioritised, and individual tables aren’t always picked up. But given that tables from other Lakehouses appear as expected, and given the ingestion errors (even though the errors do not point to the missing tables), it feels like there may be a metadata sync or processing issue rather than a simple coverage limitation. Question: Has anyone encountered this behaviour or the “invalid data payload to data map” error before? Any guidance on further troubleshooting steps would be appreciated. Thanks in advance!Empower Smarter AI Agent Investments
This curated series of modules is designed to equip technical and business decision-makers, including IT, developers, engineers, AI engineers, administrators, solution architects, business analysts, and technology managers, with the practical knowledge and guidance needed to make cost-conscious decisions at every stage of the AI agent journey. From identifying high-impact use cases and understanding cost drivers, to forecating ROI, adopting best practices, designing scalable and effective architectures, and optimizing ongoing investments, this learning path provides actionable guidance for building, deploying, and managing AI agents on Azure with confidence. Whether you’re just starting your AI journey or looking to scale enterprise adoption, these modules will help you align innovation with financial discipline, ensuring your AI agent initiatives deliver sustainable value and long-term success. Discover the full learning path here: aka.ms/Cost-Efficient-AI-Agents Explore the sections below for an overview of each module included in this learning path, highlighting the core concepts, practical strategies, and actionable insights designed to help you maximize the value of AI agent investments on Azure: Module 1: Identify and Prioritize High-Impact, Cost-Effective AI Agent Use Cases The journey begins with a strategic approach to selecting AI agent use cases that maximize business impact and cost efficiency. This module introduces a structured framework for researching proven use cases, collaborating across teams, and defining KPIs to evaluate feasibility and ROI. You’ll learn how to target “quick wins” while ensuring alignment with organizational goals and resource constraints. Explore this module Module 2: Understand the Key Cost Drivers of AI Agents Building on the foundation of use case selection, Module 2 dives into the core cost drivers of AI agent development and operations on Azure. It covers infrastructure, integration, data quality, team expertise, and ongoing operational expenses, offering actionable strategies to optimize spending at every stage. The module emphasizes right-sizing resources, efficient data preparation, and leveraging Microsoft tools to streamline development and ensure sustainable, scalable success. Explore this module Module 3: Forecast the Return on Investment (ROI) of AI agents With a clear understanding of costs, the next step is to quantify value. Module 3 empowers both business and technical leaders with practical frameworks for forecasting and communicating ROI, even without a finance background. Through step-by-step guides and real-world examples, you’ll learn to measure tangible and intangible outcomes, apply NPV calculations, and use sensitivity analysis to prioritize AI investments that align with broader organizational objectives. Explore this module Module 4: Implement Best Practices to Empower AI Agent Efficiency and Ensure Long-Term Success To drive efficiency and governance at scale, Module 4 introduces essential frameworks such as the AI Center of Excellence (CoE), FinOps, GenAI Ops, the Cloud Adoption Framework (CAF), and the Well-Architected Framework (WAF). These best practices help organizations accelerate adoption, optimize resources, and foster operational excellence, ensuring AI agents deliver measurable value, remain secure, and support sustainable enterprise growth. Explore this module Module 5: Maximize Cost Efficiency by Choosing the Right AI Agent Development Approach Selecting the right development approach is critical for balancing speed, customization, and cost. In Module 5, you’ll learn how to align business needs and technical skills with SaaS, PaaS, or IaaS options, empowering both business users and developers to efficiently build, deploy, and manage AI agents. The module also highlights how Microsoft Copilot Studio, Visual Studio, and Azure AI Foundry can help your organization achieve its goals. Explore this module Module 6: Architect Scalable and Cost-Efficient AI Agent Solutions on Azure As your AI initiatives grow, architectural choices become paramount. Module 6 explores how to leverage Azure Landing Zones and reference architectures for secure, well-governed, and cost-optimized deployments. It compares single-agent and multi-agent systems, highlights strategies for cost-aware model selection, and details best practices for governance, tagging, and pricing, ensuring your AI solutions remain flexible, resilient, and financially sustainable. Explore this module Module 7: Manage and Optimize AI Agent Investments on Azure The learn path concludes with a focus on operational excellence. Module 7 provides guidance on monitoring agent performance and spending using Azure AI Foundry Observability, Azure Monitor Application Insights, and Microsoft Cost Management. Learn how to track key metrics, set budgets, receive real-time alerts, and optimize resource allocation, empowering your organization to maximize ROI, stay within budget, and deliver ongoing business value. Explore this module Ready to accelerate your AI agent journey with financial confidence? Start exploring the new learning path and unlock proven strategies to maximize the cost efficiency of your AI agents on Azure, transforming innovation into measurable, sustainable business success. Get started todayAzure Policies for Automating Azure Governance - Automating Policies
In the earlier post, I covered issues and concerns organizations may face and how many built in Azure policies can address these problems. Now we are going to take it a step further and discuss how to enforce policies and automate their creation
