Service Principals in Microsoft Power Platform
Shared Passwords Are a Ticking Time Bomb Let’s be blunt: if your Power Platform automation runs on a shared service account password that three people know, you are one resignation away from every flow going dark, one password reset, one MFA change… That is all it takes. The fix? A service principal, a non‑human application identity in Microsoft Entra ID that authenticates with certificates, never logs in interactively, and keeps running no matter who leaves. In Power Platform, it shows up as an Application User in Dataverse and can own flows, manage Power BI datasets, run Dataverse operations, and power your CI/CD pipelines. Unlike a traditional shared service account (which carries the risk of password expiration, MFA prompts breaking automation, and credentials being overshared), a service principal authenticates using certificates or client secrets, has no mailbox, no interactive login, and cannot be accidentally used by a person. For organizations running Power Automate flows, calling APIs, running Power Platform Pipelines or managing environments programmatically, service principals offer a fundamentally more secure, auditable, and manageable identity. They enable least-privilege access, integrate cleanly with Conditional Access policies, and eliminate the single point of failure that comes with tying critical automation to an individual employee's account. This lack of tether to an individual account means workloads do not need to be reassigned once someone leaves your organization. Put more simply, if your Power Platform workloads are still running under a named user or a shared "svc_powerautomate@company.com" account, it's time to reconsider. Microsoft Advisory: “Best practice is to use service principals as the preferred identity model for Power Automate wherever supported, because shared user‑based service accounts introduce security, audit, and operational risks.” In this blog, we will highlight opportunities to strengthen your security posture across the Power Platform with Service Principals alongside ideas for when a Service Principal may not be applicable for your scenario. Getting Started: Setting up a service principal is straightforward. Everything is documented on Microsoft Learn: 🔗Register an app in Entra ID — Microsoft Learn 🔗Manage application users — Microsoft Learn 🔗Service principal owned flows — Microsoft Learn There are also many good blogs about step-by-step setup of the service principal, assigning permissions and assigning to an application user we won't be covering the setup here. Let's go into detail how you can utilize a Service Principal to secure your Power Platform workloads. Where Service Principals Shine in the Power Platform ⚡ Power Automate In Power Automate, service principals establish durable ownership and authentication for enterprise automation. Flows run under a non‑human application identity, eliminating dependency on individual users and preventing failures caused by password rotation, MFA enforcement, or user departure. Flow Ownership: Assign the service principal as the primary owner to ensure flows continue running regardless of personnel changes. Service Principal‑owned flows require either a Process license (~$150/flow/month, stackable up to 10 for 2.5M actions/day) or pay‑as‑you‑go billing via Azure. Authentication: Service principals authenticate using client credentials, removing the standard 90‑day connection expiry. Only the client secret or certificate has an expiration, which can be configured (up to 24 months). For production workloads, store secrets in Azure Key Vault with rotation alerts. Dataverse Connector: Dataverse is the only standard connector with native service principal sign‑in. Selecting Connect with Service Principal ensures all Dataverse actions execute under the application identity with full audit attribution. Non‑Service Principal Connectors: Connectors such as SharePoint, Outlook, and Teams require delegated user context by design. Where app‑only execution is required, the HTTP connector can be used to call Microsoft Graph with application permissions, introducing a premium dependency. ✓ Service Principal‑Friendly ✓ Microsoft Dataverse (native SP sign‑in) ✓ Custom connectors (app‑only OAuth) ✓ HTTP via Graph API (app permissions) ⚠ Requires User Context ⚠ SharePoint (standard connector) ⚠ Outlook / Office 365 ⚠ Teams (many actions) ⚠ Planner, OneDrive 🔗Support for service principal owned flows - Power Automate | Microsoft Learn 🔗 Manage connections to Dataverse — Microsoft Learn 📱 Power Apps and Dataverse Your Service Principal’s Application User executes Dataverse operations from flows triggered by model‑driven apps. Audit logs clearly separate automated changes from human activity. Assign custom security roles scoped to exact tables (skip System Admin), use separate Service Principalss per solution area, and know that canvas apps can trigger Service Principal‑backed flows behind the scenes. 🔗Create a Dataverse application user — Microsoft Learn 📊 Power BI This is where Service Principals quietly save organizations from one of the most common Power BI failures: dataset refresh breaking because the owner left. Take over semantic model ownership via the REST API and refreshes never fail from expired tokens again. Workspace access: add Service Principal as Member or Admin Semantic model ownership takeover via REST API Automated refresh from PowerShell, Logic Apps, Azure Data Factory, or custom apps XMLA endpoint access for model deployment and DAX queries (Premium/Fabric) App Owns Data embedding for external users without Power BI licenses Requirement: Tenant admin enables “Service principals can use Fabric APIs”. Service Principal added to workspace. API refresh needs Premium, Premium Per User, or Fabric capacity. 🔗Automate Premium tasks with Service Principals — Microsoft Learn 🔗Enhanced refresh REST API — Microsoft Learn 🛠️ Application Lifecycle Management (ALM) with Pipelines Power Platform Pipelines are a built-in option for application lifecycle management native to the Power Platform. Pipelines bring ALM automation and continuous integration and continuous delivery (CI/CD) functionality into a native service that's designed to be usable by all members of your low-code team, regardless of technical capability. To learn how to set up pipelines in Power Platform, Microsoft has some fantastic documentation as well as a learning path on Learn.microsoft to guide you through the process. We can utilize Service Principals in Pipelines as well. By default, a pipeline deploys as the requesting maker, meaning the maker needs elevated permissions to deploy to the target environment. Delegated deployments with Service Principals allow deployment without the maker needing elevated permissions in the downstream environment. The pipeline can run as a service principal (or pipeline stage owner), allowing makers to deploy without needing elevated permissions in target environments like production. Approvals may be required for security reasons, and can be automated or manual depending on your security requirements. 🔗PAC CLI: createserviceprincipal - Microsoft Learn 🛡️ Alternatives for Secure Workloads While service principals offer an avenue to securing most workload scenarios on the Power Platform, they cannot cover all workload scenarios. In these cases, there are alternative approaches. Azure Logic Apps support system-assigned and user-assigned managed identities, allowing workflows to authenticate directly to Azure resources without storing or managing any login credentials. This can offer an option when your workload can live outside Power Automate. Some teams opt to retain shared service accounts. This is not best practice, but when undertaken, there are steps to harden these accounts. Dedicated Entra Conditional Access policies to enforce MFA, restricting sign in locations and devices can provide additional protection. For password rotation, Azure Key Vault centralizes credential storage and enables automatic secret rotation, reducing the risk of stale or exposed passwords. Each of these approaches can be layered or combined with service principals depending on your organization's risk tolerance, licensing constraints, and connector support limitations. 🔗Governing Entra service accounts — Microsoft Learn All Your Options at a Glance Not sure which approach fits? Here is every option compared — from the gold standard to the fallback you hope to leave behind: Option MS Rec? Key Benefit Main Tradeoff Security SP + Process License Yes Gold standard. Certificate auth, clean audit. ~$150/flow/mo. Solutions required. Highest SP + Flow Groups Yes (GA May ’26) 1 license across up to 25 flows. Shared 250K/day action pool. Highest SP + Pay-as-you-go Yes Available today. No upfront cost. Variable cost. Azure sub needed. Highest Hybrid: SP + SA Pragmatic SP owns; SA for delegated connectors. SA still exists for connections. Med‑High Hardened SA Only No Simplest and no migration needed. Interactive login risk. MS discourages. Low Logic Apps + MI Yes (diff product) Zero credentials. Consumption billing. Full platform migration required. Highest ⚠ All costs are approximate and vary by enterprise agreement. Verify with your Microsoft representative. Microsoft‑validated: The dedicated service account fallback is a supported approach, not a workaround. Any alternative to service principals would need to be designed and accepted at the customer’s own risk. Summary Service principals replace shared, user‑based accounts with a non‑human application identity that is secure, auditable, and resilient. Eliminating password risk, improving governance, and ensuring Power Platform workloads continue running independent of employee lifecycle events across the entire platform. The result is higher security, operational stability, and governance consistency. The Bottom Line Service principals are not optional polish. They are how Microsoft expects you to run enterprise automation in 2026. One Entra ID app registration. Four products. Zero shared passwords. ✓ Blocking interactive admin logins is correct. No supported scenario requires it. ✓ Hybrid model when needed. Service Principals where supported, hardened service accounts where not. Both documented by Microsoft. ✓ Start with pay‑as‑you‑go. Pilot a batch. Measure costs. Then commit. No upfront purchase needed. Authors: Sheldon Dsouza + Marc Lotorto | Contributors: Rasha Al-Silmi, Ahmed Shaalan, Josh Flicker All guidance aligned with Microsoft Learn and validated by Microsoft Advisory, April 2026Enterprise Security Assessment: A Strategic Lens for Mission Critical Environments
Understanding Enterprise Security at Scale Understanding security posture at scale requires more than isolated control reviews or point‑in‑time assessments. The Enterprise Security Assessment (ESA) helps organizations understand their security posture across Azure, Microsoft 365, and hybrid environments from a true enterprise perspective. Instead of assessing individual services or workloads in isolation, ESA provides a single, enterprise‑wide view of security. By examining identity, data security, endpoints, threat protection, and cloud infrastructure together, ESA helps uncover gaps that often span multiple teams and platforms. This broader perspective enables clearer prioritization, stronger alignment across security teams, and a more resilient foundation for long‑term security improvement. ESA complements other Microsoft assessments, such as workload‑specific reviews, by connecting the bigger picture - to align security priorities across teams and platforms, fostering a more cohesive and resilient security approach. From Standard Engagement to Strategic Partnership An Enterprise Security Assessment is typically delivered as a focused engagement designed to establish an enterprise‑wide view of security posture. At Microsoft, we begin by reviewing Secure Score insights, analyzing a defined set of core security datasets, and correlating those signals across Azure and Microsoft 365. For many organizations, this approach works well. Collecting and evaluating these datasets provides a high‑level understanding of security posture, highlights common gaps, and identifies priority improvement areas. In standard enterprise environments, ESA delivers actionable insights with minimal disruption and sets a solid foundation for security improvements. How ESA Evolves in Mission‑Critical Environments In large or mission‑critical environments, security is often distributed across multiple teams and tools. Operational constraints, regulatory requirements, and business dependencies introduce complexity that standard assessments cannot fully capture. For mission‑critical customers, ESA goes beyond a baseline review and becomes more consultative. This typically includes: 📝 Structured discovery sessions across multiple security domains 🤝 Deep‑dive workshops with specialized teams 🎯 Validation of findings against real‑world operating models 🔄 Iterative analysis to validate findings against real operational conditions This ensures recommendations reflect how security is actually managed, not just how it is documented. Why Going Deeper Matters to Customers For organizations operating at scale, this consultative ESA approach delivers significantly more than a standard readout: A realistic, enterprise‑wide understanding of security posture, grounded in actual configurations and operating models Clear visibility into cross‑team dependencies and systemic risks Prioritized recommendations aligned to existing licenses, third‑party tools, and regulatory requirements A realistic, phased security roadmap focused on adoption, not theory The result is a clear starting point for security improvements that teams can execute with confidence. A Continuous Improvement Model ESA is not a one‑time exercise. For most customers, it becomes the foundation for ongoing security maturity. Once a baseline is established, future ESAs are faster and more efficient, allowing organizations to track progress, validate improvements, and maintain alignment as environments evolve. Over time, ESA functions as an annual enterprise security health check, supported by follow‑up reviews and continuous improvement. In mission‑critical environments, this means: The first ESA requires deeper engagement investment Building cross-team alignment takes time Future assessments become smoother and more efficient once a baseline is established Over time, ESA functions as an enterprise security health check that supports continuous improvement. It works best when treated as a starting point for continuous improvement, and Enterprise Security Alignment. What Customers Gain from an Enterprise Security Assessment A true enterprise view Visibility across identity, data, devices, cloud workloads, and threat signals - without losing sight of critical details. A customized security roadmap Recommendations aligned to existing licenses, third‑party tools, hybrid footprints, and regulatory requirements - making adoption realistic, not aspirational. Momentum and measurability Many organizations track progress using dashboards or scorecards to measure improvement and sustain focus over time. Repeatability Once a baseline is established, future ESAs become easier and more efficient - serving as a regular health check rather than a brand‑new effort. A consultative model ESA delivers far more value than a one‑time assessment by fostering collaboration, shared understanding, and long‑term alignment. A Foundation for Continuous Improvement Enterprise security is complex, especially at scale. In mission‑critical environments, security success depends on embracing complexity, aligning teams, and moving beyond a standard assessment playbook. An Enterprise Security Assessment is more than a snapshot. It’s an opportunity to build alignment, inform strategy, and create a resilient security foundation that evolves with the organization.
Building a Production-Ready Azure Lighthouse Deployment Pipeline with EPAC
Recently I worked on an interesting project for an end-to-end Azure Lighthouse implementation. What really stood out to me was the combination of Azure Lighthouse, EPAC, DevOps, and workload identity federation. The deployment model was so compelling that I decided to build and validate the full solution hands-on in my own personal Azure tenants. The result is a detailed article that documents the entire journey, including pipeline design, implementation steps, and the scripts I prepared along the way. You can read the full article here
Azure Key Vault Replication: Why Paired Regions Alone Don’t Guarantee Business Continuity
As customers modernize toward multi‑region architectures in Azure, one question comes up repeatedly: “If my region goes down, will Azure Key Vault continue to work without disruption?” The short answer: it depends on what you mean by “work.” Azure Key Vault provides strong durability and availability guarantees, but those guarantees are often misunderstood—especially when customers assume paired‑region replication equals full disaster recovery. In reality, Azure Key Vault replication is designed for survivability, not uninterrupted write access or customer‑controlled failover. This post explains: How Azure Key Vault replication actually works (per Microsoft Learn) Why paired‑region failover does not equal business continuity Two reference architectures that implement true multi‑region Key Vault availability, with Terraform How Azure Key Vault Replication Works (Per Microsoft Learn) Azure Key Vault includes multiple layers of Microsoft‑managed redundancy. In‑Region and Zone Resiliency Vault contents are replicated within the region. In regions that support availability zones, Key Vault is zone‑resilient by default. This protects against localized hardware or zone failures. Paired‑Region Replication If a Key Vault is deployed in a region with an Azure‑defined paired region, its contents are asynchronously replicated to that paired region. This replication is automatic and cannot be configured, observed, or tested by customers. Microsoft‑Managed Regional Failover If Microsoft declares a full regional outage, requests are automatically routed to the paired region. After failover, the vault operates in read‑only mode: ✅ Read secrets, keys, and certificates ✅ Perform cryptographic operations ❌ Create, update, rotate, or delete secrets, keys, or certificates This is a critical distinction. Paired‑region replication preserves access — not operational continuity. Why Paired‑Region Replication Is Not Business Continuity From a reliability and DR perspective, several limitations matter: Failover is Microsoft‑initiated, not customer‑controlled No write operations during regional failover No secret rotation or certificate renewal No way to test DR Accidental deletions replicate No point‑in‑time recovery without backups Microsoft Learn explicitly states that critical workloads may require custom multi‑region strategies beyond built‑in replication. For many customers, this means Azure Key Vault becomes a single‑region dependency in an otherwise multi‑region application design. The Multi‑Region Key Vault Pattern The two GitHub repositories below implement a common architectural shift: Multiple independent Key Vaults deployed in separate regions, with customer‑controlled replication and failover. Instead of relying on invisible platform replication, the vaults become first‑class, region‑scoped resources, aligned with application failover. Solution 1: Private, Locked‑Down Multi‑Region Key Vault Replication Repository: 👉 https://github.com/jclem2000/KeyVault-MultiRegion-Replication-Private Architecture Highlights Independent Key Vault per region Private Endpoints only No public network exposure Terraform‑based deployment Controlled replication using Event Based synchronization What This Enables ✅ Full read/write access during regional outages ✅ Continued secret rotation and certificate renewal ✅ Customer‑defined failover and RTO ✅ DR testing and validation ✅ Strong alignment with zero‑trust and regulated environments Trade‑offs Higher operational complexity Requires automation and application awareness of multiple vaults Solution 2: Low‑Cost Public Multi‑Region Key Vault Replication Repository: 👉 https://github.com/jclem2000/KeyVault-MultiRegion-Replication-Public Architecture Highlights Independent Key Vault per region Public endpoints Minimal networking dependencies Terraform‑based Controlled replication using Event Based synchronization Optimized for simplicity and cost What This Enables ✅ Full read/write availability in any region ✅ Clear and testable DR posture ✅ Lower cost than private endpoint designs ✅ Suitable for many non‑regulated workloads Trade‑offs Public exposure (mitigated via firewall rules, RBAC, and conditional access) Not appropriate for all compliance requirements Requires automation and application awareness of multiple vaults Azure Native Replication vs Customer‑Managed Multi‑Region Vaults Capability Azure Paired Region Multi‑Region Vaults Read access during outage ✅ ✅ Write access during outage ❌ ✅ Secret rotation during outage ❌ ✅ Customer‑controlled failover ❌ ✅ DR testing ❌ ✅ Isolation from accidental deletion ❌ ✅ Predictable RTO ❌ ✅ Azure Key Vault’s native replication optimizes for platform durability. The multi‑region pattern optimizes for application continuity. When to Use Each Approach Paired‑Region Replication Is Often Enough When: Secrets are mostly static Read‑only access during outages is acceptable RTO is flexible You prefer Microsoft‑managed recovery Multi‑Region Vaults Are Recommended When: Secrets or certificates rotate frequently Applications must remain writable during outages Deterministic failover is required DR testing is mandatory Regulatory or operational isolation is needed Closing Thoughts Azure Key Vault behaves exactly as documented on Microsoft Learn—but it’s important to be clear about what those guarantees mean. Paired‑region replication protects your data, not your ability to operate. If your application is designed to survive a regional outage, Key Vault must follow the same multi‑region design principles as the application itself. The reference architectures above show how to extend Azure’s native durability model into true operational resilience, without waiting for a platform‑level failover decision.
Conditional Access for Canvas Apps with Entra
In today's Power Platform landscape, administrators have a tough task securing the ever-increasing inventory of Canvas Apps across their tenant. Canvas apps often connect to sensitive data, run on a variety of devices, and serve diverse groups of users. That is why Conditional Access has become one of the most powerful tools in an admin’s toolkit, giving you fine grained control over how, where, and under what conditions users can access your apps. In this post, I will walk through what Conditional Access means for canvas apps, how it empowers admins to maintain strong security without adding friction for legitimate users, and example steps to apply your own conditional access policies to an app with PowerShell. What Conditional Access Brings to Canvas Apps Conditional Access brings granular, app-level security controls from Microsoft Entra ID directly into Power Apps. Instead of applying blanket restrictions across the entire tenant, you can enforce requirements—like MFA, compliant devices, or trusted networks—only on the apps that need them. This lets you match security to the sensitivity of each individual app. Key Benefits for Admins Tailored Protection for Sensitive Apps Not every app requires strict controls. Conditional Access allows you to tighten security only for apps that handle sensitive or regulated data, without over restricting everything else. Control Access by Device Type Admins can easily block or allow specific device categories—like preventing mobile access to a high-risk app or requiring managed devices for apps that contain confidential information. Alignment With Zero Trust Conditional Access enforces identity, device, and session checks in real time, supporting a Zero Trust approach without adding unnecessary friction for legitimate users. Environment-Specific Flexibility You can apply stricter policies in production and lighter ones in development or testing, helping teams build efficiently while keeping sensitive environments locked down. A Stronger Security Model Conditional Access does not replace existing apps or data permissions—it complements them. App-level security roles control what users can do inside an app, while Conditional Access governs whether they can get into the app at all. Together, they create a much more robust security posture. How to enable conditional access for a Canvas App example In this example, I will detail steps to set up conditional access for a Canvas App to ensure tenant guest users are not able to access the app. Step 1: Create an Authentication Context in Entra ID Go to the Microsoft Entra Admin Center. Navigate to Protection → Conditional Access → Authentication context. Click + New authentication context. Name it (e.g., BlockGuests_PowerAppX) Enable Publish to apps Save and note the Authentication Context ID Step 2: Create a Conditional Access Policy Go to Conditional Access → Policies → + New policy. Name the policy (e.g., Block Guests from Power App X). Assignments: Users or workload identities: Include: Guest or external users Target resources: Choose Authentication context Select the one you created earlier Access controls: Grant: Select Block access Enable the policy and click Create. Step 3: Assign the Authentication Context to the Power App Use PowerShell to bind the Authentication Context to the specific Power App: Open PowerShell as Administrator. Connect to Power Apps Add-PowerAppsAccount Run the command to attach the context to your canvas app Set-AdminPowerAppConditionalAccessAuthenticationContextIds -EnvironmentName "<your-environment-name>" ` -AppName "<your-app-id>" ` -AuthenticationContextIds "<your-auth-context-id>" This binding tells Power Apps: “When this app opens, trigger the Conditional Access policy tied to this context.” Step 4: Test the Policy Try accessing the app as a guest user. You should see access blocked based on the Conditional Access policy. Wrap Up A Stronger Security Model Conditional Access does not replace existing apps or data permissions—it complements them. App-level security roles control what users can do inside an app, while Conditional Access governs whether they can get into the app at all. Together, they create a much more robust security posture. Bottom Line Conditional Access gives admins the flexibility to apply the right security to the right app. Whether you are enforcing MFA, restricting device types, or securing production environments, it helps you protect sensitive data without slowing down the organization. Documentation for further reading: Manage Power Apps - Power Platform | Microsoft Learn Demo from Power CAT: Conditional Access Policies for Canvas Apps - Power CAT LiveHow to Fix Azure Event Grid Entra Authentication issue for ACS and Dynamics 365 integrated Webhooks
Introduction: Azure Event Grid is a powerful event routing service that enables event-driven architectures in Azure. When delivering events to webhook endpoints, security becomes paramount. Microsoft provides a secure webhook delivery mechanism using Microsoft Entra ID (formerly Azure Active Directory) authentication through the AzureEventGridSecureWebhookSubscriber role. Problem Statement: When integrating Azure Communication Services with Dynamics 365 Contact Center using Microsoft Entra ID-authenticated Event Grid webhooks, the Event Grid subscription deployment fails with an error: "HTTP POST request failed with unknown error code" with empty HTTP status and code. For example: Important Note: Before moving forward, please verify that you have the Owner role assigned on app to create event subscription. Refer to the Microsoft guidelines below to validate the required prerequisites before proceeding: Set up incoming calls, call recording, and SMS services | Microsoft Learn Why This Happens: This happens because AzureEventGridSecureWebhookSubscriber role is NOT properly configured on Microsoft EventGrid SP (Service Principal) and event subscription entra ID or application who is trying to create event grid subscription. What is AzureEventGridSecureWebhookSubscriber Role: The AzureEventGridSecureWebhookSubscriber is an Azure Entra application role that: Enables your application to verify the identity of event senders Allows specific users/applications to create event subscriptions Authorizes Event Grid to deliver events to your webhook How It Works: Role Creation: You create this app role in your destination webhook application's Azure Entra registration Role Assignment: You assign this role to: Microsoft Event Grid service principal (so it can deliver events) Either Entra ID / Entra User or Event subscription creator applications (so they can create event grid subscriptions) Token Validation: When Event Grid delivers events, it includes an Azure Entra token with this role claim Authorization Check: Your webhook validates the token and checks for the role Key Participants: Webhook Application (Your App) Purpose: Receives and processes events App Registration: Created in Azure Entra Contains: The AzureEventGridSecureWebhookSubscriber app role Validates: Incoming tokens from Event Grid Microsoft Event Grid Service Principal Purpose: Delivers events to webhooks App ID: Different per Azure cloud (Public, Government, etc.) Public Azure: 4962773b-9cdb-44cf-a8bf-237846a00ab7 Needs: AzureEventGridSecureWebhookSubscriber role assigned Event Subscription Creator Entra or Application Purpose: Creates event subscriptions Could be: You, Your deployment pipeline, admin tool, or another application Needs: AzureEventGridSecureWebhookSubscriber role assigned Although the full PowerShell script is documented in the below Event Grid documentation, it may be complex to interpret and troubleshoot. Azure PowerShell - Secure WebHook delivery with Microsoft Entra Application in Azure Event Grid - Azure Event Grid | Microsoft Learn To improve accessibility, the following section provides a simplified step-by-step tested solution along with verification steps suitable for all users including non-technical: Steps: STEP 1: Verify/Create Microsoft.EventGrid Service Principal Azure Portal → Microsoft Entra ID → Enterprise applications Change filter to Application type: Microsoft Applications Search for: Microsoft.EventGrid Ideally, your Azure subscription should include this application ID, which is common across all Azure subscriptions: 4962773b-9cdb-44cf-a8bf-237846a00ab7. If this application ID is not present, please contact your Azure Cloud Administrator. STEP 2: Create the App Role "AzureEventGridSecureWebhookSubscriber" Using Azure Portal: Navigate to your Webhook App Registration: Azure Portal → Microsoft Entra ID → App registrations Click All applications Find your app by searching OR use the Object ID you have Click on your app Create the App Role: Display name: AzureEventGridSecureWebhookSubscriber Allowed member types: Both (Users/Groups + Applications) Value: AzureEventGridSecureWebhookSubscriber Description: Azure Event Grid Role Do you want to enable this app role?: Yes In left menu, click App roles Click + Create app role Fill in the form: Click Apply STEP 3: Assign YOUR USER to the Role Using Azure Portal: Switch to Enterprise Application view: Azure Portal → Microsoft Entra ID → Enterprise applications Search for your webhook app (by name) Click on it Assign yourself: In left menu, click Users and groups Click + Add user/group Under Users, click None Selected Search for your user account (use your email) Select yourself Click Select Under Select a role, click None Selected Select AzureEventGridSecureWebhookSubscriber Click Select Click Assign STEP 4: Assign Microsoft.EventGrid Service Principal to the Role This step MUST be done via PowerShell or Azure CLI (Portal doesn't support this directly as we have seen) so PowerShell is recommended You will need to execute this step with the help of your Entra admin. # Connect to Microsoft Graph Connect-MgGraph -Scopes "AppRoleAssignment.ReadWrite.All" # Replace this with your webhook app's Application (client) ID $webhookAppId = "YOUR-WEBHOOK-APP-ID-HERE" #starting with c5 # Get your webhook app's service principal $webhookSP = Get-MgServicePrincipal -Filter "appId eq '$webhookAppId'" Write-Host " Found webhook app: $($webhookSP.DisplayName)" # Get Event Grid service principal $eventGridSP = Get-MgServicePrincipal -Filter "appId eq '4962773b-9cdb-44cf-a8bf-237846a00ab7'" Write-Host " Found Event Grid service principal" # Get the app role $appRole = $webhookSP.AppRoles | Where-Object {$_.Value -eq "AzureEventGridSecureWebhookSubscriber"} Write-Host " Found app role: $($appRole.DisplayName)" # Create the assignment New-MgServicePrincipalAppRoleAssignment ` -ServicePrincipalId $eventGridSP.Id ` -PrincipalId $eventGridSP.Id ` -ResourceId $webhookSP.Id ` -AppRoleId $appRole.Id Write-Host "Successfully assigned Event Grid to your webhook app!" Verification Steps: Verify the App Role was created: Your App Registration → App roles You should see: AzureEventGridSecureWebhookSubscriber Verify your user assignment: Enterprise application (your webhook app) → Users and groups You should see your user with role AzureEventGridSecureWebhookSubscriber Verify Event Grid assignment: Same location → Users and groups You should see Microsoft.EventGrid with role AzureEventGridSecureWebhookSubscriber Sample Flow: Analogy For Simplification: Lets think it similar to the construction site bulding where you are the owner of the building. Building = Azure Entra app (webhook app) Building (Azure Entra App Registration for Webhook) ├─ Building Name: "MyWebhook-App" ├─ Building Address: Application ID ├─ Building Owner: You ├─ Security System: App Roles (the security badges you create) └─ Security Team: Azure Entra and your actual webhook auth code (which validates tokens) like doorman Step 1: Creat the badge (App role) You (the building owner) create a special badge: - Badge name: "AzureEventGridSecureWebhookSubscriber" - Badge color: Let's say it's GOLD - Who can have it: Companies (Applications) and People (Users) This badge is stored in your building's system (Webhook App Registration) Step 2: Give badge to the Event Grid Service: Event Grid: "Hey, I need to deliver messages to your building" You: "Okay, here's a GOLD badge for your SP" Event Grid: *wears the badge* Now Event Grid can: - Show the badge to Azure Entra - Get tokens that say "I have the GOLD badge" - Deliver messages to your webhook Step 3: Give badge to yourself (or your deployment tool) You also need a GOLD badge because: - You want to create event grid event subscriptions - Entra checks: "Does this person have a GOLD badge?" - If yes: You can create subscriptions - If no: "Access denied" Your deployment pipeline also gets a GOLD badge: - So it can automatically set up event subscriptions during CI/CD deployments Disclaimer: The sample scripts provided in this article are provided AS IS without warranty of any kind. The author is not responsible for any issues, damages, or problems that may arise from using these scripts. Users should thoroughly test any implementation in their environment before deploying to production. Azure services and APIs may change over time, which could affect the functionality of the provided scripts. Always refer to the latest Azure documentation for the most up-to-date information. Thanks for reading this blog! I hope you found it helpful and informative for this specific integration use case 😀
Microsoft Sentinel and Dataverse Integration
Integrating Microsoft Sentinel with Microsoft Dataverse brings advanced, unified security monitoring to your Power Platform environments. Microsoft Sentinel is a cloud-native SIEM (Security Information and Event Management) that collects and correlates logs from across Azure, Microsoft 365, and more to detect and respond to threats in real time. By streaming Dataverse audit logs into Sentinel, organizations gain centralized visibility into Power Platform activity and can leverage Sentinel’s analytics and automation to rapidly detect suspicious behavior and enhance governance. Benefits of Connecting Dataverse to Sentinel Unified Visibility and Threat Detection: All Dataverse audit events (e.g. record access, changes, logins) are ingested into Sentinel, where they can be correlated with signals from identities, devices, and other applications. This holistic view enables detection of suspicious patterns that might be missed in isolation. For example, security analysts can spot anomalies like: mass data exports unusual access locations sudden policy changes in Dataverse operations Sentinel provides out-of-the-box analytics rules to flag many of these risky behaviors (such as a departing user downloading large datasets or deleting records) without requiring custom queries. 2. Faster Investigation and Response: With Dataverse logs in Sentinel, analysts can use Kusto Query Language (KQL) to quickly search and correlate Dataverse activity with other security logs (identity sign-ins, Office 365 events, etc.). This speeds up root-cause analysis during incidents. Moreover, Sentinel’s SOAR capabilities mean you can trigger automated playbooks in response to Dataverse threats. For instance, if Sentinel detects an anomalous privilege escalation in Dataverse, it could automatically disable the user’s account or alert an admin via Teams. This rapid, automated response helps contain threats immediately, reducing the time to mitigate incidents. 3) Improved Governance and Compliance: Integrating Dataverse with Sentinel strengthens oversight of Power Platform usage. All audit logs are stored in Sentinel’s scalable data lake, allowing long-term retention for compliance at a lower cost than storing in Dataverse. By correlating Dataverse activity with other enterprise logs, organizations can ensure that Power Platform apps adhere to security policies and can prove compliance. Prerequisites Before deploying the integration, ensure the following prerequisites are in place: Microsoft Sentinel workspace: Microsoft Sentinel enabled in the workspace: (See Here to Onboard to a Microsoft Sentinel Workspace) 2. You must have permission to create Data Collection Rules (DCR) and Data Collection Endpoints in that workspace. Roles and permissions in the Microsoft Sentinel platform | Microsoft Learn Data collection rules in Azure Monitor - Azure Monitor | Microsoft Learn Dataverse environment: 3. The Dataverse environment must be a production environment (Dataverse logging for Sentinel is only supported for production, not sandbox). 4. Your organization should be using Dynamics 365 Customer Engagement and/or Power Platform apps (since those rely on Dataverse). 5. Audit logging enabled: Auditing must be turned on for the Dataverse environment in the Power Platform admin settings (which also routes audit logs to Microsoft Purview). Setup With prerequisites met, follow these high-level steps to set up the Sentinel integration: Enable Dataverse Auditing (if not already enabled).In the Power Platform admin center, ensuretenant-level and environment-level auditing is on. Dataverse auditing is not enabled by default, so this is critical. You’ll also need to enable auditing on the entity (table) level for all relevant Dataverse tables. Microsoft provides a managed solution to simplify this: Import the Audit Settings solution: If your environment uses Dynamics 365 CE apps, import the solution from aka.ms/AuditSettings/Dynamics; otherwise use aka.ms/AuditSettings/DataverseOnly. This managed solution will turn on detailed auditing for all standard tables (entities) in Dataverse. For any custom tables, manually enable auditing in their settings (toggle on Auditing for the entity). In each entity’s settings, under Auditing, enable the options for “Single record auditing” and “Multiple record auditing” to capture detailed create/update/delete events. Then save and publish these changes. Enabling these ensures you capture granular audit logs needed for Sentinel. 2. Install the Sentinel Solution and Connect Data Sources. In the Azure Sentinel portal Azure Portal or the Security Admin Portal Defender Portal navigate to the Content hub (or in Microsoft Defender portal, go to Content hub for Sentinel) and install the “Microsoft Sentinel Solution for Microsoft Business Applications.” This deploys all the components (analytics rules, workbooks, connectors, etc.) for Power Platform integration. Once installed, go to Configuration > Data connectors in Sentinel. You will see new connectors available for: Microsoft Dataverse Microsoft Power Platform Admin Activity Microsoft Power Automate (A connector for Dynamics 365 Finance & Operations is also included for organizations using Dynamics F&O). Follow this link for more details on F&O connector: Connect Microsoft Dynamics 365 Finance and Operations to Microsoft Sentinel | Microsoft Learn For each relevant data connector (Dataverse, Power Platform Admin, Power Automate), open its page and select Connect. This step links your Dataverse environment’s audit stream to Sentinel via the Azure Monitor DCR infrastructure. After connecting, Sentinel will start listening for the audit logs from Purview. ValidateData Ingestion. With auditing enabled and connectors in place, perform some sample activities in your Dataverse environment to generate logs (for example, create or update a row in a Dataverse table, change a Power Platform environment setting, run a Power Automate flow). In general, Dataverse and Power Automate activity logs should begin flowing into Sentinel within a few minutes, whereas Power Platform admin logs (e.g. environment or D365 admin actions) may take up to an hour for the first time. Be patient and then verify ingestion by querying the logs in Sentinel. You can run KQL queries in the Logs blade of the Sentinel Azure portal (or the Hunting section of Defender portal) to confirm data is arriving. For example, run a query on the PowerPlatformAdminActivity table to see if recent records exist. After a successful setup, Microsoft Sentinel will be populating three main Log Analytics tables with your Dataverse-related logs (as listed below): Log Analytics Table Data Collected PowerPlatformAdminActivity Power Platform administrative logs (e.g. environment settings changes, user role assignments) PowerAutomateActivity Power Automate (Flow) activity logs (creation, runs, etc.) DataverseActivity Dataverse and model-driven app business data activity logs (create, update, delete events on records, etc.) These tables contain the audit data that Sentinel will use for analysis. For instance, an update to a row in a Dataverse table will generate an event in the DataverseActivity table, whereas an administrative action like changing a Data Loss Prevention (DLP) policy or adding a user to an environment appears in PowerPlatformAdminActivity. Analytics and Threat Detection Capabilities Once the data is flowing, you can leverage Microsoft Sentinel’s powerful analytics and automation on your Dataverse logs. The Microsoft Business Apps Sentinel solution you installed comes with prebuilt analytics rules tailored to Dataverse and Power Platform scenarios. These rules will automatically generate incidents for suspicious patterns, such as: Mass record downloads or deletions (which could indicate a potential data theft or misuse) Anomalous access, like a user accessing Dataverse from an unusual location or at an odd time Privilege escalations or policy changes, e.g. a user suddenly gaining a system admin role, or a DLP policy being turned off. Security teams can also create custom detection rules using KQL to address organization-specific threats. All the Dataverse audit logs in Sentinel are fully queryable – for example, you could write a query to find when a particular record was modified and by whom, or to detect an unusual spike in Power Automate flow failures. These queries can be turned into new alert rules as needed. Sentinel provides interactive workbooks and a hunting interface to help visualize and drill into this data for proactive threat hunting. In addition to detection, Sentinel enables automated response for Power Platform incidents. Using Playbooks (Logic Apps), you might automate actions like disabling a user’s Power Platform account, alerting the IT team via Microsoft Teams, or creating a ticket in ServiceNow whenever a high-severity Dataverse incident is detected. For example, if multiple deletion events are detected in a short span on a sensitive table, a playbook could immediately notify a security channel and suspend the user's access pending investigation. This kind of end-to-end automation greatly improves your security posture by reducing response times and ensuring consistent actions. In summary, connecting Dataverse auditing to Microsoft Sentinel equips organizations with a unified view of business application activity and the advanced tools to detect, investigate, and respond to potential security issues in their Power Platform environments. It marries the rich audit data from Dataverse with Sentinel’s powerful SIEM capabilities – enabling proactive monitoring of user actions, quick identification of anomalies, and automated defense measures to protect your low-code applications. With the integration set up, Power Platform admins and security analysts can rest easier knowing that any unusual or malicious activity in Dataverse will light up on the Sentinel radar and be handled swiftly. For detailed step-by-step guidance, refer to Microsoft’s documentation on connecting Power Platform (Dataverse) to Sentinel and Connect Microsoft Dynamics 365 Finance and Operations to Microsoft Sentinel | Microsoft Learn These resources provide deeper instructions and additional tips for a successful integration.Syncing Security Groups with team membership
In this post, I present a PowerShell script to synchronize the membership between security groups and Office 365 groups, a long-standing request from Microsoft Teams admins and team owners. Source code on GitHub at https://github.com/danspot/Danspot-Scripts-and-Samples-Emporium.Preparing for Azure PostgreSQL Certificate Authority Rotation: A Comprehensive Operational Guide
The Challenge It started with a standard notification in the Azure Portal: Tracking-ID YK3N-7RZ. A routine Certificate Authority (CA) rotation for Azure Database for PostgreSQL. As Cloud Solution Architects, we’ve seen this scenario play out many times. The moment “certificate rotation” is mentioned, a wave of unease ripples through engineering teams. Let’s be honest: for many of us—ourselves included—certificates represent the edge of our technical “comfort zone.” We know they are critical for security, but the complexity of PKI chains, trust stores, and SSL handshakes can be intimidating. There is a silent fear: “If we touch this, will we break production?” We realized we had a choice. We could treat this as an opportunity, and we could leave that comfort zone. We approached our customer with a proactive proposal: Let’s use this event to stop fearing certificates and start mastering them. Instead of just patching the immediate issue, we used this rotation as a catalyst to review and upgrade the security posture of their database connections. We wanted to move from “hoping it works” to “knowing it’s secure.” The response was overwhelmingly positive. The teams didn’t just want a quick fix; they wanted “help for self-help.” They wanted to understand the mechanics behind sslmode and build the confidence to manage trust stores proactively. This guide is the result of that journey. It is designed to help you navigate the upcoming rotation not with anxiety, but with competence—turning a mandatory maintenance window into a permanent security improvement. Two Levels of Analysis A certificate rotation affects your environment on two distinct levels, requiring different expertise and actions: Level Responsibility Key Questions Actions Platform Level Cloud/Platform Teams Which clusters, services, and namespaces are affected? How do we detect at scale? Azure Service Health monitoring, AKS scanning, infrastructure-wide assessment Application Level Application/Dev Teams What SSL mode? Which trust store? How to update connection strings? Code changes, dependency updates, trust store management This article addresses both levels - providing platform-wide detection strategies (Section 5) and application-specific remediation guidance (Platform-Specific Remediation). Business Impact: In production environments, certificate validation failures cause complete database connection outages. A single missed certificate rotation has caused hours of downtime for enterprise customers, impacting revenue and customer trust. Who’s Affected: DevOps engineers, SREs, database administrators, and platform engineers managing Azure PostgreSQL instances - especially those using: - Java applications with custom JRE cacerts - Containerized workloads with baked-in trust stores - Strict SSL modes (sslmode=verify-full, verify-ca) The Solution What we’ll cover: 🛡️ Reliability: How to prevent database connection outages through proactive certificate management 🔄 Resiliency: Automation strategies that ensure your trust stores stay current 🔒 Security: Maintaining TLS security posture while rotating certificates safely Key Takeaway: This rotation is a client trust topic, not a server change. Applications trusting root CAs (DigiCert Global Root G2, Microsoft RSA Root CA 2017) without intermediate pinning are unaffected. Risk concentrates where strict validation meets custom trust stores. 📦 Platform-Specific Implementation: Detailed remediation guides for Java, .NET, Python, Node.js, and Kubernetes are available in our GitHub Repository. Note: The GitHub Repository. contains community-contributed content provided as-is. Test all scripts in non-production environments before use. 1. Understanding Certificate Authority Rotation What Changes During CA Rotation? Azure Database for PostgreSQL uses TLS/SSL to encrypt client-server connections. The database server presents a certificate chain during the TLS handshake: Certificate Chain Structure: Figure: Certificate chain structure showing the rotation from old intermediate (red, deprecated) to new intermediate (blue, active after rotation). Client applications must trust the root certificates (green) to validate the chain. 📝 Diagram Source: The Mermaid source code for this diagram is available in certificate-chain-diagram.mmd. Why Root Trust Matters Key Principle: If your application trusts the root certificate and allows the chain to be validated dynamically, you are not affected. The risk occurs when: Custom trust stores contain only the old intermediate certificate (not the root) Certificate pinning is implemented at the intermediate level Strict validation is enabled (sslmode=verify-full in PostgreSQL connection strings) 2. Who Is Affected and Why Risk Assessment Matrix Application Type Trust Store SSL Mode Risk Level Action Required Cloud-native app (Azure SDK) OS Trust Store require 🟢 Low None - Azure SDK handles automatically Java app (default JRE) System cacerts verify-ca 🟡 Medium Verify JRE version (11.0.16+, 17.0.4+, 8u381+) Java app (custom cacerts) Custom JKS file verify-full 🔴 High Update custom trust store with new intermediate .NET app (Windows) Windows Cert Store require 🟢 Low None - automatic via Windows Update Python app (certifi) certifi bundle verify-ca 🟡 Medium Update certifi package (pip install --upgrade certifi) Node.js app (default) Built-in CAs verify-ca 🟢 Low None - Node.js 16+, 18+, 20+ auto-updated Container (Alpine) /etc/ssl/certs verify-full 🔴 High Update base image or install ca-certificates-bundle Container (custom) Baked-in certs verify-full 🔴 High Rebuild image with updated trust store How to Read This Matrix Use the above matrix to quickly assess whether your applications are affected by CA rotation. Here is an overview, how you read the matrix: Column Meaning Application Type What kind of application do you have? (e.g., Java, .NET, Container) Trust Store Where does the application store its trusted certificates? SSL Mode How strictly does the application validate the server certificate? Risk Level 🟢 Low / 🟡 Medium / 🔴 High - How likely is a connection failure? Action Required What specific action do you need to take? Risk Level Logic: Risk Level Why? 🟢 Low Automatic updates (OS/Azure SDK) or no certificate validation 🟡 Medium Manual update required but straightforward (e.g., pip install --upgrade certifi) 🔴 High Custom trust store must be manually updated - highest outage risk SSL Mode Security Posture Understanding SSL modes is critical because they determine both security posture AND rotation impact. This creates a dual consideration: SSL Mode Certificate Validation Rotation Impact Security Level Recommendation disable ❌ None ✅ No impact 🔴 INSECURE Never use in production allow ❌ None ✅ No impact 🟠 WEAK Not recommended prefer ❌ Optional ✅ Minimal 🟡 WEAK Not recommended require ❌ No (Npgsql 6.0+) ✅ No impact 🟡 WEAK Upgrade to verify-full verify-ca ✅ Chain only 🔴 Critical 🔵 MODERATE Update trust stores verify-full ✅ Chain + hostname 🔴 Critical 🟢 SECURE Recommended - Update trust stores Key Insight: Applications using weak SSL modes (everything below verify-ca) are technically unaffected by CA rotation but represent security vulnerabilities. The safest path is verify-full with current trust stores. ⚖️ The Security vs. Resilience Trade-off The Paradox: Secure applications (verify-full) have the highest rotation risk 🔴, while insecure applications (require) are unaffected but have security gaps. Teams discovering weak SSL modes during rotation preparation face a critical decision: Option Approach Rotation Impact Security Impact Recommended For 🚀 Quick Fix Keep weak SSL mode (require) ✅ No action needed ⚠️ Security debt remains Emergency situations only 🛡️ Proper Fix Upgrade to verify-full 🔴 Requires trust store updates ✅ Improved security posture All production systems Our Recommendation: Use CA rotation events as an opportunity to improve your security posture. The effort to update trust stores is a one-time investment that pays off in long-term security. Common Scenarios Scenario 1: Enterprise Java Application Problem: Custom trust store created 2+ years ago for PCI compliance Risk: High - contains only old intermediate certificates Solution: Export new intermediate from Azure, import to custom cacerts Scenario 2: Kubernetes Microservices Problem: Init container copies trust store from ConfigMap at startup Risk: High - ConfigMap never updated since initial deployment Solution: Update ConfigMap, redeploy pods with new trust store Scenario 3: Legacy .NET Application Problem: .NET Framework 4.6 on Windows Server 2016 (no Windows Update) Risk: Medium - depends on manual certificate store updates Solution: Import new intermediate to Windows Certificate Store manually 3. Trust Store Overview A trust store is the collection of root and intermediate CA certificates that your application uses to validate server certificates during TLS handshakes. Understanding where your application’s trust store is located determines how you’ll update it for CA rotations. Trust Store Locations by Platform Category Platform Trust Store Location Update Method Auto-Updated? OS Level Windows Cert:\LocalMachine\Root Windows Update ✅ Yes Debian/Ubuntu /etc/ssl/certs/ca-certificates.crt apt upgrade ca-certificates ✅ Yes (with updates) Red Hat/CentOS /etc/pki/tls/certs/ca-bundle.crt yum update ca-certificates ✅ Yes (with updates) Runtime Level Java JRE $JAVA_HOME/lib/security/cacerts Java security updates ✅ With JRE updates Python (certifi) site-packages/certifi/cacert.pem pip install --upgrade certifi ❌ Manual Node.js Bundled with runtime Node.js version upgrade ✅ With Node.js updates Custom Custom JKS Application-specific path keytool -importcert ❌ Manual Container image /etc/ssl/certs (baked-in) Rebuild container image ❌ Manual ConfigMap mount Kubernetes ConfigMap Update ConfigMap, redeploy ❌ Manual Why This Matters for CA Rotation Applications using auto-updated trust stores (OS-managed, current runtime versions) generally handle CA rotations automatically. The risk concentrates in: Custom trust stores created for compliance requirements (PCI-DSS, SOC 2) that are rarely updated Baked-in container certificates from images built months or years ago Outdated runtimes (old JRE versions, frozen Python environments) that haven’t received security updates Air-gapped environments where automatic updates are disabled When planning for CA rotation, focus your assessment efforts on applications in the “Manual” update category. 4. Platform-Specific Remediation 📦 Detailed implementation guides are available in our GitHub repository: azure-certificate-rotation-guide Quick Reference: Remediation by Platform Platform Trust Store Location Update Method Guide Java $JAVA_HOME/lib/security/cacerts Update JRE or manual keytool import java-cacerts.md .NET (Windows) Windows Certificate Store Windows Update (automatic) dotnet-windows.md Python certifi package pip install --upgrade certifi python-certifi.md Node.js Built-in CA bundle Update Node.js version nodejs.md Containers Base image /etc/ssl/certs Rebuild image or ConfigMap containers-kubernetes.md Scripts & Automation Script Purpose Download State Scan-AKS-TrustStores.ps1 Scan all pods in AKS for trust store configurations PowerShell tested validate-connection.sh Test PostgreSQL connection with SSL validation Bash not tested update-cacerts.sh Update Java cacerts with new intermediate Bash not tested 5. Proactive Detection Strategies Database-Level Discovery: Identifying Connected Clients One starting point for impact assessment is querying the PostgreSQL database itself to identify which applications are connecting. We developed a SQL query that joins pg_stat_ssl with pg_stat_activity to reveal active TLS connections, their SSL version, and cipher suites. 🔍 Get the SQL Query: Download the complete detection script from our GitHub repository: detect-clients.sql Important Limitations This query has significant constraints that you must understand before relying on it for CA rotation planning: Limitation Impact Mitigation Point-in-time snapshot Only shows currently connected clients Run query repeatedly over days/weeks to capture periodic jobs and batch processes No certificate details Cannot identify which CA certificate the client is using Requires client-side investigation (trust store analysis) Connection pooling May show pooler instead of actual application Use application_name in connection strings to identify true source Idle connections Long-running connections may be dormant Cross-reference with application activity logs Recommended approach: Use this query to create an initial inventory, then investigate each unique application_name and client_addr combination to determine their trust store configuration and SSL mode. Proactive Monitoring with Azure Monitor To detect certificate-related issues before and after CA rotation, configure Azure Monitor alerts. This enables early warning when SSL handshakes start failing. Why this matters: After CA rotation, applications with outdated trust stores will fail to connect. An alert allows you to detect affected applications quickly rather than waiting for user reports. Official Documentation: For complete guidance on creating and managing alerts, see Azure Monitor Alerts Overview and Create a Log Search Alert. Here is a short example of an Azure Monitor Alert definition as a starting point. { "alertRule": { "name": "PostgreSQL SSL Connection Failures", "severity": 2, "condition": { "query": "AzureDiagnostics | where ResourceType == 'SERVERS' and Category == 'PostgreSQLLogs' and Message contains 'SSL error' | summarize count() by bin(TimeGenerated, 5m)", "threshold": 5, "timeAggregation": "Total", "windowSize": "PT5M" } } } Alert Configuration Notes: Setting Recommended Value Rationale Severity 2 (Warning) Allows investigation without triggering critical incident response Threshold 5 failures/5min Filters noise while catching genuine issues Evaluation Period 5 minutes Balances responsiveness with alert fatigue Action Group Platform Team Ensures quick triage and coordination 6. Production Validation Pre-Rotation Validation Checklist Inventory all applications connecting to Azure PostgreSQL Identify trust store locations for each application Verify root certificate presence in trust stores Test connection with new intermediate in non-production environment Update monitoring alerts for SSL connection failures Prepare rollback plan if issues occur Schedule maintenance window (if required) Notify stakeholders of potential impact Testing Procedure We established a systematic 3-step validation process to ensure zero downtime. This approach moves from isolated testing to gradual production rollout. 🧪 Technical Validation Guide: For the complete list of psql commands, connection string examples for Windows/Linux, and automated testing scripts, please refer to our Validation Guide in the GitHub repository. Connection Testing Strategy The core of our validation strategy was testing connections with explicit sslmode settings. We used the psql command-line tool to simulate different client behaviors. Test Scenario Purpose Expected Result Encryption only (sslmode=require) Verify basic connectivity Connection succeeds even with unknown CA CA validation (sslmode=verify-ca) Verify trust store integrity Connection succeeds only if CA chain is valid Full validation (sslmode=verify-full) Verify strict security compliance Connection succeeds only if CA chain AND hostname match Pro Tip: Test with verify-full and an explicit root CA file containing the new Microsoft/DigiCert root certificates before the rotation date. This validates that your trust stores will work after the intermediate certificate changes. Step 1: Test in Non-Production Validate connections against a test server using the new intermediate certificate (Azure provides test endpoints during the rotation window). Step 2: Canary Deployment Deploy the updated trust store to a single “canary” instance or pod. Monitor: - Connection success rate - Error logs - Response times Step 3: Gradual Rollout Once the canary is stable, proceed with a phased rollout: 1. Update 10% of pods 2. Monitor for 1 hour 3. Update 50% of pods 4. Monitor for 1 hour 5. Complete rollout 7. Best Practices and Lessons Learned Certificate Management Best Practices Practice Guidance Example Trust Root CAs, Not Intermediates Configure trust stores with root CA certificates only. This provides resilience against intermediate certificate rotations. Trust Microsoft TLS RSA Root G2 and DigiCert Global Root G2 instead of specific intermediates Automate Trust Store Updates Use OS-provided trust stores when possible (automatically updated). For custom trust stores, implement CI/CD pipelines. Schedule bi-annual trust store audits Use SSL Mode Appropriately Choose SSL mode based on security requirements. verify-ca is recommended for most scenarios. See Security Posture Matrix in Section 2 Maintain Container Images Rebuild container images monthly to include latest CA certificates. Use init containers for runtime updates. Multi-stage builds with CA certificate update step Avoid Certificate Pinning Never pin intermediate certificates. If pinning is required for compliance, implement automated update processes. Pin only root CA certificates if absolutely necessary SSL Mode Decision Guide SSL Mode Security Level Resilience When to Use require Medium High Encrypted traffic without certificate validation. Use when CA rotation resilience is more important than MITM protection. verify-ca High Medium Validates certificate chain. Recommended for most production scenarios. verify-full Highest Low Strictest validation with hostname matching. Use only when compliance requires it. Organizational Communication Model Effective certificate rotation requires structured communication across multiple layers: Layer Responsibility Key Action Azure Service Health Microsoft publishes announcements to affected subscriptions Monitor Azure Service Health proactively Platform/Cloud Team Receives Azure announcements, triages criticality Follow ITSM processes, assess impact Application Teams Execute application-level changes Update trust stores, validate connections Security Teams Define certificate validation policies Set compliance requirements Ownership and Responsibility Matrix Team Responsibility Deliverable Platform/Cloud Team Monitor Azure Service Health, coordinate response Impact assessment, team notifications Application Teams Application-level changes (connection strings, trust stores) Updated configurations, validation results Security Teams Define certificate policies, compliance requirements Policy documentation, audit reports All Teams (Shared) Certificate lifecycle collaboration Playbooks, escalation paths, training Certificate Rotation Playbook Components Organizations should establish documented playbooks including: Component Recommended Frequency Purpose Trust Store Audits Bi-annual (every 6 months) Ensure certificates are current Certificate Inventory Quarterly review Know what certificates exist where Playbook Updates Annual or after incidents Keep procedures current Team Training Annual Build knowledge and confidence Field Observations: Common Configuration Patterns Pattern Observation Risk Implicit SSL Mode Teams don’t explicitly set sslmode, relying on framework defaults Unexpected behavior during CA rotation Copy-Paste Configurations Connection strings copied without understanding options Works until certificate changes expose gaps Framework-Specific Defaults Java uses JRE trust store, .NET uses Windows Certificate Store, Python depends on certifi package Some require manual updates, some are automatic Framework Trust Store Defaults Framework Default Trust Store Update Method Risk Level Java/Quarkus JRE cacerts Manual or JRE update Medium - requires awareness .NET Windows Certificate Store Windows Update Low - automatic Node.js Bundled certificates Node.js version update Low - automatic Python certifi package pip install --upgrade certifi High - manual intervention required Knowledge and Confidence Challenges Challenge Impact Mitigation Limited certificate knowledge Creates uncertainty and risk-averse behavior Proactive education, hands-on workshops Topic intimidation “Certificates” can seem complex, leading to avoidance Reality: Implementation is straightforward once understood Previous negative experiences Leadership concerns based on past incidents Document successes, share lessons learned Visibility gaps Lack of visibility into application dependencies Maintain certificate inventory, use discovery tools Monitoring Strategy (Recommended for Post-Rotation): While pre-rotation monitoring focuses on inventory, post-rotation monitoring should track: Key Metrics: - Connection failure rates (group by application, SSL error types) - SSL handshake duration (detect performance degradation) - Certificate validation errors (track which certificates fail) - Application error logs (filter for “SSL”, “certificate”, “trust”) Recommended Alerts: - Threshold: >5 SSL connection failures in 5 minutes - Anomaly detection: Connection failure rate increases >50% - Certificate expiry warnings: 30, 14, 7 days before expiration Dashboard Components: - Connection success rate by application - SSL error distribution (validation failures, expired certificates, etc.) - Certificate inventory with expiry dates - Trust store update status across infrastructure These metrics, alerts and thresholds are only starting points and need to be adjusted based on your environment and needs. Post-Rotation Validation and Telemetry Note: This article focuses on preparation for upcoming certificate rotations. Post-rotation metrics and incident data will be collected after the rotation completes and can inform future iterations of this guidance. Recommended Post-Rotation Activities: Here are some thoughts on post-rotation activities that could create more insights on the effectiveness of the preparation. Incident Tracking: After rotation completes, organizations should track: - Production incidents related to SSL/TLS connection failures - Services affected and their business criticality - Mean Time to Detection (MTTD) for certificate-related issues - Mean Time to Resolution (MTTR) from detection to fix Success Metrics to Measure Pre-Rotation Validation: - Number of services inventoried and assessed - Percentage of services requiring trust store updates - Testing coverage (dev, staging, production) Post-Rotation Outcomes: - Zero-downtime success rate (percentage of services with no impact) - Applications requiring emergency patching - Time from rotation to full validation Impact Assessment Telemetry to Collect: - Total connection attempts vs. failures (before and after rotation) - Duration of any service degradation or outages - ustomer-facing impact (user-reported issues, support tickets) - Geographic or subscription-specific patterns Continuous Improvement Post-Rotation Review: - What worked well in the preparation phase? - Which teams or applications were unprepared? - What gaps exist in monitoring or alerting? - How can communication be improved for future rotations? Documentation Updates: - Update playbooks with lessons learned - Refine monitoring queries based on observed patterns - Enhance team training materials - Share anonymized case studies across the organization 8. Engagement & Next Steps Discussion Questions We’d love to hear from the community: What’s your experience with certificate rotations? Have you encountered unexpected connection failures during CA rotation events? Which trust store update method works best for your environment? OS-managed, runtime-bundled, or custom trust stores? How do you handle certificate management in air-gapped environments? What strategies have worked for your organization? Share Your Experience If you’ve implemented proactive certificate management strategies or have lessons learned from CA rotation incidents, we encourage you to: Comment below with your experiences and tips Contribute to the GitHub repository with additional platform guides or scripts Connect with us on LinkedIn to continue the conversation Call to Action Take these steps now to prepare for the CA rotation: Assess your applications - Use the Risk Assessment Matrix (Section 2) to identify which applications use sslmode=verify-ca or verify-full with custom trust stores Import root CA certificates - Add DigiCert Global Root G2 and Microsoft RSA Root CA 2017 to your trust stores Upgrade SSL mode - Change your connection strings to at least sslmode=verify-ca (recommended: verify-full) for improved security Document your changes - Record which applications were updated, what trust stores were modified, and the validation results Automate for the future - Implement proactive certificate management so future CA rotations are handled automatically (OS-managed trust stores, CI/CD pipelines for container images, scheduled trust store audits) 9. Resources Official Documentation Azure PostgreSQL: Azure PostgreSQL SSL/TLS Concepts Azure PostgreSQL - Connect with TLS/SSL PostgreSQL & libpq: PostgreSQL libpq SSL Support - SSL mode options and environment variables PostgreSQL psql Reference - Command-line tool documentation PostgreSQL Server SSL/TLS Configuration Certificate Authorities: DigiCert Root Certificates Microsoft PKI Repository Microsoft Trusted Root Program Community Resources Let’s Encrypt Root Expiration (2021 Incident) NIST SP 800-57: Key Management Guidelines OWASP Certificate Pinning Cheat Sheet Neon Blog: PostgreSQL Connection Security Defaults Tools and Scripts PowerShell AKS Trust Store Scanner (see Platform-Specific Remediation) PostgreSQL Interactive Terminal (psql) PostgreSQL JDBC SSL Documentation Industry Context Certificate rotation challenges are not unique to Azure PostgreSQL. Similar incidents have occurred across the industry: Historical Incidents: - Let’s Encrypt Root Expiration (2021): Widespread impact when DST Root CA X3 expired, affecting older Android devices and legacy systems - DigiCert Root Transitions: Multiple cloud providers experienced customer impact during CA changes - Internal PKI Rotations: Enterprises face similar challenges when rotating internally-issued certificates Relevant Standards: - NIST SP 800-57: Key Management Guidelines (certificate lifecycle best practices) - OWASP Certificate Pinning: Guidance on balancing security and operational resilience - CIS Benchmarks: Recommendations for TLS/SSL configuration in cloud environments Authors Author Role Contact Andreas Semmelmann Cloud Solution Architect, Microsoft LinkedIn Mpho Muthige Cloud Solution Architect, Microsoft LinkedIn Disclaimers Disclaimer: The information in this blog post is provided for general informational purposes only and does not constitute legal, financial, or professional advice. While every effort has been made to ensure the accuracy of the information at the time of publication, Microsoft makes no warranties or representations as to its completeness or accuracy. Product features, availability, and timelines are subject to change without notice. For specific guidance, please consult your legal or compliance advisor. Microsoft Support Statement: This article represents field experiences and community best practices. For official Microsoft support and SLA-backed guidance: Azure Support: https://azure.microsoft.com/support/ Official Documentation: https://learn.microsoft.com/azure/ Microsoft Q&A: https://learn.microsoft.com/answers/ Production Issues: Always open official support tickets for production-impacting problems. Customer Privacy Notice: This article describes real-world scenarios from customer engagements. All customer-specific information has been anonymized. No NDAs or customer confidentiality agreements were violated in creating this content. AI-generated content disclaimer: This content was generated in whole or in part with the assistance of AI tools. AI-generated content may be incorrect or incomplete. Please review and verify before relying on it for critical decisions. See terms Community Contribution: The GitHub repository referenced in this article contains community-contributed scripts and guides. These are provided as-is for educational purposes and should be tested in non-production environments before use. Tags: #AzurePostgreSQL #CertificateRotation #TLS #SSL #TrustStores #Operations #DevOps #SRE #CloudSecurity #AzureDatabase
