Getting Started with Azure SQL Data Sync Using User-Assigned Managed Identity (UAMI)
Azure SQL Data Sync is a powerful service that enables data synchronization across multiple Azure SQL Databases. Traditionally, Data Sync relied on SQL Authentication, requiring administrators to manage usernames and passwords for both hub and member databases. With the introduction of User-Assigned Managed Identity (UAMI) support, Azure SQL Data Sync now provides a more secure and modern authentication model built on Microsoft Entra ID. This enhancement reduces credential management overhead, eliminates password storage concerns, and helps organizations strengthen their security posture. In this article, we'll explore how to onboard Azure SQL Data Sync with UAMI, configure the required permissions, create synchronization components, and review best practices for long-term management. Why Use UAMI with Azure SQL Data Sync? User-Assigned Managed Identity offers several advantages over traditional SQL authentication: Eliminates password storage and rotation requirements Reduces credential exposure risks Integrates with Microsoft Entra ID authentication Supports centralized identity management Improves compliance and security posture Simplifies operational maintenance Whether you're deploying a new synchronization topology or modernizing an existing Data Sync environment, UAMI provides a secure cloud-native alternative to password-based authentication. Architecture Overview A typical UAMI-enabled Azure SQL Data Sync deployment looks like this: Authentication Method: Microsoft Entra ID via User-Assigned Managed Identity The same managed identity can be used by Data Sync to authenticate against all participating databases. Prerequisites Before configuring Azure SQL Data Sync with UAMI, ensure the following prerequisites are completed: 1. Enable Microsoft Entra Authentication The Azure SQL Server hosting both Hub and Member databases must have Microsoft Entra authentication enabled and an Entra administrator configured. 2. Create a User-Assigned Managed Identity Create a UAMI within Azure and collect the following information: Managed Identity Name Client ID Resource ID These values will be required during configuration. 3. Install the Required PowerShell Module UAMI support requires the Azure SQL preview PowerShell module: Install-Module Az.Sql -RequiredVersion 6.6.0-preview ` -AllowPrerelease ` -Force ` -AllowClobber Or a later version that includes Data Sync UAMI support. Phase 1: Configure Database Access Before Data Sync can use a managed identity, the identity must be granted access to all databases participating in synchronization. Connect to each Hub and Member database using a Microsoft Entra administrator account and create a user for the managed identity. Example: DECLARE @MSIname SYSNAME = '<UAMI_NAME>'; DECLARE @dbName SYSNAME = DB_NAME(); DECLARE @clientId UNIQUEIDENTIFIER = '<CLIENT_ID>'; -- Create User -- Grant db_datareader -- Grant db_datawriter -- Grant CONTROL permissions The UAMI must be created and granted permissions on every database participating in synchronization. Phase 2: Create the Sync Group After permissions are configured, create the Sync Group using UAMI authentication. Example PowerShell: New-AzSqlSyncGroup ` -ResourceGroupName $resourceGroup ` -ServerName $serverName ` -DatabaseName $hubDatabase ` -Name $syncGroup ` -HubDatabaseAuthenticationType UserAssigned ` -ResourceId $identityResourceId Key Parameters Parameter Description HubDatabaseAuthenticationType Authentication method for Hub Database UserAssigned Enables UAMI authentication ResourceId Full ARM Resource ID of the UAMI Phase 3: Add Sync Members Once the Sync Group is created, add member databases and specify UAMI authentication. Example: New-AzSqlSyncMember ` -ResourceGroupName $resourceGroup ` -ServerName $serverName ` -DatabaseName $hubDatabase ` -SyncGroupName $syncGroup ` -Name $memberName ` -MemberDatabaseAuthenticationType UserAssigned ` -ResourceId $identityResourceId At this stage, both Hub and Member databases are configured to authenticate using the managed identity. Phase 4: Configure Synchronization Refresh Hub Schema Refresh the schema metadata before selecting tables for synchronization. Update-AzSqlSyncSchema ` -ResourceGroupName $resourceGroup ` -ServerName $serverName ` -DatabaseName $hubDatabase ` -SyncGroupName $syncGroup Configure the Synchronization Schema Data Sync requires an explicit schema definition specifying which tables and columns will participate in synchronization. Example schema: { "Tables": [ { "QuotedName": "[dbo].[contacts]", "Columns": [ { "QuotedName": "[id]" }, { "QuotedName": "[name]" } ] } ] } Apply the schema: Update-AzSqlSyncGroup ` -ResourceGroupName $resourceGroup ` -ServerName $serverName ` -DatabaseName $hubDatabase ` -Name $syncGroup ` -HubDatabaseAuthenticationType UserAssigned ` -ResourceId $identityResourceId ` -SchemaFile "C:\schema.json" Trigger Synchronization After configuration is complete, trigger the initial synchronization. Start-AzSqlSyncGroupSync ` -ResourceGroupName $resourceGroup ` -ServerName $serverName ` -DatabaseName $hubDatabase ` -SyncGroupName $syncGroup Phase 5: Rotating UAMIs Over time, organizations may need to replace an existing managed identity. UAMI_v1 → UAMI_v2 Only one UAMI can be assigned to a Sync Group or Sync Member at a time. To rotate identities: Grant required database permissions to the new UAMI. Remove the current UAMI. Assign the new UAMI in the same operation. Update Sync Group Update-AzSqlSyncGroup ` -HubDatabaseAuthenticationType UserAssigned ` -ResourceId $newUamiResourceId ` -RemoveIdentityResourceId $oldUamiResourceId Update Sync Member Update-AzSqlSyncMember ` -MemberDatabaseAuthenticationType UserAssigned ` -ResourceId $newUamiResourceId ` -RemoveIdentityResourceId $oldUamiResourceId Common Pitfalls During implementation, the following issues are commonly encountered: Pitfall #1: Missing Entra Administrator If the SQL Server does not have an Entra administrator configured, UAMI authentication will fail. Pitfall #2: Missing Database Permissions The managed identity must be created and granted permissions in every participating database. Pitfall #3: Multiple UAMIs Assigned Azure SQL Data Sync supports only one UAMI per Sync Group or Sync Member at a time. Pitfall #4: Identity Included During PATCH Updates When updating schemas through REST APIs, avoid re-submitting the identity block if the managed identity is already assigned. This may result in a DataSyncMultipleIdentities error. Validation Checklist Before moving to production, verify the following: Microsoft Entra administrator configured UAMI created successfully UAMI user created in all Hub and Member databases Required permissions granted Sync Group configured with UserAssigned authentication Sync Members configured with UserAssigned authentication Schema refreshed successfully Synchronization schema applied Initial synchronization completed successfully Data synchronized correctly between Hub and Member databases Best Practices For secure and scalable deployments: Prefer UAMI over SQL Authentication for new deployments. Use dedicated managed identities for Data Sync workloads. Follow least-privilege access principles where possible. Test configuration changes in non-production environments first. Maintain documentation for identity ownership and rotation procedures. Monitor synchronization health after identity updates. Standardize naming conventions for managed identities. References The following Microsoft resources provide additional details for implementing Azure SQL Data Sync with User-Assigned Managed Identities: 1. Configure Microsoft Entra Authentication for Azure SQL Database Configure an Entra administrator for Azure SQL Database, which is a prerequisite for UAMI-based authentication. https://learn.microsoft.com/azure/azure-sql/database/authentication-aad-configure 2. Manage User-Assigned Managed Identities Learn how to create and manage User-Assigned Managed Identities in Azure. Managed identities eliminate the need to manage credentials in code and can be reused across multiple Azure resources. Manage user-assigned managed identities using the Azure portal explains the process and prerequisites. [Manage use...soft Learn | Learn.Microsoft.com] 3. Connect to Azure SQL Using Microsoft Entra Authentication Configure and validate Entra-based connectivity for Azure SQL Database before granting permissions to the managed identity. https://learn.microsoft.com/azure/azure-sql/database/authentication-microsoft-entra-connect-to-azure-sql 4. Azure SQL Data Sync PowerShell Documentation Review PowerShell cmdlets used to create Sync Groups, Sync Members, refresh schemas, and trigger synchronization. 5. Azure SQL REST API Documentation Use REST APIs for automation and Infrastructure-as-Code deployment scenarios involving Azure SQL Data Sync. Conclusion User-Assigned Managed Identity support represents a significant security enhancement for Azure SQL Data Sync. By eliminating dependency on stored credentials and leveraging Microsoft Entra authentication, organizations can improve security, simplify operations, and reduce administrative overhead. Whether you're building a new synchronization topology or migrating from SQL Authentication, UAMI provides a modern, scalable, and cloud-native authentication model for Azure SQL Data Sync. As cloud environments continue to adopt identity-first security practices, implementing UAMI for Azure SQL Data Sync is an important step toward a more secure and manageable data platform.Azure SQL Data Sync Fails with "Cannot Insert NULL": Understanding the Root Cause
Azure SQL Data Sync is a powerful service that enables data synchronization across multiple Azure SQL Databases. While synchronization failures are relatively uncommon, one error that administrators occasionally encounter is SQL Server Error 515, indicating that a NULL value cannot be inserted into a non-nullable column. At first glance, this appears to be a straightforward data-quality problem. However, in many cases, the actual root cause lies elsewhere: inconsistencies between Azure SQL Data Sync tracking metadata and the underlying source data. This article explains: Common causes of Error 515 during synchronization How to troubleshoot the issue How to identify invalid tracking records Safe mitigation approaches to restore synchronization The Error A synchronization operation may fail with an error similar to the following: SqlException Error Code: -2146232060 SqlError Number: 515 Message: Cannot insert the value NULL into column 'column_name', table 'dbo.table_name'; column does not allow nulls. INSERT fails. SqlError Number: 3621 The statement has been terminated. Although the error references a NULL value being inserted into a destination table, the root cause is not always missing data. In many cases, the issue originates from synchronization metadata maintained by Azure SQL Data Sync. How Azure SQL Data Sync Tracks Changes Azure SQL Data Sync relies on internal tracking tables to detect and replicate data changes between Hub and Member databases. Whenever rows are inserted, updated, or deleted, synchronization metadata is recorded in tracking tables. Data Sync uses this metadata to determine what changes need to be propagated to other databases. If the tracking metadata becomes inconsistent with the actual source table contents, Data Sync may attempt to synchronize invalid records, resulting in failures such as: Cannot insert the value NULL into column... Common Root Causes Scenario 1: Schema Mismatch Between Databases One of the most common causes of synchronization failures is a schema mismatch between synchronized databases. For example: Database Column Definition Hub NULL Allowed Member A NOT NULL Member B NOT NULL If Data Sync replicates a row containing a NULL value from the Hub database, synchronization will fail when the destination database does not allow NULL values. Areas to Validate Ensure the following are identical across all synchronized databases: Column nullability (NULL vs NOT NULL) Data types Column length Constraints Primary key definitions Even small schema differences can cause synchronization failures. Scenario 2: Invalid Tracking Metadata A less obvious but frequently encountered scenario involves orphaned records in Data Sync tracking tables. This can occur when: Primary key values are updated directly Data is modified outside expected application workflows Historical tracking records become disconnected from source data Synchronization metadata references rows that no longer exist When Data Sync processes these stale entries, synchronization may fail with Error 515 even though the source data itself appears valid. Troubleshooting Process Step 1: Verify Column Definitions Begin by examining the affected table and column identified in the error message. For example: sp_help 'dbo.table_name' Review the schema on both Hub and Member databases and verify that: The affected column has the same definition everywhere NULL settings are identical Data types and lengths match If discrepancies exist, align the schemas across all synchronized databases before proceeding. Step 2: Review the Table Schema If the schema appears consistent, review the complete definition of the affected table. Pay particular attention to: Primary key columns Identity columns Constraints Nullable settings Identifying the primary key is especially important for the next validation step. Step 3: Check for Orphaned Tracking Records Run the following query against both Hub and Member databases. Replace: table_name primary_key with the actual table and primary key column names. SELECT COUNT(*) FROM DataSync.table_name_dss_tracking t WHERE sync_row_is_tombstone = 0 AND NOT EXISTS ( SELECT * FROM dbo.table_name s WHERE t.primary_key = s.primary_key ); For tables with composite primary keys, include all key columns in the comparison. How to Interpret the Results Result > 0 One or more orphaned tracking records exist. This indicates that the tracking table contains entries that reference records no longer present in the source table. This is a strong indicator that invalid synchronization metadata is causing the failure. Result = 0 No orphaned records were detected. If the synchronization error persists, further investigation should focus on schema consistency, data quality, and additional synchronization diagnostics. Mitigation Option 1: Correct Schema Differences If schema inconsistencies are found: Align the table definition across all synchronized databases. Ensure NULL and NOT NULL settings are consistent. Verify primary key definitions match. Reinitialize synchronization if necessary. After schema alignment, synchronization can typically resume successfully. Mitigation Option 2: Clean Invalid Tracking Data If orphaned tracking records are identified, remove the invalid synchronization metadata. Important: Always validate and test cleanup operations in a non-production environment before executing them in production. The following query removes tracking entries that no longer correspond to records in the source table: DELETE FROM DataSync.table_name_dss_tracking WHERE sync_row_is_tombstone = 0 AND NOT EXISTS ( SELECT * FROM dbo.table_name s WHERE DataSync.table_name_dss_tracking.primary_key = s.primary_key ); Replace: table_name primary_key with the appropriate values for your environment. After cleanup, Data Sync can rebuild valid change tracking information and synchronization typically returns to a healthy state. Additional Validation Query The following query can help identify historical deletion records that exist in tracking tables: SELECT tr.id1 FROM DataSync.table2_dss_tracking tr LEFT JOIN dbo.table2 orig ON tr.id1 = orig.id1 WHERE tr.sync_row_is_tombstone = 1 AND orig.id1 IS NULL AND tr.last_change_datetime > DATEADD(day, -20, GETUTCDATE()); This can provide additional insight into how synchronization metadata is tracking deleted records. Understanding the Underlying Cause The most important takeaway is that the NULL value reported in the synchronization error is often not the actual problem. A common sequence looks like this: A primary key value is modified directly. UPDATE dbo.table_name SET primary_key = new_value; Data Sync tracking metadata continues to reference the original key value. The source table and tracking table become inconsistent. During synchronization, Data Sync attempts to process the stale tracking record. The synchronization operation fails and surfaces a "Cannot insert the value NULL into column" error. In these scenarios, cleaning invalid tracking records resolves the inconsistency and restores successful synchronization. Best Practices to Prevent Recurrence To minimize the likelihood of synchronization failures: Keep schemas identical across all synchronized databases Avoid updating primary key values whenever possible Use surrogate keys for synchronized tables Validate schema consistency before deploying schema changes Periodically investigate Data Sync tracking tables when troubleshooting synchronization failures Test schema modifications in non-production environments before deployment Conclusion When Azure SQL Data Sync reports a: Cannot insert the value NULL into column... error, it is important not to assume that the problem is caused by missing data in the source table. A structured troubleshooting approach should include: Verifying schema consistency across synchronized databases Reviewing primary key definitions Investigating Data Sync tracking tables for orphaned records Cleaning invalid synchronization metadata when appropriate In many real-world cases, stale tracking records are the true root cause. Identifying and removing these invalid entries can restore synchronization quickly and avoid unnecessary application or schema changes. Have you encountered similar Azure SQL Data Sync issues in your environment? Share your experience and troubleshooting techniques in the comments below.🚀 Git-Driven Deployments for Microsoft Fabric Using GitHub Actions
👋 Introduction If you've been working with Microsoft Fabric, you've likely faced this question: "How do we promote Fabric items from DEV → QA → PROD reliably, consistently, and with proper governance?" Many teams default to the built-in Fabric Deployment Pipelines — and they work great for simpler scenarios. But what happens when your enterprise demands: 🔒 Centralized governance across all platforms (infra, app, and data) 📜 Full audit trail of every change tied to a Git commit ✅ Approval gates with reviewer-based promotion 🔑 Per-environment service principal isolation 🧩 Alignment with your existing DevOps standards That's exactly the problem we set out to solve. In this post, I'll walk you through a production-ready, enterprise-grade CI/CD solution for Microsoft Fabric using the fabric-cicd Python library and GitHub Actions — with zero dependency on Fabric Deployment Pipelines. 🎯 What Problem Are We Solving? Traditional Fabric promotion workflows often look like this: Step Method Problem Build in DEV workspace Fabric Portal UI ✅ Works fine Promote to QA Fabric Deployment Pipeline or manual copy ⚠️ No Git traceability Promote to PROD Fabric Deployment Pipeline with approval ⚠️ Separate governance model from app/infra CI/CD Rollback 🤷 Manual recreation ❌ No deterministic rollback path Audit "Who clicked what, when?" ❌ Limited trail The Core Issue Fabric Deployment Pipelines introduce a parallel governance model that's disconnected from how your platform and application teams already work. You end up with: 🔀 Two different promotion systems (GitHub Actions for apps, Fabric Pipelines for data) 🕳️ Governance blind spots between the two 😰 Cultural friction ("Why do data teams have a different process?") Our Approach: Git as the Single Source of Truth 📖 ┌─────────────┐ push to main ┌─────────────┐ │ Developer │ ──────────────────▶ │ GitHub │ │ commits to │ │ Actions │ │ Git repo │ │ Workflow │ └─────────────┘ └──────┬──────┘ │ ┌─────────────────┼─────────────────┐ ▼ ▼ ▼ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ 🟢 DEV │ │ 🟡 QA │ │ 🔴 PROD │ │ Auto │────▶│ Approval │────▶│ Approval │ │ Deploy │ │ Required │ │ Required │ └──────────┘ └──────────┘ └──────────┘ Every deployment originates from Git. Every promotion is traceable to a commit SHA. Every environment has its own approval gate. One pipeline model — across everything. 🏗️ Solution Architecture 📁 Repository Structure fabric-cicd-project/ │ ├── 📂 .github/ │ ├── 📂 workflows/ │ │ └── 📄 fabric-cicd.yml # GitHub Actions pipeline │ ├── 📄 CODEOWNERS # Review enforcement │ └── 📄 dependabot.yml # Automated dependency updates │ ├── 📂 config/ │ └── 📄 parameter.yml # Environment-specific parameterization │ ├── 📂 deploy/ │ ├── 📄 deploy_workspace.py # Main deployment entrypoint │ └── 📄 validate_repo.py # Pre-deployment validation │ ├── 📂 workspace/ # Fabric items (Git-integrated / PBIP) │ ├── 📄 .env.example # Environment variable template ├── 📄 .gitignore ├── 📄 ruff.toml # Python linting config ├── 📄 requirements.txt # Pinned dependencies ├── 📄 SECURITY.md # Vulnerability disclosure policy └── 📄 README.md 🔧 Key Components Component Purpose fabric-cicd Python library Deploys Fabric items from Git to workspaces (handles all Fabric API calls internally) deploy_workspace.py CLI entrypoint — authenticates, configures, deploys, logs parameter.yml Find-and-replace rules for environment-specific values (connections, lakehouse IDs, etc.) validate_repo.py Pre-flight checks — validates repo structure, parameter.yml presence, .platform files fabric-cicd.yml GitHub Actions workflow — orchestrates validate → DEV → QA → PROD ✨ Feature Deep Dive 1️⃣ Per-Environment Service Principal Isolation 🔐 Instead of a single shared service principal, each environment gets its own: DEV_TENANT_ID / DEV_CLIENT_ID / DEV_CLIENT_SECRET QA_TENANT_ID / QA_CLIENT_ID / QA_CLIENT_SECRET PROD_TENANT_ID / PROD_CLIENT_ID / PROD_CLIENT_SECRET Why this matters: 🛡️ Least-privilege access — the DEV SP can't touch PROD 🔍 Audit clarity — you know which identity deployed where 💥 Blast radius reduction — a compromised DEV secret doesn't affect PROD The deploy script automatically resolves the correct credentials based on TARGET_ENVIRONMENT, with fallback to shared FABRIC_* variables for simpler setups. 2️⃣ Environment-Specific Parameterization 🎛️ A single parameter.yml drives all environment differences: find_replace: - find: "DEV_Lakehouse" replace_with: DEV: "DEV_Lakehouse" QA: "QA_Lakehouse" PROD: "PROD_Lakehouse" - find: "dev-sql-server.database.windows.net" replace_with: DEV: "dev-sql-server.database.windows.net" QA: "qa-sql-server.database.windows.net" PROD: "prod-sql-server.database.windows.net" ✅ Same Git artifacts → different runtime bindings per environment ✅ No manual edits between promotions ✅ Easy to review in pull requests 3️⃣ Approval-Gated Promotions ✅ The GitHub Actions workflow uses GitHub Environments with reviewer requirements: Environment Trigger Approval 🟢 DEV Automatic on push to main None — deploys immediately 🟡 QA After successful DEV deploy ✅ Requires reviewer approval 🔴 PROD After successful QA deploy ✅ Requires reviewer approval Reviewers see a rich job summary in GitHub showing: 📌 Git commit SHA being deployed 🎯 Target workspace and environment 📦 Item types in scope ⏱️ Deployment duration ✅ / ❌ Final status 4️⃣ Pre-Deployment Validation 🔍 Before any deployment runs, a dedicated validate job checks: Check What It Does 📂 workspace exists Ensures Fabric items are present 📄 parameter.yml exists Ensures parameterization is configured 📄 .platform files present Validates Fabric Git integration metadata 🐍 ruff check deploy/ Lints Python code for syntax errors and bad imports If validation fails, no deployment runs — across any environment. 5️⃣ Full Git SHA Traceability 📜 Every deployment logs and surfaces the exact Git commit being deployed: Why this matters: 🔄 Rollback = git revert <sha> + push → pipeline redeploys previous state 🕵️ Audit = every PROD deployment tied to a specific commit, reviewer, and timestamp 🔀 Diff = git diff v1..v2 shows exactly what changed between deployments 6️⃣ Concurrency Control 🚦 concurrency: group: fabric-deploy-${{ github.ref }} cancel-in-progress: false Two rapid pushes to main won't cause parallel deployments fighting over the same workspace. The second run queues until the first completes. 7️⃣ Smart Path Filtering 🧠 paths-ignore: - "**.md" - "docs/**" - ".vscode/**" A README-only commit? A docs update? No deployment triggered. This saves runner minutes and avoids unnecessary approval requests for QA/PROD. 8️⃣ Retry Logic with Exponential Backoff 🔁 The deploy script wraps fabric-cicd calls with retry logic: Attempt 1 → fails (HTTP 429 rate limit) ⏳ Wait 5 seconds Attempt 2 → fails (HTTP 503 transient) ⏳ Wait 15 seconds Attempt 3 → succeeds ✅ Transient Fabric service issues don't break your pipeline — the deployment retries automatically. 9️⃣ Orphan Cleanup 🧹 Set CLEAN_ORPHANS=true and items that exist in the workspace but not in Git get removed: Workspace has: Notebook_A, Notebook_B, Notebook_C Git repo has: Notebook_A, Notebook_B → Notebook_C gets removed (orphan) This ensures your workspace exactly matches your Git state — no drift, no surprises. 🔟 Dependency Management with Dependabot 🤖 # .github/dependabot.yml updates: - package-ecosystem: "pip" schedule: interval: "weekly" - package-ecosystem: "github-actions" schedule: interval: "weekly" fabric-cicd, azure-identity, and GitHub Actions versions are automatically monitored. When updates are available, Dependabot opens a PR — keeping your pipeline secure and current. 1️⃣1️⃣ CODEOWNERS Enforcement 👥 # .github/CODEOWNERS /deploy/ @platform-team /config/ @platform-team /.github/workflows/ @platform-team Changes to deployment scripts, parameterization, or the workflow require review from the platform team. No one accidentally modifies the pipeline without oversight. 1️⃣2️⃣ Job Timeouts ⏱️ Job Timeout Validate 10 minutes Deploy (DEV/QA/PROD) 30 minutes A hung process won't burn 6 hours of runner time. It fails fast, alerts the team, and frees the runner. 1️⃣3️⃣ Security Policy 🛡️ A dedicated SECURITY.md provides: 📧 Responsible vulnerability disclosure process ⏰ 48-hour acknowledgement SLA 📋 Best practices for contributors (no secrets in code, least-privilege SPs, 90-day rotation) 🔄 The Complete Workflow Here's what happens end-to-end when a developer merges a PR: 1. 👨💻 Developer merges PR to main │ 2. 🔍 VALIDATE job runs │ ✅ Repo structure checks │ ✅ Python linting (ruff) │ ✅ parameter.yml validation │ 3. 🟢 DEPLOY-DEV job runs (automatic) │ 🔑 Authenticates with DEV SP │ 📦 Deploys all items to DEV workspace │ 📝 Logs commit SHA + summary │ 4. 🟡 DEPLOY-QA job waits for approval │ 👀 Reviewer checks job summary │ ✅ Reviewer approves │ 🔑 Authenticates with QA SP │ 📦 Deploys all items to QA workspace │ 5. 🔴 DEPLOY-PROD job waits for approval │ 👀 Reviewer checks job summary │ ✅ Reviewer approves │ 🔑 Authenticates with PROD SP │ 📦 Deploys all items to PROD workspace │ 6. 🎉 Done — all environments in sync with Git 🆚 Comparison: This Approach vs. Fabric Deployment Pipelines Capability Fabric Deployment Pipelines This Solution (fabric-cicd + GitHub Actions) Source of truth Workspace ✅ Git Promotion trigger UI click / API call ✅ Git push + approval Approval gates Fabric-native ✅ GitHub Environments (same as app teams) Audit trail Fabric activity log ✅ Git commits + GitHub Actions history Rollback Manual ✅ git revert + auto-redeploy Cross-platform governance Separate model ✅ Unified with infra/app CI/CD Parameterization Deployment rules ✅ parameter.yml (reviewable in PR) Secret management Fabric-managed ✅ GitHub Secrets + per-env SP isolation Drift detection Limited ✅ Orphan cleanup (CLEAN_ORPHANS=true) 🚀 Getting Started Prerequisites 3 Fabric workspaces (DEV, QA, PROD) Service principal(s) with Contributor role on each workspace GitHub repository with Actions enabled GitHub Environments configured (dev, qa, prod) Quick Setup # 1. Clone the repo git clone https://github.com/<your-org>/fabric-cicd-project.git # 2. Install dependencies pip install -r requirements.txt # 3. Copy and fill environment variables cp .env.example .env # 4. Run locally against DEV python deploy/deploy_workspace.py GitHub Actions Setup Create GitHub Environments: dev, qa (add reviewers), prod (add reviewers) Add secrets to each environment: DEV_TENANT_ID, DEV_CLIENT_ID, DEV_CLIENT_SECRET QA_TENANT_ID, QA_CLIENT_ID, QA_CLIENT_SECRET PROD_TENANT_ID, PROD_CLIENT_ID, PROD_CLIENT_SECRET DEV_WORKSPACE_ID, QA_WORKSPACE_ID, PROD_WORKSPACE_ID Push to main — the pipeline takes over! 🎉 💡 Lessons Learned After implementing this pattern across several engagements, here are the key takeaways: ✅ What Works Well Teams love the Git traceability once they experience a clean rollback Approval gates in GitHub feel natural to platform engineers Parameter.yml changes in PRs create great review conversations about environment differences Job summaries give reviewers confidence to approve without digging into logs ⚠️ Watch Out For Cultural resistance is the #1 blocker — invest in enablement, not just automation Fabric items with runtime state (data in lakehouses, refresh history) aren't captured in Git Secret rotation across 3+ environments needs process discipline (consider OIDC federated credentials) Run a "portal vs. pipeline" side-by-side demo early — it changes minds fast 🤝 For CSAs: Sharing This With Customers This solution is ideal for customers who: ☑️ Already use GitHub Actions for application or infrastructure CI/CD ☑️ Have governance requirements that demand Git-based audit trails ☑️ Operate multiple Fabric workspaces across environments ☑️ Want to standardize their promotion model across all workloads ☑️ Are moving from Power BI Premium to Fabric and want to modernize their DevOps practices 🗣️ Conversation Starters "How are you promoting Fabric items between environments today?" "Is your data team using the same CI/CD patterns as your app teams?" "If something goes wrong in production, how quickly can you roll back to the previous version?" 📚 Resources 📦 fabric-cicd on PyPI 📖 fabric-cicd Documentation 🐙 GitHub Actions Documentation 🏗️ Microsoft Fabric Git Integration 🌐Git Repository URL: vinod-soni-microsoft/FABRIC-CICD-PROJECT: Enterprise-grade CI/CD solution for Microsoft Fabric using fabric-cicd Python library and GitHub Actions. Git-driven deployments across DEV → QA → PROD with environment approval gates, per-environment service principal isolation, and parameterized promotion — no Fabric Deployment Pipelines required. 🏁 Conclusion The shift from UI-driven promotion to Git-driven CI/CD for Microsoft Fabric isn't just a technical upgrade — it's a governance and cultural alignment decision. By using fabric-cicd with GitHub Actions, you get: 📖 One source of truth (Git) 🔄 One promotion model (GitHub Actions) ✅ One approval process (GitHub Environments) 🔍 One audit trail (Git history + Actions logs) 🔐 One security model (GitHub Secrets + per-env SPs) No parallel governance. No hidden drift. No "who clicked what in the portal." Just Git, code, and confidence. 💪 Have questions or want to share your experience? Drop a comment below — I'd love to hear how your team is approaching Fabric CI/CD! 👇
