Lessons Learned #551: Azure SQL Connection Timeouts: Three Things to Check
An application starts reporting intermittent timeouts when connecting to Azure SQL Database. Some requests succeed, others fail, and a test from a developer’s laptop works perfectly. The database appears online, no recent deployment seems related, and the natural reaction is to ask: Is Azure SQL unavailable? Is the firewall blocking the connection? Should we increase the connection timeout? Should we change the driver or scale the database? Those are reasonable questions, but they may lead the investigation in the wrong direction. The most important lesson is simple: A timeout tells us how long the application waited. It does not tell us what the application was waiting for. Not every “SQL timeout” happens inside Azure SQL From the application’s point of view, opening a database connection may involve several operations: Resolving the server name. Reaching the SQL endpoint. Obtaining a Microsoft Entra access token. Waiting for an available pooled connection. Completing the SQL login. Executing the first command. When all these operations are reported through the same application method or log entry, it can look as though Azure SQL took thirty seconds to accept the connection. In reality, only part of that time may have been spent connecting to the database. In one anonymized support scenario, the application experienced problems mainly on its first connection. Network tests were successful and no corresponding SQL connection failure was identified. The investigation eventually showed that access-token acquisition was consuming a significant part of the available time. Increasing the SQL timeout or changing the firewall would not have addressed the real delay. Check 1: Capture the complete error and the exact time A screenshot containing only “Connection Timeout Expired” is rarely enough. Capture: The complete exception and inner exception. The operation being performed. The driver and version. The authentication method. The exact timestamp in UTC. Whether the issue affects every connection or only some of them. The wording around the timeout matters. For example, a timeout while obtaining a connection from the pool points toward the application’s pooling and concurrency behavior. A pre-login or TLS error belongs to a different investigation. A command timeout after the connection was established is usually a query-performance problem rather than a connection problem. Check 2: Measure the application timeline The application should record important operations separately. A simple timeline can completely change the investigation: 10:14:20.100 Token acquisition started 10:14:28.400 Token acquired 10:14:28.405 SQL connection started 10:14:29.050 SQL connection established The complete operation took almost nine seconds, but Azure SQL connection establishment took less than one second. Useful measurements include: Token-acquisition duration. Time waiting for a pooled connection. SQL connection-open duration. SQL command duration. Number of retry attempts. Applications using Microsoft Entra authentication must obtain an access token before authenticating to Azure SQL. Measuring that operation separately helps distinguish an identity delay from a database connectivity problem. This is particularly useful when the issue appears: On the first connection after startup. After a token expires. Only with Managed Identity or Workload Identity. Intermittently, while SQL authentication connections remain unaffected. Check 3: Test from the application environment A successful connection from a laptop does not validate the path used by an application running in: Azure App Service. Azure Functions. Azure Kubernetes Service. A virtual machine. An on-premises application server. A container or integration runtime. The laptop and the application may use different DNS servers, routes, firewalls, proxies and identities. Connectivity and DNS tests should therefore be performed from the environment that is actually failing. This becomes especially important when Private Endpoint is used. The application should continue connecting with: <server>.database.windows.net It should not use the Private Endpoint IP address or the privatelink.database.windows.net hostname directly. Direct login attempts using the private IP or the private-link FQDN fail; the normal logical-server FQDN must remain in the connection string. From the affected environment, confirm that: The expected DNS server answers the request. The server FQDN resolves to the expected private IP. The Private Endpoint connection is approved. The Private DNS zone is linked correctly. The resolved address is reachable through the intended route. A test from an unrelated machine is still useful for comparison, but it does not prove that the application path is healthy. Observed symptom Likely investigation area Timeout while obtaining a connection from the pool Application connection pooling Server name cannot be resolved DNS TCP connection to the endpoint cannot be established Network path, firewall or routing Error during the pre-login handshake TLS, driver, network interruption or pre-login processing Authentication or access-token error Microsoft Entra authentication, identity or token acquisition Timeout during the post-login phase Login completion, session initialization or server-side processing Execution or command timeout after connecting Query execution and database performance Avoid changing several things at once During a production incident, it is tempting to: Increase the timeout. Add firewall rules. Change the connection policy. Upgrade the driver. Restart the application. Clear connection pools. Applying several changes together makes it difficult to determine which one helped, and some may only hide the symptom. A better approach is to define one hypothesis: We believe DNS in the application environment is resolving the public endpoint instead of the Private Endpoint. Then define: The evidence supporting the hypothesis. One controlled change. The expected result. How the result will be measured. How the change will be reverted. Azure SQL supports Proxy and Redirect connection policies, which determine how traffic flows after reaching the Azure SQL gateway. The policy is configured for the logical server, so it should be verified before making firewall assumptions or changes. What should we collect before opening a support request? A small but precise evidence package can avoid several rounds of questions: Complete error and inner exception. Exact UTC timestamps. Application platform and location. Public or Private Endpoint. Server FQDN used by the application. Driver and version. Authentication method. Token, pool, connection and command durations. DNS result from the affected environment. Whether the issue is constant, intermittent or limited to the first connection. Recent application, network, identity or configuration changes.Announcing Automatic Backup Immutability for Azure SQL Database and Azure SQL Managed Instance
Built-in protection for your most recent backups - enabled automatically Today, we're excited to announce General Availability of automatic backup immutability up to the most recent 7 days of point-in-time restore (PITR) backups in Azure SQL Database and Azure SQL Managed Instance, at no additional cost. With this release, up to most recent 7 days of backups are automatically protected with immutability by default, regardless of your configured PITR retention period. No configuration changes, policy creation, or administrative action are required. This enhancement provides an additional layer of protection for one of your most critical recovery assets - your backups. Why backup immutability matters Cyberattacks continue to evolve, with ransomware increasingly targeting not only production data, but also backup systems. Attackers understand that if backups can be deleted, modified, or corrupted, recovery becomes significantly more difficult and costly. Traditional backup strategies focus on creating recoverable copies of data. Modern cyber-resilience strategies go further by ensuring those backups themselves cannot be altered or removed during a protected period. Immutable backups help ensure that recovery points remain available when you need them most - even in the face of malicious actions, accidental deletion, or compromised administrative credentials. What is changing? Starting with this release: Up to the most recent 7 days of Azure SQL Database and Azure SQL Managed Instance PITR backups are automatically protected by immutability Protection is enabled by default for all databases, with no additional cost No configuration or onboarding is required Protection applies regardless of the database's configured PITR retention setting Because this capability is built directly into the Azure SQL backup service, customers automatically benefit from stronger protection without changing existing backup, restore, or operational workflows. Designed for modern cyber resilience Organizations across industries increasingly require stronger safeguards around backup data as part of broader cyber-resilience programs. Automatic backup immutability helps customers: Improve protection against ransomware attacks Reduce the risk of accidental backup deletion Strengthen recovery readiness Increase confidence that recent recovery points remain available during an incident Simplify adoption of immutable backup practices without additional deployment effort This capability is particularly valuable because the backups most often used during recovery operations are typically the most recent ones. Supporting compliance and governance requirements Many industries must maintain records in a protected, tamper-resistant manner to satisfy regulatory and governance requirements. Azure Storage immutable storage capabilities have been validated for compliance scenarios involving requirements such as: SEC Rule 17a-4(f) CFTC Rule 1.31(d) FINRA record-retention requirements These regulations commonly require records to be retained in a nonerasable, non-rewritable format for a defined period of time. Azure immutable storage uses a Write Once, Read Many (WORM) model that helps organizations meet these requirements. Azure SQL database backups leverage this WORM capability from Azure storage to achieve immutability for the backups. While compliance requirements vary by organization and jurisdiction, automatic backup immutability provides an additional foundational control that can support broader security, governance, and resilience objectives. No additional complexity One of our goals with this release is to deliver stronger security without increasing operational burden. You don't need to: Create immutability policies Configure storage accounts Manage retention locks Extract data The protection is integrated directly into the Azure SQL managed backup service and works automatically for all Azure SQL Database and Azure SQL Managed Instance databases. Pricing and Availability Automatic backup immutability up to the most recent 7 days of point-in-time restore (PITR) backups is available for all Azure SQL Database and Azure SQL Managed Instance databases. There is no additional cost to use this capability. The protection is built into the Azure SQL managed backup service and is automatically applied to the most recent 7 days of backups. No configuration, policy management, or separate licensing is required. By enabling immutable protection by default and at no additional charge, Azure SQL helps customers strengthen their cyber-resilience posture, improve protection against ransomware and accidental deletion, and gain the benefits of immutable backups without added operational complexity. Building on Azure SQL's data protection foundation Automatic backup immutability is the latest enhancement in Azure SQL's ongoing investment in data protection, security, and business continuity. By combining automated backups, point-in-time restore capabilities, geo-redundant backup options, soft delete protection for your Azure SQL logical server and now automatic backup immutability for recent backups, Azure SQL continues to help organizations strengthen their resilience against both operational accidents and modern cyber threats. This is just the beginning. Azure SQL hyperscale backup immutability and a host of other additional capabilities are coming soon. FAQs Q: What is changing? A: Microsoft Azure SQL will start protecting the short-term retention backups for all Azure SQL DB and Azure SQL managed instance databases with immutability to protect against ransomware attacks. Q: Are all my backups protected? A: In this release, up to most recent 7 days of short-term retention backups are immutable, regardless of the configured retention period. For example, if the configured retention period is 7 or less, then all the backups are immutable. If the configured retention period is 35 days, then the most recent 7 days of backups are immutable. Q: Is there any additional cost for this feature? A: No. Immutability for the backups is being provided as a security feature natively. Q: When will this be available? A: The code to enable immutable policy is already in progress in all Azure regions worldwide. In the next few weeks all backups will be on immutable storage. Q: Do I need to do anything to enable/configure immutability? A: No. There is no action needed on your end. The backups will automatically be immutable once the enablement is complete. Q: How can I verify if my backups are immutable? A: In a future release, immutability status will be exposed as a database property. Q: How can I get immutability for my backups beyond 7 days? A: In this release backups up to most recent 7 days are immutable. Immutability for additional retention period will be added in a future release. Limitations Immutability for Azure SQL hyperscale is not included in this release but will be available soon. Learn more To learn more about immutable storage concepts and WORM (Write Once, Read Many) protection in Azure, see: https://learn.microsoft.com/azure/storage/blobs/immutable-storage-overview We are excited to bring this protection to every Azure SQL Database and Azure SQL Managed Instance customer automatically, helping you improve backup security and recovery readiness with no additional effort. Documentation updates More details at https://aka.ms/auto-immutability Looking ahead We are just getting started on this journey of ransomware protection. Additional flexibility and configuration options coming in future releases.Transparent data encryption in Azure SQL Database now supports AES keys (Public Preview)
For teams thinking about long-term cryptographic resilience, this preview is especially relevant. TDE with customer-managed keys has traditionally used asymmetric RSA-based key protectors, while broader industry guidance is increasingly focused on preparing for a post-quantum cryptographic (PQC) future and adopting cryptographic approaches that are better aligned with that transition. This update aligns with broader security guidance, including the NSA’s CNSA 2.0 recommendations, which emphasize modern cryptographic planning for a quantum-resistant future. For organizations building crypto agility into their platforms, AES support is a practical step in that direction. Why it matters Preparing for a post‑quantum world With current technology, breaking asymmetric algorithms such as Elliptic Curve and RSA-2048 using the best-known classical methods would take billions of years. Even with large-scale distributed computing, it is still considered computationally infeasible. Asymmetric algorithms are vulnerable to Shor’s algorithm, which means a sufficiently powerful quantum computer could break RSA-2048 much faster. That said, this would require millions of stable qubits, and current quantum systems are still far from that point. AES, as a symmetric algorithm, is not affected by Shor’s algorithm and remains more resistant to known quantum attacks, including Grover’s algorithm, when used with larger key sizes such as AES-256. The figure below highlights the difference in the estimated effort required to break RSA-2048 and AES-256. For context, the green dashed line represents the age of the universe, about 13.8 billion years. Aligning with modern security guidance Security guidance is moving toward stronger crypto agility and long-term resilience. By supporting AES keys for TDE protectors, Azure SQL Database gives customers a way to align data-at-rest protection with evolving security and compliance expectations. For a broader overview of quantum computing and cryptography, see Microsoft’s post-quantum cryptography overview. How it works (high level) At a high level, nothing changes about the purpose of TDE: it still protects data at rest by encrypting the Database Encryption Key (DEK) with a TDE protector. What changes in this preview is the type of key you can use to protect, or wrap, the AES DEK. The AES DEK encrypts database data files and log files. The TDE protector encrypts the DEK. With TDE with customer‑managed keys, the TDE protector is stored in Azure Key Vault or Azure Key Vault Managed HSM. With this preview, the TDE protector can now be a symmetric AES key instead of an RSA key. For background on customer-managed TDE, see the Customer-managed transparent data encryption (TDE) Get started If you want to try the preview, make sure the following prerequisites are in place: An Azure SQL Database logical server or database with customer-managed TDE enabled. An Azure Key Vault Premium (preview) or Azure Key Vault Managed HSM with support for AES keys. Soft-delete and purge protection enabled on the key store. The required permissions for Azure SQL Database to access the key. You can review the full prerequisites in Microsoft Learn under requirements to configure customer-managed TDE. The setup flow for AES keys is essentially the same as for RSA-based TDE protectors. The main difference is the type of key you create and register. Create an AES key (for example, AES‑256) in Azure Key Vault Managed HSM. Add the key to your Azure SQL logical server. Set the AES key as the TDE protector. Verify that encryption is enabled using system views. For step-by-step configuration guidance, see Microsoft Learn on Create Azure SQL Database Logical Server Configured with User-Assigned Managed Identity and Customer-Managed TDE. Example configuration (PowerShell) The following example shows the basic PowerShell flow: create an AES key, register it with the logical server, and then set it as the TDE protector. # Variables $hsmName = "MyHSM" $keyName = "TDE-AES-Key" $sqlServerName = "my-sql-server" $sqlResourceGroup = "my-sql-rg" # Create an AES-256 HSM-backed key in MHSM Add-AzKeyVaultKey ` -HsmName $hsmName ` -Name $keyName ` -KeyType oct-HSM ` -Size 256 # Get key URI $key = Get-AzKeyVaultKey -HsmName $hsmName -Name $keyName # Register the key with the SQL server Add-AzSqlServerKeyVaultKey ` -ResourceGroupName $sqlResourceGroup ` -ServerName $sqlServerName ` -KeyId $key.Id # Set the key as the TDE protector Set-AzSqlServerTransparentDataEncryptionProtector ` -ResourceGroupName $sqlResourceGroup ` -ServerName $sqlServerName ` -Type AzureKeyVault ` -KeyId $key.Id After you enable TDE with AES keys, you can verify the database encryption status by running the following query: SELECT DB_NAME(database_id) AS DatabaseName, encryption_state_desc, encryptor_type FROM sys.dm_database_encryption_keys WHERE database_id <> 2; If the database is encrypted, the view returns an ENCRYPTED state, or ENCRYPTION_IN_PROGRESS while encryption is still underway, with SYMMETRIC_KEY shown as the encryptor type. Public preview notice Transparent data encryption in Azure SQL Database with AES keys support is currently in Public Preview. Preview features are provided for evaluation purposes and are subject to the Azure Preview Supplemental Terms . Availability is rolling out gradually across Azure regions. You may see this capability appear over time depending on your region and service deployment status. Azure SQL Database is the first SQL offering to receive this feature, with additional SQL platforms planned in the future. Learn more Microsoft Learn: customer-managed transparent data encryption for Azure SQL Database Microsoft Learn: configure customer-managed TDE for Azure SQL Database Microsoft Research: post-quantum cryptography overview Conclusion AES key support for customer-managed TDE gives Azure SQL Database customers a practical way to strengthen their encryption strategy while preparing for long-term cryptographic change, including post quantum cryptography. Because the setup experience remains familiar, teams can evaluate this preview without rethinking how TDE works operationally. We want your feedback If you’re exploring this preview, now is a good time to test it in your environment and share feedback with the product group before general availability.Azure Data Studio Retirement
We’re announcing the upcoming retirement of Azure Data Studio (ADS) on February 6, 2025, as we focus on delivering a modern, streamlined SQL development experience. ADS will remain supported until February 28, 2026, giving developers ample time to transition. This decision aligns with our commitment to simplifying SQL development by consolidating efforts on Visual Studio Code (VS Code) with the MSSQL extension, a powerful and versatile tool designed for modern developers. Why Retire Azure Data Studio? Azure Data Studio has been an essential tool for SQL developers, but evolving developer needs and the rise of more versatile platforms like VS Code have made it the right time to transition. Here’s why: Focus on innovation VS Code, widely adopted across the developer community, provides a robust platform for delivering advanced features like cutting-edge schema management and improved query execution. Streamlined tools Consolidating SQL development on VS Code eliminates duplication, reduces engineering maintenance overhead, and accelerates feature delivery, ensuring developers have access to the latest innovations. Why Transition to Visual Studio Code? VS Code is the #1 developer tool, trusted by millions worldwide. It is a modern, versatile platform that meets the evolving demands of SQL and application developers. By transitioning, you gain access to cutting-edge tools, seamless workflows, and an expansive ecosystem designed to enhance productivity and innovation. We’re committed to meeting developers where they are, providing a modern SQL development experience within VS Code. Here’s how: Modern development environment VS Code is a lightweight, extensible, and community-supported code editor trusted by millions of developers. It provides: Regular updates. An active extension marketplace. A seamless cross-platform experience for Windows, macOS, and Linux. Comprehensive SQL features With the MSSQL extension in VS Code, you can: Execute queries faster with filtering, sorting, and export options for JSON, Excel, and CSV. Manage schemas visually with Table Designer, Object Explorer, and support for keys, indexes, and constraints. Connect to SQL Server, Azure SQL (all offerings), and SQL database in Fabric using an improved Connection Dialog. Streamline development with scripting, object modifications, and a unified SQL experience. Optimize performance with an enhanced Query Results Pane and execution plans. Integrate with DevOps and CI/CD pipelines using SQL Database Projects. Stay tuned for upcoming features—we’re continuously building new experiences based on feedback from the community. Make sure to follow the MSSQL repository on GitHub to stay updated and contribute to the project! Streamlined workflow VS Code supports cloud-native development, real-time collaboration, and thousands of extensions to enhance your workflows. Transitioning to Visual Studio Code: What You Need to Know We understand that transitioning tools can raise concerns, but moving from Azure Data Studio (ADS) to Visual Studio Code (VS Code) with the MSSQL extension is designed to be straightforward and hassle-free. Here’s why you can feel confident about this transition: No Loss of Functionality If you use ADS to connect to Azure SQL databases, SQL Server, or SQL database in Fabric, you’ll find that the MSSQL extension supports these scenarios seamlessly. Your database projects, queries, and scripts created in ADS are fully compatible with VS Code and can be opened without additional migration steps. Familiar features, enhanced experience VS Code provides advanced tools like improved query execution, modern schema management, and CI/CD integration. Additionally, alternative tools and extensions are available to replace ADS capabilities like SQL Server Agent and Schema Compare. Cross-Platform and extensible Like ADS, VS Code runs on Windows, macOS, and Linux, ensuring a consistent experience across operating systems. Its extensibility allows you to adapt it to your workflow with thousands of extensions. If you have further questions or need detailed guidance, visit the ADS Retirement page. The page includes step-by-step instructions, recommended alternatives, and additional resources. Continued Support With the Azure Data Studio retirement, we’re committed to supporting you during this transition: Documentation: Find detailed guides, tutorials, and FAQs on the ADS Retirement page. Community Support: Engage with the active Visual Studio Code community for tips and solutions. You can also explore forums like Stack Overflow. GitHub Issues: If you encounter any issues, submit a request or report bugs on the MSSQL extension’s GitHub repository. Microsoft Support: For critical issues, reach out to Microsoft Support directly through your account. Transitioning to VS Code opens the door to a more modern and versatile SQL development experience. We encourage you to explore the new possibilities and start your journey today! Conclusion Azure Data Studio has served the SQL community well,but the Azure Data Studio retirement marks an opportunity to embrace the modern capabilities of Visual Studio Code. Transitioning now ensures you’re equipped with cutting-edge tools and a future-ready platform to enhance your SQL development experience. For a detailed guide on ADS retirement , visit aka.ms/ads-retirement. To get started with the MSSQL extension, check out the official documentation. We’re excited to see what you build with VS Code!Understanding Azure SQL Long-Term Retention Immutability Configuration with Terraform and AzAPI
Executive Summary Organizations using Azure SQL Database Long-Term Retention (LTR) backup policies may encounter failures when attempting to disable backup immutability while also specifying an immutability mode in the same request. In the investigated scenario, the customer observed that the operation succeeded when performed through ARM templates but failed when executed through Terraform using AzAPI-based resources. The investigation determined that the Azure SQL resource provider enforces validation rules that prevent TimeBasedImmutabilityMode from being specified when TimeBasedImmutability is set to Disabled. The issue was not caused by the Azure SQL service itself, but rather by how configuration values were being submitted through Terraform and AzAPI resource updates. The recommended mitigation is to ensure that immutability mode is omitted or explicitly set to null when disabling time-based immutability. This allows the request to comply with the resource provider's validation requirements. Introduction Azure SQL Database supports immutable Long-Term Retention (LTR) backups to help organizations meet compliance, governance, and data protection requirements. These policies allow administrators to control whether retained backups can be modified or deleted. During an investigation involving Terraform and AzAPI deployments, a customer reported inconsistent behavior when attempting to disable backup immutability. While equivalent ARM template operations completed successfully, Terraform-based deployments generated validation errors. This article explains the observed behavior, the investigation findings, the confirmed root cause, and the recommended mitigation. Issue Description Reported Symptoms The customer reported the following behavior: Enabling and managing Long-Term Retention backup immutability worked successfully. Configurations involving immutability mode settings could be applied successfully under certain conditions. Attempts to disable immutability through Terraform resulted in failures. Similar operations appeared to succeed when performed using ARM templates. Technical Environment The discussion confirmed the following components: Azure SQL Database Long-Term Retention (LTR) backup policies Backup immutability configuration Terraform deployments AzAPI resources and AzAPI resource updates ARM template deployments Azure SQL Resource Provider validation logic Expected Behavior When administrators disable backup immutability, the configuration update should be accepted and the policy should transition to a disabled state. Actual Behavior Requests submitted through Terraform/AzAPI included both: TimeBasedImmutability = Disabled TimeBasedImmutabilityMode = Unlocked The Azure SQL resource provider rejected this configuration, returning an error indicating that an immutability policy mode cannot be specified when backup immutability is not enabled. Investigation and Troubleshooting 1. Initial Customer Question The customer sought clarification on whether enabling, disabling, locking, and unlocking backup immutability should all be possible through AzAPI resources and whether a product issue existed. 2. Review of Azure SQL Resource Provider Behavior The support team reviewed requests submitted to the Azure SQL resource provider and compared successful and unsuccessful operations. The investigation focused on configuration differences between ARM template deployments and Terraform-driven updates. Confirmed Finding When ARM templates disabled immutability, the request contained: TimeBasedImmutability = Disabled and did not include an immutability mode parameter. Confirmed Finding When Terraform attempted to disable immutability, the request included both: TimeBasedImmutability = Disabled TimeBasedImmutabilityMode = Unlocked This resulted in a validation failure from the Azure SQL resource provider. 3. Validation of Error Behavior The team verified that the error was generated by the Azure SQL resource provider and was reproducible outside Terraform, including equivalent testing through ARM deployments when the conflicting parameter combination was supplied. Confirmed Error The resource provider returned an error equivalent to: Cannot set immutability policy mode when backup immutability is not enabled. 4. Assessment of Terraform and AzAPI Behavior The investigation identified an important behavioral difference. Terraform itself did not yet expose dedicated Time-Based Immutability parameters in its SQL modules. As a result, the customer was using AzAPI resources to perform direct REST-based operations. The team discovered that: azapi_resource behaved as expected. azapi_resource_update could retrieve and reuse an existing property value when no value was explicitly provided. This behavior caused the immutability mode value to persist unexpectedly during updates. 5. Reproduction and Verification The engineering discussion included review and validation of the reported behavior. Testing confirmed that requests containing immutability mode while immutability was disabled were expected to fail due to platform validation. Root Cause Confirmed Root Cause The failure occurred because the update request attempted to disable backup immutability while simultaneously providing a value for TimeBasedImmutabilityMode. Azure SQL validation rules require immutability mode to be associated only with an enabled immutability configuration. When immutability is disabled, an immutability mode must not be supplied. An additional contributing factor was the behavior of azapi_resource_update, which could retain a previously configured immutability mode value when no new value was explicitly provided. Consequently, requests unintentionally included an immutability mode even though the intent was to disable immutability entirely. The available evidence supports this conclusion through: Comparison of successful ARM template requests and failing Terraform requests. Reproduction of the same validation behavior by the Azure SQL resource provider. Validation of the AzAPI update behavior involving retained values. Mitigation and Resolution Recommended Mitigation When disabling backup immutability: Set TimeBasedImmutability to Disabled. Do not provide TimeBasedImmutabilityMode. Terraform/AzAPI Workaround The investigation determined that explicitly setting: TimeBasedImmutabilityMode = null prevents the previous value from being reused and allows the request to be processed correctly. Configuration Matrix Discussed The support team identified the following expected behavior: Operation Immutability Mode Requirement Locking backups Mode should be set to Locked Unlocking while remaining enabled Mode may be supplied and is recommended for clarity Disabling immutability Mode should not be supplied This guidance was explicitly discussed during the investigation. Validation After applying the mitigation: The disable operation should complete without the immutability mode conflict. Requests should no longer trigger the Azure SQL validation error related to immutability mode usage. Recommendations and Best Practices Recommendations Supported by the Investigation Ensure that immutability mode is not included when disabling backup immutability. Review Terraform templates for dynamically generated properties that may continue to emit previously populated values. When using AzAPI update resources, explicitly manage nullable properties where supported to avoid unintended value persistence. Important Considerations Behavior may vary depending on: Azure SQL API version Terraform provider version AzAPI provider implementation details Existing Long-Term Retention backup state Whether previously locked backups exist Always validate deployment behavior in a non-production environment before applying configuration changes broadly. Conclusion This investigation demonstrated that the inability to disable Azure SQL Long-Term Retention backup immutability was not caused by a platform defect in Azure SQL. Instead, the failure occurred because requests attempted to specify an immutability mode while immutability itself was being disabled. The issue was further influenced by AzAPI update behavior that could preserve previously configured values unless explicitly cleared. Setting the immutability mode to null, or removing it entirely when disabling immutability, resolved the problem. The key technical takeaway is that TimeBasedImmutabilityMode and TimeBasedImmutability must be configured consistently with Azure SQL resource provider validation rules, particularly during infrastructure-as-code deployments. Public Documentation Azure SQL Database Long-Term Retention documentation Azure SQL Backup Immutability documentation ARM/Bicep resource documentation for backup Long-Term Retention policies Terraform provider documentation for Azure SQL DatabaseLessons Learned #549: Reproduce, Challenge, and Validate – Testing Technical Assumptions
During an Azure SQL investigation, the available evidence may point to a technically reasonable explanation. For example: A network change may appear to explain a connectivity issue. High CPU may appear to explain a performance degradation. A query plan change may appear to explain a longer execution time. A failover may appear to explain why the behavior disappeared. These explanations may be correct. However, before considering them confirmed, I normally try to reproduce the behavior and validate the conditions under which it occurs. A reasonable explanation is still a hypothesis until the available evidence supports it consistently. Reproduce the Smallest Useful Scenario A reproduction does not always need to recreate the complete production environment. In many cases, a smaller test provides a clearer result. For a connectivity issue, the test may require only: one client; one database endpoint; the same authentication method; the relevant network path; a clearly identified timestamp. For a performance issue, it may require: one representative query; the same parameters; the relevant database configuration; a controlled execution period. The objective is to isolate the behavior being investigated while reducing unnecessary variables. The smaller the scenario, the easier it normally becomes to understand why the result changes. Define the Expected Result Before running a test, I try to define what result I expect. For example: Hypothesis: The connection behavior depends on a specific network path. Expected Result: The behavior should occur when the connection uses that path and should not occur when an alternative path is used. Result That Would Challenge the Hypothesis: The same behavior occurs independently of the network path. The same approach can be used for performance investigations. Hypothesis: The query slowdown is caused by Data IO saturation. Expected Result: Query duration should increase when Data IO reaches its service-level limit. Result That Would Challenge the Hypothesis: The same slowdown occurs while Data IO remains low. Defining the expected result before the test helps avoid interpreting every outcome as confirmation of the initial explanation. Change One Variable at a Time When several conditions are changed simultaneously, it may be difficult to determine which one affected the result. For example, suppose a performance test includes: a larger service objective; updated statistics; a new index; lower concurrency. If performance improves, the result is positive, but it may not clearly identify which change produced the improvement. A more useful approach is to test each relevant change separately. Test Change Test 1 Original configuration Test 2 Updated statistics only Test 3 New index only Test 4 Higher service objective only This does not mean that every investigation requires an extensive test matrix. The objective is simply to avoid changing several important variables at the same time when we need to understand which one explains the behavior. Compare Where the Behavior Occurs and Where It Does Not Understanding where a behavior does not occur can be as useful as reproducing where it does. For example: Does the connection fail from one application server but succeed from another? Does the query perform normally with different parameters? Does the issue occur only through one network path? Does the previous execution plan perform better? Does the behavior disappear when concurrency is reduced? These comparisons help define the boundaries of the problem. They may not immediately identify the complete root cause, but they help determine which conditions are relevant and which ones are less likely to explain the result. Challenge the Explanation Once a test appears to support a hypothesis, I normally try to challenge it. For example, if performance improves after scaling the database, it may be tempting to conclude that CPU was insufficient. However, scaling may also provide: more memory; higher Data IO capacity; greater transaction-log throughput; additional workers; different resource limits. The improvement is important evidence, but additional information may still be required to identify which resource was actually limiting the workload. Similarly, if a failover restores normal performance, it may also have: disconnected blocking sessions; refreshed application connections; caused query recompilation; reset a temporary condition. The action that restores normal operation may not, by itself, fully explain the original cause. Document the Result and Its Limitations Not every test produces a definitive answer. A useful conclusion should describe: which conditions were tested; what result was expected; what result was observed; whether the result was repeatable; which differences from the original scenario remained; what the evidence did and did not allow us to confirm. For example: The behavior was reproduced only when the connection used the affected network path. The same authentication method and database endpoint worked successfully through an alternative path. Based on these tests, the network path was confirmed as a relevant condition. The available evidence did not identify the specific network component responsible for the behavior. This conclusion is useful because it clearly separates what was validated from what remains unknown. Conclusion Reproduction is one of the most valuable troubleshooting tools available to an engineer. However, its purpose is not simply to make an error occur again. A useful reproduction should help us understand: which conditions are required; which conditions are not relevant; what result supports the hypothesis; what result challenges it; how consistently the behavior can be observed. The most effective tests are often simple: reproduce the smallest useful scenario; define the expected result; change one variable at a time; compare affected and unaffected conditions; document the limitations. Reproduction does not only confirm that a behavior exists. It helps define the conditions under which the technical conclusion can be trusted.Lesson Learned #548: From Symptoms to Evidence – How I Approach an Azure SQL Investigation
After working on many Azure SQL support cases, I have learned that the initial service request details are essential, although they may not always provide sufficient information to understand the complete technical situation. When a service request is initially created, the available information may be limited to a brief description such as: The database is running slowly. CPU utilization is high. Connections are failing. The issue started after a deployment. Performance improved after a failover. The workload became slower after a service-tier migration. These details provide an important starting point for the investigation. However, the initial description may not yet include all the relevant timestamps, metrics, logs, configuration details, or historical context required for a complete technical assessment. This is completely understandable. At the time the service request is created, the immediate priority is normally to describe the observed behavior and its impact. Additional technical context can then be collected progressively during the investigation. For that reason, I normally use the initial service request details to understand the reported symptom and determine which additional information may be required. The first objective is to clarify the observed behavior, define the affected scope, and identify the evidence that may help us evaluate the different possible explanations. In this article, I would like to share the approach I normally follow when moving from an initial service request description to an evidence-based technical conclusion. Clarifying the Observed Behavior The first thing I normally try to understand is the exact impact. For example: Was the application completely unavailable, or was it slower than usual? Did the behavior affect all users or only a specific group? Did it affect all queries or only one process? Was the issue continuous or intermittent? Did it affect one database, several databases, or the complete logical server? Did the problem occur only from a particular network location? Did existing connections continue working while new connections failed? These questions may appear simple, but the answers can significantly change the direction of the investigation. For example, the initial service request details may indicate: Connections to Azure SQL Database are failing. After reviewing the scenario, we may find that: other applications can connect successfully; only one application instance is affected; existing connections continue working; only new connections are failing; the issue occurs from one specific network path. With this additional information, the situation may no longer appear to be a general Azure SQL connectivity issue. The investigation may instead need to focus on areas such as: application connection pooling; authentication; DNS resolution; token renewal; network routing; firewall rules; a specific application instance; client-side resource pressure. Similarly, a service request may indicate that the database is slow. Before looking for the cause, I normally try to understand what “slow” means in that specific situation. For example: Are all queries slower? Is only one stored procedure affected? Has execution time increased from seconds to minutes? Is the delay occurring while opening the connection? Is the delay occurring while executing the command? Is the application waiting on Azure SQL Database or on another dependency? Before reviewing metrics, logs, or execution plans, it is important to convert the initial description into a precise and measurable technical symptom. Distinguishing Observations from Possible Explanations One of the most useful habits I have developed is to separate what has been observed from what still needs to be validated. Consider the following statement: The application became slow when database CPU reached 95%. From this description, we may have two observations: the application experienced a slowdown; database CPU reached 95%. However, the relationship between those two observations still needs to be validated. High CPU may have caused the slowdown, but it could also be part of a larger chain of events. For example, CPU utilization may have increased because: the application started sending more requests; blocking caused requests to accumulate; a query execution plan changed; application retries generated additional workload; a scheduled process started running; concurrency increased; a query began processing more data; data distribution changed; statistics changed; a maintenance task started. In other words, high CPU may be the cause of the performance issue, but it may also be the result of another condition. I have seen similar situations after failovers, application restarts, and scaling operations. For example, the available details may indicate: Performance returned to normal after a failover. The failover is an important part of the investigation, but it may not, by itself, fully explain the original cause. A failover may also: disconnect blocking sessions; cause queries to compile again; refresh application connections; reset a temporary condition; clear some cached state; coincide with a reduction in workload. For this reason, I normally try to distinguish between: what was initially reported; what has been confirmed by telemetry; what is currently considered a possible explanation; what has already been validated. This distinction helps prevent an early assumption from becoming the final conclusion before sufficient evidence is available. Building a Timeline When timestamps and historical information are available, I normally try to build a timeline. A timeline is often one of the most useful parts of an investigation because several events may initially appear related until they are placed in the correct order. Consider the following example: Time Event 10:00 UTC Application deployment completed 10:05 UTC Active sessions started increasing 10:10 UTC Query duration increased 10:12 UTC Data IO reached 100% 10:15 UTC CPU reached 95% 10:20 UTC Application timeouts were reported 10:30 UTC Application service was restarted 10:35 UTC Session count returned to normal If the initial service request description mentions only the CPU peak and the application timeouts, CPU may appear to be the most likely starting point. However, the timeline shows that the number of active sessions and Data IO utilization increased before CPU reached 95%. This does not immediately confirm the root cause, but it changes the questions that should be asked. For example: Why did the number of sessions increase? Did requests start taking longer because of IO pressure? Did application retries contribute to the workload increase? Was the CPU peak the initial cause, or was it a consequence of the accumulating workload? Did the application deployment change request volume or execution patterns? Depending on the situation, the timeline may include information from: Azure Monitor metrics; Query Store; application logs; deployment history; audit events; scaling operations; failovers; configuration changes; maintenance processes; network changes. The objective is not to collect every piece of information available. The objective is to identify the sequence of events that may explain what happened before, during, and after the reported incident. Using Each Data Source for the Right Question Another lesson I have learned is that no single source of telemetry normally explains the complete situation. Different data sources answer different questions. Azure Monitor Azure Monitor can help identify whether the database reached limits related to: CPU; Data IO; log write; sessions; workers; storage; connection failures; deadlocks. These metrics are very useful for identifying when resource pressure occurred. However, a database-level metric may not directly identify the query, application, or operation responsible for that resource usage. Query Store Query Store may help identify: changes in query duration; increased CPU consumption; changes in execution count; increased logical reads; execution-plan changes; query regressions. It can be especially useful when a database-level metric needs to be correlated with specific query activity. Dynamic Management Views Dynamic Management Views may provide information about: active requests; waits; blocking; sessions; open transactions; memory grants; current resource consumption. This information is especially valuable while the issue is occurring. However, some of the data may no longer be available after the event has ended. Application Logs Application logs may help identify: connection timeouts; command duration; retry behavior; connection-pool exhaustion; authentication errors; dependency failures; changes in request volume. These logs often provide context that may not be visible from the database side. Deployment and Configuration History Deployment and configuration history may help explain why the behavior began at a particular time. This may include: application releases; schema changes; index operations; compatibility-level changes; connection-string updates; service-tier changes; network changes; security configuration changes. The important point is to choose the evidence according to the question being investigated. For example, if I am investigating whether a workload is limited by Data IO rather than CPU, reviewing only CPU percentage may not provide sufficient information. I would also want to understand: when Data IO reached its limit; whether query duration increased during the same period; which queries generated the highest number of reads; whether the workload started reading more data; whether the same degradation occurred when Data IO remained below its limit; whether additional IO capacity changed the result. The investigation becomes more effective when every piece of telemetry is connected to a specific technical question. Collecting more data does not automatically produce a better conclusion. Keeping More Than One Possible Explanation Open It is easy to identify one technically reasonable explanation and begin searching only for evidence that supports it. I normally try to avoid doing this, especially during the first stages of the investigation. For example, imagine that a database is migrated from a DTU-based service tier to a vCore-based service tier and the workload subsequently performs more slowly. One possible explanation is that the new environment does not provide sufficient CPU. However, other possibilities may include: lower Data IO capacity; lower transaction-log throughput; different memory availability; an execution-plan change; statistics changes; a compatibility-level difference; increased concurrency; a change in request volume; a change in data size or distribution; an application change unrelated to the migration. Each possible explanation requires different evidence. If CPU remains moderate while Data IO repeatedly reaches its limit, increasing the number of vCores without reviewing the storage characteristics may not address the main constraint. If Query Store shows a plan regression immediately after the migration, the selected service tier may not be the primary cause. If execution count doubled after an application deployment, the database may simply be processing more work than before. Keeping several possible explanations open does not make the investigation less decisive. It reduces the risk of reaching a conclusion before the relevant evidence has been reviewed. Defining What Would Challenge the Hypothesis One of the most useful questions I ask during troubleshooting is: What result would demonstrate that my current hypothesis may not be correct? Suppose the current hypothesis is: The performance degradation is caused by Data IO saturation. Evidence supporting this hypothesis may include: Data IO reaches 100% during the affected periods; query duration increases at the same time; the affected queries perform a high number of physical reads; CPU remains below its limit; performance improves when the workload runs with additional IO capacity. However, I should also look for results that may challenge the hypothesis. For example: Does the same slowdown occur while Data IO remains low? Are there periods with high Data IO but normal application performance? Are queries with very few reads also affected? Does additional IO capacity consistently improve the workload? Does changing another variable produce a larger improvement? Does the issue occur in an environment where the same IO pressure is not present? If an investigation only searches for supporting evidence, almost any initial theory may appear correct. Actively looking for evidence that challenges the current explanation makes the final conclusion stronger and more reliable. Conclusion After working on many support investigations, one of the most important lessons I have learned is that the initial service request details and the final technical conclusion naturally serve different purposes. The initial description provides the first available information about the observed behavior and its impact. At that stage, some of the relevant logs, metrics, timestamps, configuration details, or historical context may not yet be available. A reliable investigation begins with the initial service request details and develops progressively as additional evidence and technical context become available.SQL Data Sync Retirement: Migration Insights and Modern Alternatives
What started as a routine customer discussion quickly evolved into a strategic modernization conversation. A service that had quietly synchronized business-critical data for years was approaching retirement, prompting an important question: What should organizations do next? Every service retirement is an opportunity to reassess architecture, reduce technical debt, and build for the future. When a Retirement Notification Becomes a Business Conversation Recently, while working with a customer, we reviewed their Azure SQL Database architecture and discovered a critical dependency on SQL Data Sync For years, the service had reliably synchronized data across multiple databases, enabling applications, reporting workloads, and distributed business processes. Like many organizations, the customer viewed Data Sync as infrastructure that simply worked in the background. However, the discussion took a different turn when we reviewed Microsoft's retirement announcement: Azure SQL Data Sync will be retired on September 30, 2027. What initially appeared to be a migration challenge quickly became an opportunity to modernize the customer's data movement architecture and align with Microsoft's future investments in data integration, analytics, and cloud-native services. Understanding SQL Data Sync Azure SQL Data Sync was designed to synchronize selected data between Azure SQL Databases and, in some cases, between Azure and on-premises databases. Organizations have commonly used Data Sync for: Hybrid data synchronization Distributed application architectures Globally distributed applications Bi-directional data synchronization While the service has served customers well, organizations should begin evaluating alternative solutions now to ensure sufficient planning, testing, and adoption time before retirement. Because both databases were already in Azure, the discussion quickly moved toward identifying strategic alternatives. Customer's Setup This particular customer had a simple, familiar layout: one Azure SQL Database feeding another, both fully in Azure, connected by SQL Data Sync. No on-premises leg, no complicated topology — just two cloud databases that needed to stay aligned. Their requirements were equally straightforward, and honestly, the kind every team asks for: Reliable, dependable synchronization Low operational overhead — nobody wanted a new system to babysit Something with a real future, not another service on a retirement countdown Room to scale as data volumes grow An Azure-native fit, not a bolt-on third-party tool Because both databases already lived in Azure, the conversation moved quickly toward the platform's own native tooling — starting with the option that ended up being the strongest fit. Option 1: Azure Data Factory (Recommended Strategy) For customers running Azure SQL Database to Azure SQL Database synchronization, Azure Data Factory (ADF) emerged as the strongest strategic recommendation. Why ADF? Azure Data Factory provides: Fully managed Azure-native data movement Enterprise-grade monitoring Flexible orchestration Scalability from development through production environments Long-term Microsoft investment and support The migration pattern we typically recommend looks like: Phase1 :Initial Full Load Before anything can stay in sync, both sides need to start from the same place. Phase 1 is a one-time bulk copy: Azure Data Factory reads everything from the source database and writes it into the target, establishing a clean baseline. Phase2 :Incremental Synchronization Once both databases match, you don't need to keep copying everything — just what's changed. This is where Change Tracking (CT) or Change Data Capture (CDC) comes in: SQL Server-native features that flag which rows were inserted, updated, or deleted since the last run. ADF's incremental pipeline reads only those deltas and applies them downstream, on whatever schedule the business needs — minutes, hours, or daily. Phase 3: Scheduling and Monitoring Once both phases are live, ADF takes over the operational side: scheduling pipeline runs, monitoring their health, retrying failures automatically, and alerting your team when something needs attention. That's a level of visibility SQL Data Sync's built-in sync groups never really offered ADF handles: Scheduling Pipeline execution Monitoring Retry mechanisms Alerting This model often delivers greater visibility and operational control than traditional SQL Data Sync implementations. When You Don't Need Synchronization — You Need a Copy Partway through the engagement, one question reframed the whole discussion: do we actually need two-way synchronization, or do we just need a readable copy of the database somewhere else? That distinction matters more than it sounds, and it points to three other options worth knowing. Option 1: Active Geo-Replication If the goal is disaster recovery, serving read traffic closer to users, or keeping the business running through a regional outage, Active Geo-Replication is usually a better fit than rebuilding sync logic from scratch. It gives you a continuously updated, readable secondary — not a bi-directional sync target. Best fit Disaster recovery scenarios Read-intensive applications Global user distribution Secondary readable databases Less ideal for Complex data transformations Bi-directional updates Option 2: Database Copies and Read Replicas Some organizations do not require continuous synchronization at all. In those cases: Read Replicas Database Copy Good Use Cases Reporting databases Analytics environments Refreshable staging systems Read-only workloads This approach significantly reduces architectural complexity while still meeting many business requirements. Option 3: Microsoft Fabric Mirrored Databases As Microsoft Fabric adoption grows, another interesting alternative is Fabric Mirrored Databases. This option is particularly attractive for organizations already investing in: Microsoft Fabric OneLake Real-time analytics AI and data platform modernization Benefits include: Near real-time data availability Simplified analytics architecture Integration with Fabric workloads Reduced data silos For customers modernizing both operational and analytical platforms, this can be an excellent opportunity to rethink data architecture beyond simple synchronization. Option 4: Azure Functions for Event-Driven Synchronization Not every customer requires a large orchestration platform. For lightweight or application-specific synchronization logic, Azure Functions may offer a more agile approach. Example Use Cases Event-driven updates Custom business rules Microservices architectures Low-volume synchronization requirements The tradeoff is that customers assume additional development and operational responsibilities Lessons Learned from the Customer Engagement This engagement reinforced several important lessons: Don't Wait Until 2027 Although retirement is over a year away, large organizations often require significant planning, testing, governance approvals, and deployment cycles. Starting early reduces risk. There Is No Universal Replacement The right solution depends on: Latency requirements Read versus write workloads Synchronization direction Operational complexity DR requirements Budget limitation Different use cases require different migration paths. 3. Migration Is an Opportunity Rather than simply replacing SQL Data Sync, organizations should evaluate: Data architecture modernization Observability improvements Operational simplification Fabric adoption opportunities Long-term cloud strategy Final Recommendations For most customers currently using Azure SQL Database → Azure SQL Database synchronization: Requirement Recommended Solution Ongoing synchronization Azure Data Factory + CDC/Change Tracking Read-only replica Active Geo-Replication Simple duplication Database Copy or Read Replica Analytics modernization Fabric Mirrored Databases Event-driven custom logic Azure Functions In my customer's use case, Azure Data Factory with incremental changes (CDC) emerged as the preferred strategic path because it was Azure-native, scalable, supported long-term, and aligned with Microsoft's future direction for data movement and integration. Closing Thoughts Technology retirements often create urgency, but they also create opportunity. retirement of SQL Data Sync is not merely a migration project. It is an opportunity to reassess data movement architecture, improve resiliency, reduce technical debt, and embrace modern Azure-native services. If your organization is currently using SQL Data Sync, now is the right time to inventory your sync groups, identify dependencies, and begin evaluating alternative architectures before September 30, 2027. References SQL Data Sync Retirement Migration Guide What is SQL Data Sync for Azure SQL Database? SQL Data Sync retirement: Migrate to alternative solutions
Regex-based dynamic data masking in Azure SQL Database (preview)
Azure SQL Database introduces Regex-based dynamic data masking, a new capability that enables flexible, pattern-driven masking for string-based columns using regular expressions through the T‑SQL REGEXP_REPLACE() function. This feature extends Dynamic Data Masking beyond built-in masking functions, giving you precise control over which portions of sensitive data are masked and which remain visible, helping preserve data utility while meeting business and compliance requirements. This capability is especially useful when working with structured data patterns—such as emails, phone numbers, or identifiers—where teams need to preserve specific visible segments while masking sensitive parts, and where built-in masks may be rigid for some business workflows. Why this matters Dynamic Data Masking helps reduce accidental exposure of sensitive data by obfuscating values in query results for nonprivileged users, while keeping the original data intact in the database. However, existing built-in masking functions—such as default(), email(), random(), partial(), and datetime()—apply fixed patterns. These patterns cannot be customized, and may not be suitable for some of the real-world scenarios. For example, the built-in email() mask always transforms an address like alice.johnson@example.com into aXXX@XXXX.com, fully obscuring the domain name. In many operational scenarios, retaining the domain name is important for troubleshooting, routing, or business logic. Regex-based dynamic data masking addresses this gap by enabling precision masking without sacrificing usability. What’s new with regex-based masking Regex-based DDM allows you to define custom, pattern-driven masking rules using regular expressions. By leveraging the native REGEXP_REPLACE function in Azure SQL Database, you can precisely specify which parts of a string to mask and which to preserve—centrally enforced in the database layer. This approach supports variable-length and structured string data, enabling more precise and flexible data masking rules. Practical scenarios Regex-based masking enables common customer scenarios that are difficult to address with fixed masks: Mask email usernames while preserving domains alice.johnson@example.com → ****@example.com Preserve country codes in phone numbers +1-4155552671 → +1-XXXXXXXXXX +44-7911123456 → +44-XXXXXXXXXX Mask structured identifiers consistently Hide sensitive portions of national IDs or custom identifiers while keeping recognizable structure for support and auditing workflows. For example: AB-1234-5678 → AB-****-5678 Example The following example creates a table CustomerDetails with regex-based masking applied on Phone_Number and Email columns. The phone number mask preserves the country code and replaces the remaining digits with xxxx, and the email mask conceals the username while retaining the domain name. -- Drop the CustomerDetails table if it exists DROP TABLE IF EXISTS Data.CustomerDetails; -- Create a CustomerDetails table under a schema CREATE TABLE Data.CustomerDetails ( ID INT IDENTITY(1,1) PRIMARY KEY, Name varchar(30), Phone_Number varchar(30) MASKED WITH (FUNCTION = 'REGEXP_REPLACE("(\+\d{1,3})(?:[ -.]?\d){7,14}","(\1)-xxxx")'), Email varchar(255) MASKED WITH (FUNCTION = 'REGEXP_REPLACE("([a-zA-Z0-9._%+-]+)(@+)([a-zA-Z0-9.-]+)(\.)(\w)","*****\2\3\4\5")') ); -- Insert some dummy records to CustomerDetails table INSERT INTO Data.CustomerDetails (Name, Phone_Number, Email) VALUES ('Alice Johnson', '+1 202-555-0123', 'alice.johnson@example.com'), ('Bob Smith', '+1 415-555-0198', 'bob.smith@contoso.com'); -- Create a test user CREATE USER SupportEngineer WITHOUT LOGIN; -- Grant read permission on CustomerDetails to SupportEngineer GRANT SELECT ON Data.CustomerDetails TO SupportEngineer; -- Query CustomerDetails table as SupportEngineer EXECUTE AS USER = 'SupportEngineer'; SELECT * FROM Data.CustomerDetails; REVERT; Public Preview notice Important Regex-based dynamic data masking is currently in Preview for Azure SQL Database. Preview features are provided for evaluation purposes and are subject to the Preview Terms Of Use | Microsoft Azure. Azure SQL Database is the first SQL offering to receive this feature, with additional SQL platforms planned in the future. How to get started To learn more and access sample scripts, refer to the official documentation Regex-based dynamic data masking (preview) - Azure SQL Database | Microsoft Learn Try Regex-based dynamic data masking in your dev or test environment and tell us what works—and what doesn’t! Tell us what you need next in Data Masking Share your ideas through Azure SQL feedback forum. Disclaimer: The examples in this article use fictional customer records created solely for demonstration and testing purposes. No real customer data is included.CDK Global modernizes automotive CRM on Azure SQL Managed Instance
For automotive retailers, most customer relationships don't begin and end with a vehicle purchase. A customer might browse inventory online, visit a dealership weeks later, return for service months after that, and eventually purchase another vehicle years down the road. Every interaction creates information that helps dealerships better understand their customers and build stronger relationships over time. Helping dealerships manage those relationships is at the core of the CDK CRM platform. CDK is a leading provider of cloud-based software to dealerships and OEMs across automotive and related industries in the US and Canada, helping facilitate more than $540 billion in annual automotive commerce. It gives dealership teams visibility into customer interactions with their products and services and creates continuity across the conversations that shape the buying journey. As customer expectations evolve, CDK continues to look for new ways to help dealerships work more efficiently, make better use of information, and adapt quickly to market changes. Supporting that next phase of innovation required a technology foundation that could grow alongside the business. Working with Microsoft, CDK evolved its CRM platform on Microsoft Azure and Azure SQL Managed Instance. The project included a large migration to Azure SQL Managed Instance and marked an important step in the future of the CRM experience. Transforming a business-critical system at scale The size of the project reflected how central the CRM platform is to CDK’s business. The environment supports a broad set of applications for dealership operations and relies on a robust data foundation to keep information flowing. Microsoft and CDK worked together to implement a cloud architecture built on Azure App Service and Azure SQL Managed Instance. Additional Azure services support application delivery, networking, and data movement across the architecture. The teams developed a Terraform-based infrastructure-as-code framework that standardizes how environments are deployed and managed. That foundation brings greater consistency to the development process. Engineering teams can work within environments that are configured in a predictable way, reducing the likelihood of unexpected differences between testing and production. The project also provided an opportunity to strengthen security and governance practices. CDK updated applications to use managed identities, reducing reliance on static credentials, and implemented Microsoft Entra ID authentication and private connectivity patterns that help secure access to critical resources. While dealership users may not see these changes directly, they help deliver the stability and security that customers expect from the platform. Preserving continuity while moving to the cloud As CDK evaluated its cloud strategy, the database layer became one of the most important decisions in the project. The CRM platform supports business-critical dealership operations throughout the day. Customer interactions, sales activity, and service records all depend on information moving quickly and reliably between systems. Any technology transformation would need to preserve that experience while creating a path to future growth. Azure SQL Managed Instance stood out because it offered a familiar SQL Server environment while reducing much of the operational overhead associated with managing database infrastructure. It also aligned well with CDK’s long-term goals around resiliency, scalability, and continuous innovation. Equally important, Azure SQL Managed Instance provided a migration path that worked with the realities of the CRM environment. “One of the reasons Azure SQL Managed Instance appealed to us was that it allowed us to innovate without redesigning the CRM platform from the ground up,” says Stan Leong, Vice President of Modern Retailing Engineering at CDK. “We could preserve compatibility with the applications our dealerships depend on while taking advantage of a fully managed cloud service.” Unlike some projects that can move applications gradually, the CDK CRM platform required a coordinated transition. The databases that support the platform are highly interconnected, which meant the company needed an approach that would allow the environment to move together while minimizing disruption for customers. To prepare for that transition, CDK and Microsoft used Azure SQL Managed Instance link to establish near real-time replication between environments. This allowed teams to begin validating the migration long before the production cutover. Engineers could confirm that data was flowing correctly, identify potential issues, and gain confidence in the process before dealerships were ever affected. The approach also gave CDK an added layer of flexibility during the transition period. Rather than making a one-way move, the company could maintain a rollback option while teams validated production operations in Azure. Because Azure SQL Managed Instance link kept environments synchronized, CDK retained the ability to fail back to its on-premises environment if needed while preserving continuity for dealership operations. After failing over to Azure, the company continued running with synchronized environments for more than four weeks, allowing teams to validate production workloads before completing the final cutover. Executing a migration measured in terabytes That preparation became especially important because the migration would take place during a single maintenance window. By establishing synchronization ahead of time, CDK was able to keep data aligned between environments before the failover to Azure. When migration time arrived, production data was already synchronized in Azure, allowing the maintenance event to focus on transitioning operations rather than moving large volumes of data for the first time. Planning and architecture work that led to the migration spanned several months. The final migration preparation and validation effort was completed in just six weeks, with engineers working together to test migration scenarios, optimize replication performance, and validate data consistency ahead of production. CDK migrated more than 1,000 databases and hundreds of terabytes of data to Azure SQL Managed Instance. The environment now processes billions of database queries every day. The failover to Azure was completed during a single weekend with minimal failover time and no data loss. In many cases, database failovers completed in seconds, with most finishing within minutes. Microsoft engineering, product, support, and field teams remained engaged throughout the event, working alongside CDK to monitor the transition and address issues in real time. “This was one of the most significant technology initiatives we’ve undertaken for our CRM platform,” says Leong. “Working closely with Microsoft, we migrated more than 1,000 databases to Azure SQL Managed Instance while supporting dealerships throughout the process. The collaboration between our teams helped us execute the transition with minimal disruption to customers.” Strengthening reliability for dealership operations Delivering more than a new cloud environment, the migration also gave CDK an opportunity to evolve the CRM platform operations. As part of this effort, CDK implemented resilience architecture built on Azure. The design incorporates Azure Front Door, geo-redundant storage, and Azure SQL Managed Instance failover capabilities, allowing the company to maintain continuity when unexpected disruptions occur. These improvements rarely attract attention when everything is working as expected, but they help the platform remain available when it matters most. Since the migration, CDK has heard positive feedback from dealerships that report faster and more responsive application experiences. Preparing dealerships for what’s next With the CRM platform now running on Azure, CDK is focused on the next phase of its CRM strategy. Beyond supporting today’s dealership operations, the cloud-based foundation gives the company greater flexibility to introduce new functionality and continue advancing the platform over time. AI capabilities are already helping dealership employees surface relevant information at the right moment while reducing the effort required to complete routine tasks. And AI is accelerating software development and deployment, enabling engineering teams to deliver new capabilities more efficiently. CDK is also investing in new reporting experiences designed to make insights easier to access. Dealerships generate enormous amounts of data, but its value depends on how quickly users can find relevant information and act on it. The CRM migration has become an important reference point for future platform initiatives across the organization. By successfully moving a business-critical platform at this scale, CDK established a blueprint for future cloud initiatives. “The migration was an important milestone, but it’s really the starting point,” says Leong. “With Azure and Azure SQL Managed Instance, our teams can focus more energy on delivering new capabilities for dealerships while using AI to power experiences that help customers operate more efficiently.”
