azure sql
643 TopicsStrengthening Ransomware Resilience with Immutable Backups in Azure SQL DB and Azure SQL MI
Ransomware attacks are becoming increasingly sophisticated. Attackers no longer focus only on encrypting or stealing production data. They also target backups, knowing that eliminating an organization's recovery options can turn a security incident into a prolonged business disruption. Imagine discovering that your database has been compromised, only to find that the backups you were counting on have also been deleted. Even with a comprehensive backup strategy, recovery becomes significantly more challenging when attackers can tamper with the very data needed to restore business operations. That's why backup immutability matters. We're continuing to strengthen data protection in Azure SQL Database and Azure SQL Managed Instance with a consistent set of backup immutability capabilities spanning both point-in-time restore (PITR) and long-term retention (LTR) backups. Whether you're protecting recent transactions, maintaining historical backups for years, or preserving data for regulatory requirements, Azure SQL provides multiple layers of protection to help ensure your recovery points remain available when you need them most. What makes immutable backups different? Traditional backup retention policies define how long backups should be kept. But retention alone doesn't necessarily prevent someone with sufficient privileges from deleting backups before they expire. Backup immutability adds another layer of defense. With immutability, protected backups cannot be modified, overwritten, or deleted during their immutability period, even by administrators with elevated privileges. This is particularly valuable when attackers gain access to privileged accounts. While compromised credentials may allow attackers to disrupt production environments, immutable backups help prevent them from destroying protected recovery points. Think of it as adding a safety barrier around your backups, one that helps preserve your ability to recover even when other defenses have been compromised. 1. Built-in protection for your recent backups, automatically One of the simplest ways to improve security is to make protection automatic. Azure SQL Database and Azure SQL Managed Instance now provide immutable protection for up to the most recent seven days of PITR backups by default. The best part? There's nothing you need to do to get started. No additional configuration. No separate storage accounts to manage. No immutability policies to create. No changes to existing backup and restore workflows. No additional charge for the default immutability protection. This capability is integrated directly into the Azure SQL managed backup service. For customers, that means an additional layer of protection is already working behind the scenes, helping safeguard recent recovery points against accidental deletion, malicious activity, and compromised administrative credentials. Whether you're running a business-critical application or a smaller operational database, this built-in protection provides a stronger starting point for your recovery strategy. 2. Need longer protection? Lock immutability to your PITR retention period Seven days of automatic protection is a great starting point, but every organization has different recovery requirements. Some businesses need additional time to detect security incidents, investigate suspicious activity, and determine when their data was last known to be healthy. For these scenarios, Azure SQL offers the option to extend immutable protection across the entire configured PITR backup retention period, up to 35 days where supported. For example, if your database is configured to retain 35 days of PITR backups, you can lock immutability to that retention period, protecting those recovery points against deletion or modification. Once enabled, the lock is permanent. The configured retention period can be increased, but it cannot be shortened. This provides an important safeguard against attempts to weaken backup protection, including those made using compromised administrative credentials. With this capability, organizations can align their backup protection with their recovery requirements, rather than relying solely on the default protection window. The experience is consistent across Azure SQL Database and Azure SQL Managed Instance, making it easier to establish a common data protection strategy across your Azure SQL environment. 3. Protect your long-term backups for years, not just days Recovering from a recent incident is only one part of a comprehensive data protection strategy. Many organizations also need to preserve historical backups for business continuity, regulatory compliance, internal governance, or audit requirements. Azure SQL long-term retention (LTR) supports retaining backups for up to 10 years. With LTR backup immutability, customers can go beyond simply retaining those backups and protect them against unauthorized modification or deletion. Time-based immutability allows organizations to specify how long backups must remain protected. Once immutability is enabled and locked, affected backups cannot be deleted until their retention period expires, even if an administrator later changes or removes the retention policy. Consider a financial services organization that needs to preserve database backups for seven years. By configuring locked time-based immutability, the organization can help ensure those backups remain protected throughout the required retention period. The same approach is valuable for healthcare organizations, enterprises with strict governance requirements, and businesses seeking additional safeguards against ransomware. With a consistent LTR immutability experience across Azure SQL Database and Azure SQL Managed Instance, customers can apply the same protection principles regardless of which service hosts their databases. 4. Preserve critical backups with legal holds Not every data preservation requirement has a predictable end date. An organization may need to retain a specific backup while responding to an audit, conducting an internal investigation, or addressing a legal matter. In these situations, a fixed retention period may not be sufficient. Azure SQL LTR backup immutability also supports legal holds, allowing customers to protect specific existing backups until the hold is explicitly removed. For example, if an organization needs to preserve a database backup associated with an ongoing investigation, it can apply a legal hold to prevent that backup from being deleted while the investigation continues. Legal holds can coexist with time-based immutability, providing flexibility for organizations that need both predictable retention policies and event-driven preservation requirements. This makes immutable backups valuable not only for ransomware resilience, but also for meeting evolving business and governance needs. 5. Consistent protection across Azure SQL Database and Azure SQL Managed Instance For organizations operating both Azure SQL Database and Azure SQL Managed Instance, consistency matters. Security and compliance teams shouldn't have to build entirely different backup protection strategies depending on where a database runs. With a consistent immutability experience across both services, customers can adopt a common approach to protecting recent and historical backups. Here's how the capabilities work together: Capability Customer benefit Default PITR immutability Automatic protection for up to seven days of recent backups Locked PITR immutability Extended protection aligned with configured PITR retention, up to 35 days Time-based LTR immutability Protect historical backups for their configured retention period, up to 10 years LTR legal holds Preserve selected backups until the hold is explicitly removed Together, these capabilities provide flexibility to address different recovery and retention requirements, from recovering a database to a recent point in time to preserving historical backups for years. And because the protection is integrated into Azure SQL's managed backup experience, customers don't need to build or maintain separate immutable backup storage infrastructure. 6. Building a stronger defense against ransomware Let's revisit a common ransomware scenario. An attacker gains access to a privileged account and attempts to disrupt business operations by modifying production data and deleting available backups. Without immutability, the attacker may be able to eliminate critical recovery options. With immutable backups, the story changes. Recent recovery points are protected: Default PITR immutability helps preserve backups from the most recent seven days. Extended recovery history can be safeguarded: Locked PITR immutability can protect a longer recovery window, giving organizations more options when investigating an incident. Historical backups remain protected: Locked LTR immutability helps prevent attackers from deleting backups that organizations need to retain for months or years. Critical records can be preserved: Legal holds provide additional protection for specific backups that must remain available during investigations or audits. While backup immutability doesn't prevent ransomware attacks or guarantee that every restore point is free from compromise, it makes it significantly harder for attackers to eliminate protected backups. That distinction is important. A strong ransomware recovery strategy isn't just about having backups. It's about being able to trust that your backups will still be there when you need them. Immutable backups complement other security and resilience practices, including identity protection, least-privilege access, monitoring, appropriate backup retention, and regular recovery testing. 7. Get started with immutable backups in Azure SQL Getting started is straightforward. If you're using a supported Azure SQL Database or Azure SQL Managed Instance database, you already benefit from default PITR backup immutability for up to seven days. From there, consider your organization's recovery and compliance needs: Review your PITR retention settings. Determine whether the default protection window meets your recovery objectives or whether you need a longer immutable retention period. Consider locked PITR immutability. For databases requiring extended protection, align immutability with your configured PITR retention. Evaluate your long-term backup requirements. Configure LTR immutability for backups that must remain protected for extended periods. Use legal holds when needed. Preserve specific historical backups for audits, investigations, or legal requirements. Test your recovery procedures. Validate that your organization can restore the right data when an incident occurs. Before applying permanent retention locks, carefully evaluate your requirements, because locked protection cannot be removed early. Also remember that backup immutability protects backups, not the database resource itself, so resource deletion protection and recovery planning remain important. Protecting your data. Preserving your ability to recover. As cyber threats continue to evolve, organizations need confidence not only that their data is protected, but also that they can recover when something goes wrong. With backup immutability spanning PITR and LTR backups across Azure SQL Database and Azure SQL Managed Instance, we're making it easier for customers to build stronger, more consistent data protection strategies. From automatic protection for recent backups to long-term immutable retention and legal holds, these capabilities help organizations safeguard critical recovery points, reduce ransomware-related risks, and strengthen their overall resilience. Because when the unexpected happens, the most important backup is the one you can still recover from. Learn more: PITR backup immutability for both Azure SQL DB and Azure SQL MI - Automatic backup immutability for Azure SQL Database and Azure SQL Managed Instance - Azure SQL | Microsoft Learn Azure SQL DB LTR backup immutability - Backup Immutability for Long-Term Retention Backups - Azure SQL Database | Microsoft Learn Azure SQL MI LTR backup immutability - LTR backup immutability - Azure SQL Managed Instance | Microsoft Learn My colleague SudhanshuKhera authored a comprehensive ransomware recovery plan for Azure SQL DB and Azure SQL MI - Overview: Recover Azure SQL Database and Azure SQL Managed Instance from a ransomware attack - Azure SQL Database | Microsoft Learn187Views0likes1CommentDatabase Hub in Microsoft Fabric is generally available: your entire database estate in one place
As applications grow, so does the database estate behind them. Many organizations now run hundreds or thousands of databases across cloud, hybrid and multicloud environments. The signals are already there: inventory, health, performance, security, risk and optimization. The hard part is making sense of them, knowing what needs attention and deciding what to do next. Today, we're announcing the general availability of Database Hub in Microsoft Fabric. Database Hub gives you one place to see your whole estate, monitor it at scale and investigate where attention is needed. It covers Azure SQL, Azure Database for PostgreSQL, Azure Cosmos DB, SQL Server and SQL database in Fabric. Everything in this release is built around that idea: one estate, every signal, wherever you work. Get started for free If you're new to Microsoft Fabric, you don't need to set anything up first. Open Database Hub, and you're automatically given a free Fabric license and taken straight to your estate. You only need Fabric capacity to share dashboards, create custom alerts or use Copilot. Generally available today Area What you get Overview A "what needs attention" view across security, performance and optimization, with the top issues and recent signal changes across SQL, PostgreSQL and Cosmos DB Estate inventory A single inventory of supported Azure and Fabric databases, with their issues and suggestions Performance monitoring Near real-time CPU, memory and storage trends, aggregated across the estate and per resource, plus normalized RU and throttled requests for Cosmos DB Security posture Findings such as public network access, missing private endpoints, Microsoft Entra authentication, audit logging and customer-managed keys Investigation The ability to drill from any finding into the affected resource, real-time dashboards and the tools you already use New in general availability: Bulk performance monitoring enablement Monitoring a large estate shouldn't mean configuring it one database at a time. You can now turn on performance monitoring at scale from the Estate view. Select multiple resources and enable performance monitoring data collection for all of them in one step. Start from your inventory. Turn on monitoring in the same place you review your databases, without switching to another portal or tool. Cover more of your estate, faster. Extend monitoring coverage across hundreds of databases without repeating setup for each one. Go from setup to insight. Once data collection is on, monitored resources show up in Performance Monitoring and real-time dashboards. New in general availability: Optimization insights Optimization now sits alongside security and performance in Overview and Estate. It highlights: Resources with a migration assessment ready License model opportunities, such as pay-as-you-go SQL Server instances that aren't reporting usage Now generally available: Real-time dashboards Real-time dashboards for Microsoft SQL, Azure Database for PostgreSQL and Azure Cosmos DB move from preview to general availability. Now generally available: VS Code and SSMS entry points DBAs work in SSMS, and developers work in VS Code. You can now open Database Hub from SSMS and from the Microsoft and MSSQL extensions for VS Code. You land in the same estate view, with the resources you were working on. SSMS VS Code Now generally available: Database Hub CLI skill Ask about estate inventory, health, performance and security posture from GitHub Copilot CLI, VS Code Copilot, Claude Code or Cursor. Get the Database Hub CLI skill from skills-for-fabric, or install it with one command: skills-for-fabric --skill databasehub-cli Then try asking, "How is my database estate doing?" or "Which databases need attention this week?" In preview Public preview: Investigate your estate with Copilot Ask questions about your databases in natural language and get answers grounded in Database Hub data. Explore trends, understand issues and suggestions, and follow up on specific resources. Try prompts such as: "What changed in my databases?" "Which resource should I investigate next?" "Which security findings should I address first?" Public preview: Database Agents Database Agents act as an extra set of eyes on your estate. An agent watches the databases you choose and stays quiet while they're healthy. When something needs your judgment, it raises an issue with supporting evidence and a recommendation. You stay in control: the agent brings the issue to you, and you decide what happens next. Start with the overall finding or select Analyze on a single query to get recommendations for that query. From there, keep investigating with Copilot or go deeper in SSMS. Database Hub is where you see the bigger picture. The database agent alerts you before your users notice. Public preview: Alerts for Azure SQL Database Create alerts from built-in templates or performance charts for one database or many. Alerts can send a Microsoft Teams message or start a Power Automate flow. They're powered by Fabric Activator. Preview: Performance monitoring for SQL Server on Azure VMs Review SQL Server on Azure Virtual Machines in the same Performance Monitoring experience as the rest of your estate. What's next More skills are on the way, including SQL performance monitoring, Cosmos DB performance monitoring and migration skills. Get started today Open Database Hub: https://app.fabric.microsoft.com/workloads/fdh/databaseHub/overview Learn more: https://learn.microsoft.com/en-us/fabric/database/hub Install the Database Hub CLI skill: https://github.com/microsoft/skills-for-fabric/tree/main/skills/databasehub-cli Read more about Database Hub Database Hub in Fabric: Now in Public Preview -Turn Database Signals into Guided Action | Microsoft Community Hub Public Preview: Performance monitoring for Azure SQL in Database Hub | Microsoft Community Hub A Single View of Your PostgreSQL Estate: Database Hub in Public Preview | Microsoft Community Hub Make Estate yours: Focus on the databases and issues you own in Database Hub | Microsoft Community Hub Public Preview: Alerts for Azure SQL Database in Database Hub, powered by Fabric Activator | Microsoft Community Hub Managing security with Database Hub in Fabric: From signals to action | Microsoft Community Hub Database watcher for Azure SQL is retiring: simplify your monitoring with Database Hub | Microsoft Community Hub Ask your database estate a question: Copilot in Database Hub is now in public preview | Microsoft Community Hub433Views3likes2CommentsMI link support for multiple databases in an Always On availability group is now generally available
We are excited to announce the general availability of multi-database mode for MI link, which enables you to replicate multiple databases in an Always On availability group between SQL Server and Azure SQL Managed Instance through a single MI link. Managed Instance link provides near-real-time database replication between SQL Server and Azure SQL Managed Instance, helping you modernize to Azure at your own pace. Built on distributed availability group technology, MI link provides a secure and reliable connection between SQL Server and Azure SQL Managed Instance and can be used for migration, disaster recovery, and hybrid scenarios. With multi-database mode, you can extend an existing Always On availability group to Azure SQL Managed Instance and replicate all databases in that availability group through a single MI link. You can also start with standalone databases. When standalone databases are selected in SQL Server Management Studio (SSMS), the wizard creates a single-node availability group and adds the selected databases to it. With general availability, multi-database mode makes it easier to manage scenarios involving multiple databases while maintaining the flexibility and near-real-time replication provided by MI link. What’s new with general availability General availability introduces these important updates compared with the public preview: All Azure SQL Managed Instance update policies are now supported, including the SQL Server 2022 update policy, which was not supported during the public preview. You can now set up bidirectional multi-database MI link mode between SQL Server 2022 and Azure SQL Managed Instance using the SQL Server 2022 update policy. Note that SQL Server 2025 and Managed Instances using the SQL Server 2025 update policy have been supported since public preview. With SQL Server 2022 CU11 and SQL Server 2025 CU28 as a part of the general availability release, we are introducing three new stored procedures to improve supportability and troubleshooting of MI link in multi-database mode: sys.sp_milink_check - provides information about the replication health of individual databases in multi-database mode. Use this stored procedure to determine whether an individual database is stuck in seeding or replication. sys.sp_milink_reseed - restarts seeding for individual databases that are stuck in seeding in multi-database mode without requiring you to remove and re-add the database to the availability group. sys.sp_milink_repair - repairs an unhealthy replication for an individual database in multi-database mode link. Use SQL Server Management Studio (SSMS) 22.12.1 when configuring and managing MI link in multi-database mode. This version includes a number of experience improvements and bug fixes. We are also introducing better support for multiple SQL Server nodes in Always On availability groups: Multi-node availability group support in the MI link wizard (preview) enables automatic configuration of all server nodes in an Always On availability group with MI link, allowing the link to persist after an availability group failover. Instead of configuring each server node yourself, the MI link wizard can configure all server nodes for you. This SSMS feature is in preview and must be enabled by selecting Tools > Manage Preview Features in SSMS. Support for the SQL Server 2022 update policy expands the range of SQL Managed Instance environments that can take advantage of multi-database mode and provides greater flexibility when planning migrations and hybrid configurations. Multi-node availability group support in the MI link wizard (preview) eliminates the need to manually configure each server node, saving time and helping ensure that the MI link persists across local availability group failovers. How multi-database mode works With multi-database mode, MI link extends an Always On availability group between SQL Server and Azure SQL Managed Instance. A single link replicates multiple databases in the availability group to the target replica. The link uses a distributed availability group to replicate changes in near real time from the current primary replica to database copies on the secondary replica, keeping the databases synchronized. You can: Extend an existing Always On availability group to Azure SQL Managed Instance. Start with standalone SQL Server databases and let SSMS create the availability group. Add or remove databases from the availability group as your environment evolves. Fail over or cut over the availability group to Azure SQL Managed Instance when you are ready. This approach simplifies working with applications that rely on multiple databases because the databases can be managed and migrated together through a single MI link rather than through separate links for each database. Supportability Requirement Details SQL Server SQL Server 2022 with CU28 and later or SQL Server 2025 with CU11 and later. Enterprise or Developer edition. Standard edition is not supported for multi-database mode because Basic availability groups support only a single database. Azure SQL Managed Instance All service tiers, General Purpose, Next-Gen and Business Critical are supported. Use the following Azure SQL Managed Instance update policy for the following configurations: For one-way replication from SQL Server to Azure SQL Managed Instance: For one-way replication from SQL Server 2022, use Azure SQL Managed Instance with update set to SQL Server 2022, SQL Server 2025, or Always-up-to-date. For one-way replication from SQL Server 2025, use Azure SQL Managed Instance with update policy set to SQL Server 2025, or Always-up-to-date. SQL Server 2022 update policy is not supported. For two-way replication and bidirectional failover (role-reversal) between SQL Server and Azure SQL Managed Instance, the update policy must match the exact version of your SQL Server: For bidirectional replication and failover between SQL Server 2022 and Azure SQL Managed Instance, use SQL Server 2022 update policy only. Other update policies are not supported. For bidirectional replication and failover between SQL Server 2025 and Azure SQL Managed Instance, use SQL Server 2025 update policy only. Other update policies are not supported. SSMS SSMS 22.12.1 or later for the multi-database link capability. Automation Az module 16.3.0 or later and Az.Sql 7.1.0 or later, or the corresponding Azure APIs. One link for an existing multi-database availability group If you already use an Always On availability group, multi-database link mode lets you extend the complete database group to Azure SQL Managed Instance without creating a separate availability group for every database. All databases in the link move together as one managed group. Databases on the primary are read-write, while their copies on the secondary are read-only. Use of your existing Always On AG listener endpoint is supported with multi-database mode MI link, allowing the link to remain operational after a local AG failover. : A multi-database Always On availability group replicated to Azure SQL Managed Instance through one Managed Instance link. The diagram illustrates the availability group relationship rather than a specific Azure SQL Managed Instance service-tier replica count. Managed Instance link is supported across all Azure SQL Managed Instance service tiers. Run multiple links in different directions A single SQL Server instance can participate in multiple links. Each link can carry a different availability group, and supported links can replicate in different directions at the same time. In the following example, AG1 (containing DB1, DB2, and DB3) replicates from SQL Server to Azure SQL Managed Instance through MI link 1. AG2 (containing DB4, DB5, and DB6) replicates from Azure SQL Managed Instance to SQL Server through MI link 2. Both links can operate at the same time between the two instances. : Two multi-database availability groups using separate links with opposite replication directions. Important: Database names must be unique in this configuration. A database cannot be renamed on the secondary while it is participating in replication to resolve a naming conflict. Replicate one availability group to multiple managed instances Multi-database mode also supports fan-out topologies. You can replicate multiple availability groups to one managed instance, or replicate the same availability group to different managed instances. For example, separate links can target managed instances in different Azure regions. : One multi-database availability group replicated through separate links to two Azure SQL managed instances. Get started To use MI link multi-database mode, install SSMS 22.12.1 and make sure that your SQL Server and Azure SQL Managed Instance environments meet the requirements for the scenario you want to configure. To enable automatic configuration of multiple server nodes in your availability group, in SSMS select Tools > Manage Preview Features, and select "Enable multi-node availability group support in the Managed Instance link wizard." Learn more about configuring MI link with multiple databases in an Always On availability group. Share your feedback We would love to hear about your experience with multi-database mode. Please share questions, feedback, and feature suggestions through the Managed Instance link feedback form.193Views0likes0CommentsFuzzy string matching is now generally available across Microsoft SQL
Four native T-SQL functions that tell you how close two strings are, not just whether they’re equal. MICROSFT isn’t MICROSOFT. MARHTA isn’t MARTHA. INV-1O08 isn’t INV-1008. To an equality check, each pair is simply different. To your business, each is a near miss worth catching. Today, fuzzy string matching is generally available in Microsoft SQL. EDIT_DISTANCE tells you MICROSFT is one edit from MICROSOFT. JARO_WINKLER_SIMILARITY scores MARTHA and MARHTA at 96 out of 100. It all runs in T-SQL, right where your data lives, with no external libraries, models, or custom code. Available across all SKUs of: Azure SQL Database Azure SQL Managed Instance SQL database in Microsoft Fabric SQL Server 2025, starting with Cumulative Update 11 (CU11), and later cumulative updates What fuzzy string matching actually means Fuzzy string matching measures how closely two strings resemble each other without requiring an exact match. Think of it as comparing the written form of two values. An exact comparison asks, “Are these equal under our comparison rules?” Fuzzy matching adds, “If not, how close are they?” For an edit-distance algorithm, closeness can be measured by the operations needed to transform one string into the other: FORM swap O and R FROM One adjacent swap makes the strings identical. Insertion, deletion, substitution, and transposition are transformation operations, not assumptions about what originally caused the difference. This is useful for typo-tolerant lookup, candidate duplicate detection, master-data reconciliation, and reviewing text captured by optical character recognition (OCR). It is not the same as understanding meaning. car and automobile are related concepts but have very different character sequences. Nor does a high similarity score establish that two records represent the same person, product, or organization. Fuzzy matching provides evidence for a decision, not the decision itself. Meet the four functions The feature provides two algorithm families, each exposed through a distance function and a similarity function. Function Algorithm Return type How to read the result EDIT_DISTANCE Damerau-Levenshtein (Optimal String Alignment) int Edit count, or an over-budget marker when limited; lower means closer EDIT_DISTANCE_SIMILARITY Damerau-Levenshtein (Optimal String Alignment) int Normalized score from 0 to 100; higher means closer JARO_WINKLER_DISTANCE Jaro-Winkler float Distance from 0 to 1; lower means closer JARO_WINKLER_SIMILARITY Jaro-Winkler int Score from 0 to 100; higher means closer Their signatures are: EDIT_DISTANCE ( character_expression , character_expression [ , maximum_distance ] ) EDIT_DISTANCE_SIMILARITY ( character_expression , character_expression ) JARO_WINKLER_DISTANCE ( character_expression , character_expression ) JARO_WINKLER_SIMILARITY ( character_expression , character_expression ) The inputs can be constants, variables, or columns. Use bounded character types such as varchar(n) or nvarchar(n); varchar(max) and nvarchar(max) are not supported. If an input is NULL, the result is NULL. A first comparison Each example is self-contained, creates no permanent objects, and applies a supported collation (Latin1_General_100_CI_AS_SC) with COLLATE, so you can run it without changing your database collation. ;WITH Pair AS ( SELECT N'Colour' COLLATE Latin1_General_100_CI_AS_SC AS SourceText, N'Color' COLLATE Latin1_General_100_CI_AS_SC AS TargetText ) SELECT SourceText, TargetText, EDIT_DISTANCE(SourceText, TargetText) AS Edits, EDIT_DISTANCE_SIMILARITY(SourceText, TargetText) AS EditScore, JARO_WINKLER_DISTANCE(SourceText, TargetText) AS JwDistance, JARO_WINKLER_SIMILARITY(SourceText, TargetText) AS JwScore FROM Pair; The query returns: Edits EditScore JwDistance, approximately JwScore 1 83 0.033333 97 One deletion removes the u. Both algorithm families recognize a close match, but they express closeness differently. A Jaro-Winkler distance of approximately 0.033333 is not a fractional number of edits. Damerau-Levenshtein (Optimal String Alignment): how many edits? EDIT_DISTANCE and EDIT_DISTANCE_SIMILARITY implement Damerau-Levenshtein (Optimal String Alignment, or OSA), the restricted Damerau-Levenshtein variant. Each permitted operation has a cost of one: Operation Example What changes Insertion CT to CAT Insert A Deletion CART to CAT Delete R Substitution CAT to CUT Replace A with U Adjacent transposition FORM to FROM Swap O and R The distance is the minimum cost of a transformation allowed by this algorithm, so each pair above has a distance of 1. Why “Optimal String Alignment” matters The qualification is important: this is not unrestricted Damerau-Levenshtein. In the Optimal String Alignment variant, the same substring cannot be edited more than once along the transformation. For example, unrestricted Damerau-Levenshtein can transform CA into ABC in two operations: CA -> AC -> ABC swap insert B That sequence re-edits the transposed region. The restricted Damerau-Levenshtein (Optimal String Alignment) algorithm does not allow that transformation path, and the distance is 3, not 2. SELECT EDIT_DISTANCE( N'CA' COLLATE Latin1_General_100_CI_AS_SC, N'ABC' COLLATE Latin1_General_100_CI_AS_SC ) AS Edits; -- Result: 3 For everyday typos this rarely matters, but it does when you compare results with another library or migrate an existing matching implementation. From edit count to similarity An absolute edit count does not tell the whole story. Removing one character from a four-character value changes a larger proportion of the string than removing one character from a six-character value. EDIT_DISTANCE_SIMILARITY normalizes edit distance using the longer input: (1 - (edit_distance / greatest(len(string1), len(string2)))) * 100 For example: CART and CAT: (1 - 1/4) * 100 = 75. Colour and Color: (1 - 1/6) * 100 is approximately 83.33; the function returns the integer 83. A score of 83 measures resemblance; it is not an 83% probability that the values refer to the same thing. Jaro-Winkler: a different way to rank resemblance Sometimes the question is not “How many edits are allowed?” but “Which names are the most plausible matches?” Jaro-Winkler considers matching characters within a positional matching window: a character must occur close enough to its position in the other string to count as a match. It also considers the relative order of those matches, then gives additional weight to a shared prefix. It does not calculate the minimum number of operations needed to transform one string into another. Consider MARTHA and MARHTA. They share the same characters, with T and H out of order, and both start with MAR. ;WITH Names AS ( SELECT NameA COLLATE Latin1_General_100_CI_AS_SC AS NameA, NameB COLLATE Latin1_General_100_CI_AS_SC AS NameB FROM (VALUES (N'MARTHA', N'MARHTA'), (N'CONTOSO HEALTH', N'CONTOSO RETAIL') ) AS n(NameA, NameB) ) SELECT NameA, NameB, EDIT_DISTANCE(NameA, NameB) AS Edits, JARO_WINKLER_DISTANCE(NameA, NameB) AS JwDistance, JARO_WINKLER_SIMILARITY(NameA, NameB) AS JwScore FROM Names ORDER BY JwScore DESC, NameA, NameB; NameA NameB Edits JwDistance, approximately JwScore MARTHA MARHTA 1 0.038889 96 CONTOSO HEALTH CONTOSO RETAIL 5 0.082143 92 Jaro-Winkler is worth evaluating for person and organization names when shared beginnings are meaningful and usually entered correctly. But prefixes can mislead: CONTOSO HEALTH and CONTOSO RETAIL score 92 despite five edits and may be different organizations. The prefix length and scaling factor are fixed; the functions don’t expose parameters for them. Choose Damerau-Levenshtein (OSA) for an explicit edit budget. Evaluate Jaro-Winkler for ranking character-based resemblance when prefix preference fits your data. Use maximum_distance when you only need a threshold Suppose your application accepts candidates within two edits. An exact distance of eight is unnecessary if all you need to know is that the candidate exceeds the budget. The optional maximum_distance argument lets EDIT_DISTANCE stop early once the distance exceeds the threshold. Within the threshold, it returns the actual distance; beyond it, it returns maximum_distance + 1. ;WITH Pairs AS ( SELECT SourceText COLLATE Latin1_General_100_CI_AS_SC AS SourceText, TargetText COLLATE Latin1_General_100_CI_AS_SC AS TargetText FROM (VALUES (N'CART', N'CAT'), (N'FORM', N'FROM'), (N'Chocolate', N'Sweets'), (N'CAT', N'CAT') ) AS p(SourceText, TargetText) ) SELECT SourceText, TargetText, EDIT_DISTANCE(SourceText, TargetText) AS ActualDistance, EDIT_DISTANCE(SourceText, TargetText, 2) AS BoundedDistance FROM Pairs ORDER BY SourceText; Pair ActualDistance BoundedDistance CART / CAT 1 1 CAT / CAT 0 0 Chocolate / Sweets 8 3 FORM / FROM 1 1 The value 3 for Chocolate / Sweets means “over the budget of two”, not “exactly three edits.” Use the bounded form to accept or reject candidates, not to rank how far apart rejected values are. Put it to work: typo-tolerant lookup Let’s return to MICROSFT. We want a short list of catalog values no more than one edit away. DECLARE @query nvarchar(40) = N'MICROSFT'; ;WITH Catalog AS ( SELECT Name COLLATE Latin1_General_100_CI_AS_SC AS Name FROM (VALUES (N'MICROSFT'), (N'MICROSOFT'), (N'CONTOSO') ) AS c(Name) ) SELECT Name, EDIT_DISTANCE( Name, @query COLLATE Latin1_General_100_CI_AS_SC ) AS Distance FROM Catalog WHERE EDIT_DISTANCE( Name, @query COLLATE Latin1_General_100_CI_AS_SC, 1 ) <= 1 ORDER BY Distance, Name; The result contains MICROSFT with distance 0, followed by MICROSOFT with distance 1; CONTOSO is outside the budget. The misspelled catalog entry is included deliberately: exact matches rank first, and the application can offer the alternatives instead of silently rewriting the input. The bounded call filters, while the unbounded call supplies the ranking value. Find candidate duplicates without comparing everyone to everyone Customer deduplication needs both textual resemblance and business context. Comparing every customer with every other customer produces a number of pairs that grows approximately with the square of the row count. A practical first step is blocking: group records using a trusted attribute, then compare names within those groups. ;WITH Customer AS ( SELECT CustomerId, Name COLLATE Latin1_General_100_CI_AS_SC AS Name, PostalCode FROM (VALUES (1, N'MARTHA', N'98052'), (2, N'MARHTA', N'98052'), (3, N'MARTHA', N'10001'), (4, N'ROBERT', N'98052') ) AS c(CustomerId, Name, PostalCode) ) SELECT a.CustomerId AS CustomerA, b.CustomerId AS CustomerB, a.Name AS NameA, b.Name AS NameB, JARO_WINKLER_SIMILARITY(a.Name, b.Name) AS Score FROM Customer AS a JOIN Customer AS b ON a.PostalCode = b.PostalCode AND a.CustomerId < b.CustomerId WHERE JARO_WINKLER_SIMILARITY(a.Name, b.Name) >= 92 ORDER BY Score DESC, CustomerA, CustomerB; Three details make the query useful: Postal-code equality creates the shortlist. Only customers within the same postal code are compared. a.CustomerId < b.CustomerId emits each pair once. It excludes self-comparisons and reversed duplicates such as (2,1) when (1,2) already exists. The similarity filter flags plausible names. MARTHA and MARHTA pass the illustrative cutoff; pairs involving ROBERT do not. The query returns one pair: customer 1 and customer 2, with a similarity score of 96. The other MARTHA, in postal code 10001, is not compared with the 98052 group. That illustrates the trade-off: blocking reduces comparisons but can miss true duplicates when the blocking attribute differs or is wrong. The cutoff of 92 is only illustrative: CONTOSO HEALTH and CONTOSO RETAIL also scored 92 earlier. Confirm candidates with independent evidence, such as a verified email or phone number. Reconcile OCR mistakes within the right business context An invoice reader captures INV-1O08, confusing the letter O with the digit 0. Comparing against every invoice would be risky. Start with the vendor, then allow one character edit: ;WITH Captured AS ( SELECT 10 AS VendorId, N'INV-1O08' COLLATE Latin1_General_100_CI_AS_SC AS Invoice ), ReferenceInvoice AS ( SELECT VendorId, Invoice COLLATE Latin1_General_100_CI_AS_SC AS Invoice FROM (VALUES (10, N'INV-1008'), (10, N'INV-1208'), (20, N'INV-1008') ) AS r(VendorId, Invoice) ) SELECT c.VendorId, c.Invoice AS CapturedInvoice, r.Invoice AS CandidateInvoice, EDIT_DISTANCE(c.Invoice, r.Invoice) AS Edits FROM Captured AS c JOIN ReferenceInvoice AS r ON c.VendorId = r.VendorId WHERE EDIT_DISTANCE(c.Invoice, r.Invoice, 1) <= 1 ORDER BY c.VendorId, r.Invoice; The query returns two candidates for vendor 10: Candidate invoice Transformation Edits INV-1008 Replace O with 0 1 INV-1208 Replace O with 2 1 Vendor 20’s invoice is excluded by the vendor condition. Within vendor 10, both substitutions cost one edit; the function doesn’t know that OCR often confuses O and 0. Attributes such as amount and date must decide which invoice was intended. Useful with AI, without being an AI model These functions require no embeddings, model training, or inference endpoint. They also fit naturally into AI-assisted workflows. For example, a language model might extract CONTOS from a document. The application can use fuzzy matching to shortlist names in its organization directory. The same SQL works if the input came from a person or a batch import instead. DECLARE @entity nvarchar(100) = N'CONTOS'; ;WITH Organization AS ( SELECT OrganizationId, Name COLLATE Latin1_General_100_CI_AS_SC AS Name, CountryCode FROM (VALUES (1, N'CONTOSO', N'US'), (2, N'CONTOSO HEALTH', N'US'), (3, N'CONTOSO RETAIL', N'US'), (4, N'CONTOSO', N'GB') ) AS o(OrganizationId, Name, CountryCode) ) SELECT TOP (5) OrganizationId, Name, JARO_WINKLER_SIMILARITY( Name, @entity COLLATE Latin1_General_100_CI_AS_SC ) AS Score FROM Organization WHERE CountryCode = N'US' AND JARO_WINKLER_SIMILARITY( Name, @entity COLLATE Latin1_General_100_CI_AS_SC ) >= 80 ORDER BY Score DESC, OrganizationId; The query returns three US candidates: CONTOSO with a score of 97, followed by CONTOSO HEALTH and CONTOSO RETAIL at 89 each. The GB record is excluded. Change @entity to N'ZZZZ' and the query returns no rows; the application should treat that as “no match” rather than force a best guess. Fuzzy matching won’t expand an acronym into a company name or resolve related meanings. Use an alias table or a semantic approach for those tasks. Where this fits in the SQL matching toolbox Fuzzy string matching extends existing capabilities; it does not replace them. Question Appropriate tool Example Are these values equal under the intended comparison rules? Equality, joins, and an appropriate collation Match a trusted customer key Does the text contain a known pattern or substring? LIKE, CHARINDEX, PATINDEX Find a known prefix Does the text follow a structured pattern? REGEXP_* functions Validate or extract a formatted value Do these values have similar phonetic representations? SOUNDEX, DIFFERENCE Coarse screening of English-oriented names How far apart are these values in character edits? EDIT_DISTANCE, EDIT_DISTANCE_SIMILARITY Measure edit count or normalized edit similarity How similar are character alignment and prefixes? JARO_WINKLER_DISTANCE, JARO_WINKLER_SIMILARITY Rank plausible name matches Which documents contain relevant words or phrases? Full-text search Linguistic document retrieval Which representations are close in embedding space? Vector similarity/search Semantic retrieval for AI applications For example, SOUNDEX('Robert') and SOUNDEX('Rupert') both return R163: a phonetic signal, which is different from the character-similarity scores announced here. Good to know Collation: The functions support Windows collations and binary (BIN, BIN2) collations. Non-binary SQL_* collations, such as the common default SQL_Latin1_General_CP1_CI_AS, are not supported. Apply COLLATE to the inputs, as the examples do; you don’t need to change the database collation. Scale: Narrow the candidate set with trusted business keys, such as region, vendor, or category, before scoring. Scores: Test thresholds on your own labeled examples. A score is evidence of a possible match, not proof. Get started Azure SQL Database and SQL database in Microsoft Fabric: The functions are available now. Azure SQL Managed Instance: Use the SQL Server 2025 or Always-up-to-date update policy. SQL Server 2025: Install CU11 or later. If you tried the functions in preview, you no longer need the PREVIEW_FEATURES database scoped configuration to use them. Pick a column you already struggle to match, try the examples above on your own data, and see what turns up. Whether you’re cleaning customer records, forgiving a typo in a search box, or grounding an AI-extracted name, fuzzy string matching is now a native part of T-SQL, ready wherever your data lives. We’d love to hear what you build with it. Learn more Fuzzy string matching overview EDIT_DISTANCE reference EDIT_DISTANCE_SIMILARITY reference JARO_WINKLER_DISTANCE reference JARO_WINKLER_SIMILARITY reference97Views0likes0CommentsTake control of transparent data encryption scans: suspend and resume now in preview
Suspend and resume for transparent data encryption (TDE) scans in Azure SQL Database lets you temporarily halt an encryption scan during busy periods and continue it later from the exact point where it stopped, instead of starting over. Why we built it TDE protects your databases, backups, and transaction log files at rest, without any changes to your application. When you turn encryption on or off, the service runs an encryption scan across the database. The scan duration is proportional to the size of the database, and for large databases it can take a long time. While it runs, the scan consumes CPU and I/O, which can affect user workloads during peak hours. Until now, scans were all or nothing: once started, a scan had to complete. What is new Suspend and resume gives database administrators and owners control over the encryption scan: Suspend temporarily halts the encryption scan on the data and log files and releases the resources it was using. Resume restarts the scan from the point of suspension. While a scan is suspended, new or updated data pages continue to be handled according to the direction of the scan. If you were encrypting, new pages are still encrypted. If you were decrypting, new pages are written unencrypted. You can also toggle direction while a scan is suspended. For example, you can start encryption, suspend it, and then turn encryption off. The decryption scan starts from the beginning. The same applies in reverse. This works for TDE with service-managed keys and with customer-managed keys. Manage encryption status in the Azure portal You can see and control the state of the encryption scan directly in the Azure portal. On the transparent data encryption page of your database, the encryption status shows whether the database is encrypted, being encrypted, or suspended, and offers the matching action. Encryption in progress While the scan is running, the status shows that encryption is in progress and a suspend button appears to pause the scan. Suspended When you suspend the scan, the status changes to suspended and a resume button appears so you can continue from where you left off. Encrypted Once the scan completes, the database shows as encrypted. Use T-SQL to suspend and resume The same commands that exist in SQL Server are available in Azure SQL Database. Suspend an encryption scan ALTER DATABASE my_db_name SET ENCRYPTION SUSPEND Resume a suspended encryption scan ALTER DATABASE my_db_name SET ENCRYPTION RESUME Start encryption, suspend it, then decrypt again ALTER DATABASE my_db_name SET ENCRYPTION ON ALTER DATABASE my_db_name SET ENCRYPTION SUSPEND ALTER DATABASE my_db_name SET ENCRYPTION OFF Monitor the scan Track the state of the scan with the sys.dm_database_encryption_keys dynamic management view, which reports the encryption state of the database and its encryption keys. SELECT DB_NAME(database_id), * FROM sys.dm_database_encryption_keys The percent_complete column shows the progress of the encryption state change. Get started and share feedback Suspend and resume is available in public preview for Azure SQL Database. Public preview features are covered by the Supplemental Terms of Use for Microsoft Azure Previews. To learn more about how TDE works, including service-managed and customer-managed keys, see the TDE overview on Microsoft Learn. Try it on your databases and tell us what you think. Your feedback helps us shape the feature before general availability.111Views0likes0CommentsWhat's new in databases at Microsoft Ignite 2026: AI-ready data for apps and agents
By Shireesh Thota, Corporate Vice President, Azure Databases and Fabric Applications, Microsoft Organizations putting agents to work are learning quickly that an agent is only as good as the business context behind it. That context starts with trusted business data. Every transaction, customer interaction and operational event tells AI what is happening right now, and much of it lives in operational databases. Microsoft IQ helps bring that context together for agents. As AI becomes more capable, the database becomes even more important. With Azure databases and Microsoft Fabric, we're making that data estate work as one: unified, rich with business context and governed at the source. But that data is spread across today's business applications, while new intelligent apps demand more scale, speed, and context. Organizations need to do three things at once: Build what comes next on cloud-native databases designed for scale, analytics, and AI. Modernize the trusted data they already have onto AI-ready foundations. Operate a growing estate with the help of agents. Here's how our top announcements at Microsoft Ignite help with each. Build on one PostgreSQL foundation, from Azure to Fabric At Ignite last year, we introduced Azure HorizonDB in preview. Today, Azure HorizonDB is generally available: a fully managed, PostgreSQL-compatible database service built for mission-critical and AI-powered applications. With Azure HorizonDB, teams can build AI applications directly on transactional data using built-in vector search, AI Functions and AI Pipelines that invoke models from SQL queries, along with built-in AI model management in preview. With the PostgreSQL extension for VS Code, developers can manage, tune, and monitor their databases with AI assistance in the same editor where they write their application code. For critical workloads, multi-zone high availability and distributed storage support fast recovery to help keep applications running. Microsoft Entra ID, private endpoints, managed identities, and service- or customer-managed keys help protect sensitive data, bringing resilience and security together with HorizonDB’s AI capabilities. "Our AI ambitions depend on secure, reliable, high-performance API access to LSEG’s data. Azure HorizonDB meets our enterprise security requirements while providing a PostgreSQL foundation for our existing API services and the MCP layer powering new AI experiences.” Grzegorz Giedrojc, Director of Engineering, LSEG (London Stock Exchange Group) Serverless PostgreSQL in Fabric, from first query to OneLake In Microsoft Fabric, we're bringing the HorizonDB foundation closer to analytics with serverless PostgreSQL built for applications and agents. As AI accelerates development, databases need to start just as quickly and handle bursty workloads. Microsoft Fabric HorizonDB provisions databases in seconds, pauses after five minutes of inactivity and resumes with the next query, giving developers and agents PostgreSQL that scales with how they build, without infrastructure to manage. Fabric HorizonDB is also available as a PostgreSQL database for Fabric apps. With the Rayfin SDK, PostgreSQL joins data APIs, authentication, storage, and hosting in one managed backend, so builders can focus on application logic. Beyond serverless operations, Fabric HorizonDB supports translytical capabilities, combining transactions and analytics, on OneLake, bringing the scale and performance of Fabric’s analytical engines to applications working with live transactional data: Run translytical queries. PostgreSQL queries are transparently pushed down to Fabric Data Warehouse for distributed CPU- or GPU-powered execution. Full and partial pushdown allow a single query to join PostgreSQL tables with data in OneLake, with analytical queries running an order of magnitude faster than they would natively in PostgreSQL. Bring operational data into analytics automatically. Mirroring continuously replicates HorizonDB tables into Fabric for large-scale analytics and enrichment without separate data integration pipelines. Put enriched data back into applications. Enriched data from Fabric Data Warehouse flows back into HorizonDB synced tables, where applications and agents can access it with low-latency, high-concurrency performance. The result is a continuous loop. Application data flows into OneLake, analytics enrich it, and insights flow back to the applications and agents that act on them, all within Fabric. I'm especially excited to bring HorizonDB to both Azure and Microsoft Fabric, giving teams one PostgreSQL foundation for mission-critical apps and AI-native development. For the full story, read what’s new in HorizonDB. Modernize with a plan, accelerated by agents The applications that run your business hold your most valuable context, and modernizing them onto AI-ready foundations, with less migration friction, makes that context available to new intelligent experiences. Agents are changing how that work gets done, from discovery to code conversion. Every modernization starts with a plan. Azure Catalyst (formerly Azure Migrate), introduced at Ignite, brings IT teams and developers together around one plan for applications and databases, from discovery and assessment through migration and modernization. Agents help identify what's ready to move, where changes are needed and the right Azure destination for each workload, and GitHub Copilot modernization carries that plan into the code. “Transforming a legacy mainframe archive into a modern cloud solution is not just a technical upgrade; it's an opportunity to innovate, streamline, and future-proof your environment. Embrace the challenge, and you'll unlock new efficiencies and capabilities that were once unimaginable.” - Daniel Tanner, Stream Technical Lead, Client Document & Records Management (CDRM), UBS AI-assisted migrations from Oracle to managed PostgreSQL on Azure Some workloads ask more of that plan than others. For many organizations, Oracle workloads can be the hardest to move, because migrating the data is only half the job. The application code that depends on it must move, too. Two new previews address both halves. Oracle to PostgreSQL migration in Azure Database Migration Service now supports offline data migrations to Azure Database for PostgreSQL. Teams can configure connections, select schemas, and start and track the transfer in one experience, without building custom data-movement processes. For application code, AI-assisted Oracle application conversion provides a plug-in that converts Oracle code to work with Azure Database for PostgreSQL. Available through GitHub Copilot CLI or Visual Studio Code, and directly within existing modernization workflows such as Modernize CLI, the skills identify Oracle-specific dependencies and propose code changes for PostgreSQL compatibility. Developers review and validate the changes before accepting them. Together, they bring agentic assistance to the whole migration, not just the data. There's much more to the PostgreSQL story at Ignite – check it out. Run your apps where your data lives with DocumentDB Last year, we introduced Azure DocumentDB, our MongoDB-compatible document database powered by the open-source DocumentDB engine governed by the Linux Foundation. Now we’re expanding how organizations can deploy DocumentDB across hybrid and multicloud environments. DocumentDB enabled by Azure Arc and DocumentDB on Azure Local, both in preview, let organizations run MongoDB compatible workloads on supported Kubernetes clusters or customer-owned infrastructure, including on-premises and edge environments. Learn more about these new announcements here. Build intelligent apps faster Cloud-native applications need cloud-native data services designed for elastic scale, resilience, and security, with diverse data models and direct connections to analytics and AI. AI is raising the bar on all of it as apps and agents are built faster, grow less predictably and need context in real time. Our news this week meets developers where they build, from deploying complete apps in Fabric to scaling SQL, giving agents memory and serving AI experiences in at speed. Build and deploy intelligent apps with Fabric apps Fabric apps are now generally available, giving developers and coding agents a simpler path to building and deploying complete business applications. They build with Rayfin, the open-source SDK and CLI that provides databases, authentication, functions, storage, and observability, and deploy to Fabric, connected to enterprise data and governance. In just five months of preview, builders created more than 100,000 apps with Rayfin and deployed them on Fabric. Fabric apps connect to existing semantic models, warehouses, and lakehouses, so teams can build on trusted data without recreating data models or access controls. Teams can also turn analytics into applications that move users from insight to action. Content Security Policy settings and private sites add more protection. Builders can also start in familiar tools. With Microsoft Copilot, they can go from a simple prompt to a working application, using Fabric as a ready-made backend for databases, storage, and data APIs. Integrations with Power BI Pro and Power BI Desktop, both in preview, help teams turn trusted semantic models into interactive business applications. At Thomson Reuters, the Monthly Business Review now runs as a Fabric app, built on the same certified Power BI semantic models that ground the company's DeepVista AI framework. "This is a breakthrough in how Finance works. Certified semantic models give us numbers we can trust. Thomson Reuters DeepVista's AI agents let analysts and leaders ask questions in natural language and get insights they can act on. The monthly business hub solution automates the business review, so we spend our time on what to do next." Michael Goddard, Head of Corporate Finance, Thomson Reuters Fabric apps are just one part of a big week for Microsoft Fabric. Catch up on all the Fabric news from Ignite to see what else is new. Build locally, scale confidently with Azure SQL Database Hyperscale The SQL applications organizations trust most are ready for their next chapter: cloud-native scale, better economics, and AI. Azure SQL Database Hyperscale runs the SQL engine teams already know, on a cloud native distributed architecture that separates compute from storage, enabling applications to scale with demand. With relational, JSON, graph and vector data unified under familiar SQL security and governance, developers can build AI powered experiences grounded in real-time business data without a separate vector database. That makes Hyperscale the destination for modernizing SQL applications and the foundation for new AI applications. Great applications start on a laptop, not in a provisioning queue. Azure SQL Database container, now in public preview, brings the SQL engine to local development without requiring an Azure subscription. Developers get familiar T-SQL, system views, and driver behavior, along with vector support and AI_GENERATE_EMBEDDINGS to prototype AI-powered applications. When ready to deploy, the new local-to-cloud deployment wizard in the MSSQL extension for VS Code, also in public preview, provides a streamlined path from the developer's laptop to Azure SQL Database Hyperscale. Growing applications often split data across databases. Cross-database querying for Hyperscale, now in limited preview, lets developers use familiar T-SQL three-part names to query across databases in the same elastic pool from one connection, while each database enforces its own permissions. From laptop to cloud scale, it's the same SQL the whole way. Ready to start building? Read more about these and other Azure SQL announcements at Ignite to see how teams are modernizing and building what’s next on Hyperscale. Modernize and build AI apps that remember Azure Cosmos DB powers some of the world's most demanding AI applications, and more organizations want to bring their existing applications onto the same foundation. With Relational-to-CosmosDB modernization assistant, now in preview, London Stock Exchange Group (LSEG) modernized several legacy Oracle databases of record, including real-time pricing and instrument data, to Azure Cosmos DB, improving the latency and scale at which LSEG serves that data to traders. “Thanks to Azure Cosmos DB scale and performance, we have centered on Cosmos DB as the database strategy going forward for our database of records, the authoritative datasets used for downstream services and products.” Claire Armstrong, Head of Engineering Instrument & Quote, LSEG Data & Analytics For new applications, the AI-assisted data modeler in Azure Cosmos DB extension for Visual Studio Code, now in preview, helps developers get the data model right from the start. And as these workloads become business-critical, Azure Backup for Azure Cosmos DB, now generally available, adds immutable backups and long-term retention in a vault isolated from the source account, a key requirement in regulated industries. Modernizing the data layer matters even more as organizations put agents to work, because an agent is only as useful as what it remembers. That memory lives in the database, and Cosmos DB is a natural home for it. To keep that memory relevant as it grows, automatic embedding generation for Azure Cosmos DB, now in public preview, lets developers generate vector embeddings right where their data lives. We are excited to announce Microsoft.Embedding-1, a new multimodal embedding model with single- and multi-vector support for more granular matching, developed by the Cosmos DB team in partnership with Microsoft Research. The model is available via the Cosmos DB embedding generation service, and can be used with and without Cosmos DB. At the time of this publication, the Microsoft.Embedding-1 tops the ViDoRe embedding models leaderboard. And semantic reranker, now generally available, reorders retrieved results by meaning and query intent, sharpening relevance for search, RAG, and agents without custom ranking pipelines. Agent Memory Toolkit takes advantage of these core database capabilities to provide end-to-end serverless agent memory solution for facts, episodic, and procedural memories with built-in memory workflows for summation, reconciliation, and promotion. Explore these and other Azure Cosmos DB announcements at Ignite to see how teams are modernizing and building smarter AI apps. Relevant context is only half the equation. AI experiences also need to respond in real time. That's where Azure Managed Redis comes in. Teams are moving beyond traditional caching to use Redis as a real-time data layer for AI-driven applications, serving the session state, cached results and vector lookups that keep experiences fast. Redis 8 is now available in Azure Managed Redis, bringing the latest Redis innovation to Azure's fully managed service. With double the throughput, richer queries, atomic JSON updates, field-level expiration and fine-grained security, Redis 8 simplifies development and helps you build sophisticated applications faster. Scale operations with confidence As estates grow and as agents create more applications and more databases, operations can’t scale on human effort alone. The next step is bringing agents into the database lifecycle itself, with people defining goals, boundaries and trust, and agents providing the scale. Gain visibility and control across your database estate Operations start with knowing what you have. Database Hub, now generally available, brings your entire database estate together in one place, with unified visibility, observability, and centralized governance. It gives teams a single control plane for the Microsoft databases portfolio, and a place where people and agents work side by side to understand the estate, prioritize what matters and act. Manage your databases more intelligently with AI Database agent, now in preview, brings AI-assisted operations into the tools database professionals and developers already use. Database agent for SQL combines native in-database intelligence with AI-powered reasoning to continuously monitor operational signals across the SQL estate, proactively detects issues, explains their impact, and recommends actionable remediation. Database agent in VS Code for PostgreSQL brings AI-powered reasoning into the developer’s editor and recommends remediation steps or executes approved actions through Copilot agent mode. Rather than replacing human judgment, database agent works within your existing identity, permissions, approval boundaries, and audit trail. It helps teams move from observing databases to intelligently managing them, from a single database in VS Code to the full estate in Database Hub. For database teams, that means less time chasing alerts and more time on the work that moves the business forward. Learn more about Database Hub, database agent, and other exciting news. Build what’s next without leaving what matters behind Every intelligent experience starts with data. The opportunity in front of every organization is to modernize the applications that run the business while creating what comes next, on one connected foundation. The organizations that lead with AI won’t be the ones that start over. They’ll be the ones that turn the trusted data they already have into context their agents can act on. With Azure databases and Microsoft Fabric shipped together, that’s exactly what we’re building. To keep going: Explore the new Microsoft Databases roadmap site to see what’s coming next. Watch our Microsoft Ignite session, BRK362 Data Foundations for Modern Apps, AI, and Agents, on demand. Validate your skills with SQL AI Database Developer and Azure Cosmos DB AI Developer certifications.1KViews0likes0CommentsAzure SQL Hyperscale Cross-database querying: now in limited preview
Azure SQL Database Hyperscale now supports cross-database queries in limited preview. Use familiar [database].[schema].[object] names to query across databases in the same Hyperscale elastic pool from a single connection, with no external tables or extra network setup. It's built to simplify migrations of SQL Server apps that rely on cross-database queries, and to replace app-side joins and duplicated data. Sign up at aka.ms/azuresql-crossdb-prpr.350Views0likes0CommentsAsk your database estate a question: Copilot in Database Hub is now in public preview
Most database teams already collect a lot of monitoring data. What they don't have enough of is time to work out what it means. When something looks wrong, the first job is usually to open several portals, find the affected resources, and work out what changed. Database Hub in Microsoft Fabric already brings your Azure, Fabric, on-premises and multicloud databases into one view. Today we're making that view easier to use: Copilot in Database Hub is now in public preview. You can ask questions about your estate in plain language and get answers grounded in what Database Hub already knows about your databases. What you can do with Copilot in Database Hub Area What Copilot helps with Try asking Prioritize Rank the issues across your estate and show how many resources each one affects "What should I fix first?" Performance Find resources near storage, memory or compute capacity, and explain what changed "Are there any performance issues I should know about?" Troubleshooting Investigate slowdowns and blocking that affect your applications "Are there any blocked queries in my database causing delays in app response?" Security Find risky configurations, such as public network access or endpoints without private access "What are the biggest security risks in my environment?" Migration readiness Bring SQL Server assessment results into one answer "Analyze migration readiness across my SQL Server estate" Ask Copilot from wherever you're working Copilot is on every main page in Database Hub. Select Ask Copilot in the top right of the page you're on: Overview: you're looking at the issue counts and want to know what to fix first. Estate: you're going through resources and their issues and want to know which of this matter. Performance monitoring: you're watching metrics for Microsoft SQL, PostgreSQL or Cosmos DB and want to know why something changed. The Copilot pane opens beside the page, so you keep your place while you ask. Database Hub Overview page showing security, performance and optimization issues, with the Ask Copilot selected Ask Copilot button on the Estate and Performance monitoring pages in Database Hub. Get started with suggested questions Copilot opens with suggested questions grouped by the three things teams ask about most: Check performance: How are my databases performing? or Are there any performance issues I should know about? Review security: How secure are my databases?, What should I fix first to improve security? or What are the biggest security risks in my environment? Assess migration readiness: Show my existing SQL Server migration assessment results, or Analyze migration readiness across my SQL Server estate. Ask, follow up, then go deeper Here's a typical investigation: Ask a broad question. "What should I fix first to improve security?" Read the ranked summary. Copilot lists the top findings across your estate, such as endpoints without private access or public network access left on. It shows how many resources each one affects, so the biggest risks come first. Narrow it down. "Which of those are production databases?" or "Show me the resources with public network access enabled." Open the evidence. Go from Copilot's answer to the affected resources in Estate. Each resource lists its issues, such as Public network access is enabled or Memory is near capacity. Then act with the tools and processes you already use, including opening the resource in the Azure portal, SSMS or VS Code. Troubleshoot what's slowing your apps down When users report that an app is slow, the hard part is finding which database is responsible and why. Open Copilot from Performance monitoring and start with the symptom: "Are there any blocked queries in my database causing delays in app response?" "Why did CPU spike on this database in the last hour?" "Which databases are close to their storage, memory or compute limits?" "What changed on this database since yesterday?" Copilot summarizes what it finds and points you to the affected resource. Switch to the Performance monitoring charts to see the trend for yourself before you change anything. Plan your SQL Server migrations For SQL Server teams planning a move, "Analyze migration readiness across my SQL Server estate" collects assessment results into one answer, instead of you checking one server at a time. Follow up with "Which servers have blockers?" to see where to start. How Copilot works Copilot in Database Hub answers from the information Database Hub already has about your estate: your resource inventory, the security, performance and optimization issues it detects, and the performance telemetry it collects. It uses that context to rank, summarize and explain, then links you back to the resource or chart so you can check the evidence. Copilot sees only the resources your own Azure and Fabric permissions allow. It doesn't give you access to anything you couldn't already see. Tips for better answers Start broad, then narrow. Ask "What should I fix first?" and then filter by environment, service or issue type. Name what you care about. A resource name, a time window or a symptom gives you a more specific answer than a general question does. Ask from the right page. Ask about trends from Performance monitoring and about resources and issues from Estate. Check before you act. Use the linked resources and charts to confirm what Copilot found. Built with clear boundaries Grounded in Database Hub. Copilot answers from the information available in Database Hub. Your permissions, nothing more. Copilot works within your existing Azure and Fabric permissions. You stay in control. Copilot helps you understand and prioritize. It doesn't change your databases. Responsible AI. Copilot is powered by AI, so mistakes are possible. Review its answers before you act on them in production. Get started Opt in: Turn on Copilot for your organization. Copilot in Database Hub is an opt-in experience. Nothing is turned on for your users until a Fabric administrator chooses to enable it. In the Fabric admin portal, under Tenant settings > Copilot and AI, enable these settings, and make sure the enabled scope includes the users who need Copilot: Users can use Copilot, AI Agents and other AI experiences powered by Azure OpenAI Users can use Copilot, AI Agents and other AI experiences powered by OpenAI as a Microsoft Subprocessor Data sent to OpenAI as a Microsoft Subprocessor can be processed outside your capacity's geographic region, compliance boundary, or national cloud instance You can turn each setting on for the whole organization or only for specific security groups, so you can start with a pilot group. Important: Review the OpenAI as a Microsoft Subprocessor settings with your security and compliance team before you enable them. See OpenAI as a subprocessor in Microsoft Fabric for data processing commitments and current limitations. Open Database Hub in Microsoft Fabric. Select Ask Copilot on Overview, Estate or Performance monitoring, and start with a suggested question. Frequently asked questions Is Copilot on by default? No. Copilot in Database Hub is opt-in. A Fabric administrator decides whether to enable it and for whom. Which databases does Copilot cover? The same resources Database Hub shows you: Azure SQL, Fabric, on-premises and multicloud databases connected to your hub, within your permissions. Does Copilot need a specific capacity, and how is it billed? Copilot in Fabric needs a paid Fabric capacity (F2 or higher) or a Power BI Premium capacity (P1 or higher). It doesn't need a separate per-user license. Copilot usage is measured in tokens and consumes your Fabric capacity units (CUs). You can track it in the Fabric Capacity Metrics app under the operation name Copilot in Fabric. For rates and examples, see Copilot consumption, usage, and billing in Fabric. To bill all Copilot usage for a group of users to one capacity, see Fabric Copilot capacity. Does Copilot change my databases? No. Copilot helps you understand and prioritize. You make changes with your existing tools and processes. Is Copilot available in government or sovereign clouds? Not currently. OpenAI-operated models aren't available in government clouds (GCC, GCC High, DoD) or sovereign clouds. Why don't I see Ask Copilot? Check with your Fabric administrator that the tenant settings above are enabled and that their scope includes you. Learn more Database Hub overview Overview of Copilot in Fabric Enable and configure Copilot in Microsoft Fabric Copilot and Agent tenant settings OpenAI as a subprocessor in Microsoft Fabric Copilot consumption, usage, and billing in Fabric Fabric Copilot capacity What's next This preview is the start. We'll keep growing what Copilot can answer, based on what you ask it. Tell us what you ask first, and what you wish it could answer: select Feedback in Database Hub.308Views1like0CommentsClosing a monitoring blind spot: version store read metrics
When we asked two popular LLMs to "rank top RDBMS on the breadth and depth of the engine monitoring data they provide", the SQL Server database engine was ranked first by one and second by the other. With hundreds of DMVs and catalog views, thousands of extended events, query plan XML, and Query Store, you or your agent can diagnose and fix nearly any performance problem. Yet sometimes we find a blind spot. A mysterious CPU increase Imagine a busy OLTP database happily humming at 30 percent CPU as it does on any given day. Then, CPU starts climbing up without any increase in the workload. At 60 percent, it sets off an alarm. You start looking for the most common culprit - a query plan regression - but find none. Query plans are unchanged, but some queries just consume more and more CPU and run slower and slower. At this point you might capture an XPerf trace and look at the top CPU consuming call stacks. This is a powerful technique that finds the culprit in this case, but it's nowhere near as quick and simple as querying a DMV. You need access to the OS to capture the trace, you need to install the Windows Performance Toolkit, and you need to download symbols. Often that's just not an option. The culprit in this case: the version store, or rather row versions that had to be kept around for much longer than usual. Row versions are needed only while some transaction might still read them. After that, a background process cleans them up. Several things can hold that cleanup back, most commonly a long-running transaction, often an idle session that started but never committed a transaction. While cleanup is blocked, things get steadily worse: Version chains get longer. Every update to a frequently modified row adds another version, and queries that need an older version of the row must walk back through more and more of them. Until now, there was no practical way to detect this without an XPerf trace or a debugger. Versions move off-row. A row can hold only one in-row version. While cleanup is blocked, further updates to a row must store their versions off-row instead, which is more expensive both to write and to read. For frequently updated rows, nearly all new versions end up off-row. Until now, you couldn't correlate a slowdown in query execution with this shift from in-row to off-row versions. The net effect is that CPU usage and logical reads creep up across the whole workload with no single query or plan standing out. This matters because row versioning is the foundation for the RCSI and snapshot isolation levels, which avoid concurrency issues due to lock blocking. Accelerated database recovery (ADR) also uses row versioning to provide fast recovery, instant rollback, and to keep the transaction log small. Both RCSI and ADR are enabled by default in Azure SQL Database, and we recommend the same configuration for most SQL Server workloads. What was missing until now is a simple way to detect when version reads start to slow you down. Shining a light on the problem In Azure SQL Database, we added version store read metrics in SET STATISTICS IO, DMVs, extended events, and query plan XML, covering both the older version store in tempdb and the newer persistent version store (PVS) in each user database. Tempdb version store: versions created for databases without ADR. In-row PVS: small versions that are stored inside the data row itself. Off-row PVS: versions that are stored in a special-purpose system table in the user database. Online index PVS: versions that are stored in a different system table while an online index operation runs. An index operation might require these versions for as long as it runs. Keeping them apart lets the regular PVS be cleaned up in the meantime. The new metrics are now exposed as follows: SET STATISTICS IO. When an object had version reads, a new line follows the familiar IO lines: Object '[dbo].[t]'. Tempdb versions read 0, in-row PVS versions read 0, off-row PVS versions read 1000, online index PVS versions read 0. Query plan XML. In a plan with runtime statistics, the RunTimeCountersPerThread element of each operator that reads a table or index has four new attributes: ActualTempdbVersionsRead, ActualInRowPvsVersionsRead, ActualOffRowPvsVersionsRead, and ActualOnlineIndexPvsVersionsRead. This shows which operator in the plan read row versions, what kind, and how many. Extended events. The sql_statement_completed, sp_statement_completed, sql_batch_completed, and rpc_completed events have four new fields each: tempdb_versions_read, inrow_pvs_versions_read, offrow_pvs_versions_read, and online_index_pvs_versions_read. Query and module statistics. sys.dm_exec_query_stats, sys.dm_exec_procedure_stats, sys.dm_exec_trigger_stats, and sys.dm_exec_function_stats have new total_version_reads, last_version_reads, min_version_reads, and max_version_reads columns. These columns report the total, last, minimum, and maximum of the sum of all four counters. You can rank cached queries and modules by version reads and see the correlation with the CPU (worker) time, elapsed time, and logical reads. Sessions and requests. sys.dm_exec_requests and sys.dm_exec_sessions have a new version_reads column, also combining the four counters. Use the first to see how many version reads a request has done so far. Use the second to see the total for all completed requests during the session's lifetime. When version cleanup is blocked, version reads increase along with CPU time and logical reads across the workload, and you might also see a shift from in-row to off-row versions in PVS. That steers your investigation to the version store rather than to query plans or the workload itself. From there, the Monitor and troubleshoot accelerated database recovery article describes how to find and remove whatever is blocking cleanup. To learn more, see Version read metrics in documentation. Is this about CPU or IO? You might wonder - if the cost of version traversal is higher CPU, why expose it under STATISTICS IO? Indeed, there was considerable debate: should we add a new SET STATISTICS command (STATISTICS VERSION_CHAIN perhaps) specifically for these metrics? In the end, we settled on STATISTICS IO for three reasons: We didn't want you to memorize and type an extra command. We wanted to make the new statistics easy to discover. Logical reads in STATISTICS IO are already about CPU work (buffer pool access, not disk IO). Version reads are the same kind of measure, only for rows instead of pages. Keeping them together is consistent. Could we expose the IO cost of version reads instead, counting the pages read from a version store? Sure, but it would still leave a large blind spot: in-row versions. There is no extra IO cost for reading them because they are stored with the data row itself, and brought into memory when the data page is read. Yet traversing in-row versions and reconstructing older versions from in-row diffs adds nontrivial CPU cost. That cost would remain hidden if we only exposed IO statistics for off-row version pages. Why did we make this improvement? Besides the obvious reason - removing the blind spot in the database engine monitoring capabilities - we wanted to remove the uncertainty about version read overhead. While it can't be zero by its nature, the concurrency and availability benefits of RCSI and ADR are enormous. Yet we sometimes see customers hesitate to enable these features in SQL Server because it's hard to monitor and quantify the overhead. We hope that this improvement takes that hesitation away :). What's next At this point, we anticipate two questions: What about Query Store? Why not expose version reads together with other query runtime statistics? This is under consideration. Do you need version reads as a new metric in Query Store? Please leave a comment and describe how this would help. What about SQL Server and Azure SQL Managed Instance? The new metrics will soon be available in Azure SQL Managed Instance with the always-up-to-date update policy. As for nearly all database engine improvements that first appear in Azure SQL Database, we plan to include this improvement in the next version of SQL Server (and in the corresponding versioned Azure SQL Managed Instance) once they arrive. Tell us what you think about this improvement in the comments. Are there other blind spots that make it harder to see what's going on in the database engine? Share what you run into in your daily work. Example Here's a quick way to see some of the new metrics. Connect to a database in Azure SQL Database, and execute as a single batch. Note that the graphical query plan in SSMS doesn't yet show the new metrics even though they are present in the actual plan XML. That will be available in a future version of SSMS. SET NOCOUNT ON; /* Create a table with 1000 rows. */ DROP TABLE IF EXISTS dbo.t; CREATE TABLE dbo.t (id int PRIMARY KEY, v int NOT NULL); INSERT INTO dbo.t (id, v) SELECT TOP (1000) ROW_NUMBER() OVER (ORDER BY (SELECT NULL)), 0 FROM sys.all_columns; /* Create versions in Persistent Version Store (PVS). With Accelerated Database Recovery (ADR), rollback is instant, but the row versions from an aborted transaction stay in place until background cleanup happens later. */ BEGIN TRAN; UPDATE dbo.t SET v = v + 1; ROLLBACK; /* Immediately, read the data from the table. This requires version reads because PVS cleanup hasn't completed yet, but the SELECT must see the data before the aborted transaction. Version read statistics are returned after IO statistics. Showplan XML contains four new attributes under RunTimeCountersPerThread. */ SET STATISTICS IO, XML ON; SELECT COUNT(*) AS cnt FROM dbo.t WHERE v = 0; SET STATISTICS IO, XML OFF; /* Version reads are also present in the current request statistics. The value is doubled because of a statistics update by the previous SELECT, which also reads the same row versions. */ SELECT version_reads FROM sys.dm_exec_requests WHERE session_id = @@SPID; /* Aggregate version reads are also available for queries with cached plans, such as the previous SELECT (cached after auto-parameterization). */ SELECT qs.execution_count, qs.total_version_reads, qs.last_version_reads, qs.min_version_reads, qs.max_version_reads FROM sys.dm_exec_query_stats AS qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) AS st WHERE st.text LIKE '%SELECT COUNT(*) \[cnt\] FROM \[dbo\].\[t\] WHERE%' ESCAPE '\';1.2KViews4likes4Comments