Generally Available: Microsoft Entra Server Principals and Server Roles for Azure SQL Database
The problem we're solving Previously, Microsoft Entra identities in Azure SQL Database could only be created as contained database users - principals scoped to a single database with no server-level presence. That meant: No granular server-level delegation. You couldn't assign a server role such as ##MS_ServerStateReader## (to query DMVs across databases) or ##MS_LoginManager## (to manage logins) to an Entra principal. Only the Entra admin or a SQL login could perform these server-scoped tasks. Per-database provisioning overhead. Each Entra principal had to be created separately as a contained database user in every database that required access, with no way to inherit server-scoped permissions. No centralized “disable” switch. Offboarding meant tracking down a contained database user in every database - there was no server-level login to disable. These gaps forced many teams to keep SQL authentication for administrative tasks, even when they wanted to go password-less with Entra. What changes with GA Microsoft Entra logins become first-class server principals in the logical master database, just like SQL logins. This capability has been in public preview on Azure SQL Database (and is already generally available on Azure SQL Managed Instance and SQL Server 2022+); with this release it reaches general availability on Azure SQL Database, unlocking three things for production use: 1. Server role assignment for Entra identities Azure SQL Database's seven fixed server-level roles can be assigned to Entra server principals(logins). These roles cover database connectivity, database management, definition and security-definition reads, login management, and server-state read/manage. This means you can give your monitoring service principal read-only DMV access across all databases (##MS_ServerStateReader##), delegate login management to a security team member (##MS_LoginManager##), or let a DevOps app create databases (##MS_DatabaseManager##). All without SQL auth, all with Entra identities. 2. Server-wide login model Instead of provisioning contained users independently in every database, you can create database users mapped to a server login (CREATE USER ... FROM LOGIN). These users inherit server-scoped permissions automatically. One login, many databases — managed from a single place. For the T-SQL syntax, see Create and utilize Microsoft Entra server logins. 3. Centralized logins enable/disable ALTER LOGIN [user@contoso.com] DISABLE - one command blocks that identity from connecting to every database on the server. No more hunting down per-database users during an offboarding or incident response. When you re-enable the login, access is restored everywhere. Note: ALTER LOGIN ... DISABLE applies only to login-based users, not contained database users. It blocks new connections only; existing sessions remain active until terminated with KILL if needed. For immediate effect, see cache propagation. Microsoft Entra group logins are not supported; see the server principals documentation for alternatives. What does this unlock for your organization Ability to go password-less. With server principals and roles now generally available, organizations can adopt Entra-only authentication without a remaining server-level functionality gap. Entra logins bring parity with SQL logins closer, making it practical to disable SQL authentication entirely and using Entra as the sole authentication path. Least-privilege administration. Server-level roles simplify permission management by enabling customers to delegate common management and monitoring responsibilities without requiring admin privileges, enabling adherence to least privilege and separation of duties at scale, while making administration across databases on the same logical server much easier. Server roles let you scope access precisely, previously, the only server-wide option for an Entra identity was the all-powerful Entra admin. Give your security auditors ##MS_SecurityDefinitionReader## role instead of 'db_owner'. Give your monitoring tools ##MS_ServerStateReader## instead of an over-privileged administrator role. Zero-touch DevOps. A service principal with ##MS_DatabaseManager## and ##MS_LoginManager## can automate database and user provisioning end-to-end. After the initial Entra admin bootstrap, no human needs to be in the loop for routine operations. Faster incident response. When a principal is compromised, disable the login at the server level. New connections are blocked across all databases immediately - without needing to know which databases the user had access to. To cut off active sessions immediately, flush the authentication caches and KILL existing sessions. Geo-replica support. Entra logins created on the primary server are automatically available on geo-replicas, with read-only access to replicated databases. Key things to know Bootstrap requirements. The Microsoft Entra admin must create the first Entra login. After that, any Entra principal with ALTER ANY LOGIN or ##MS_LoginManager## membership can create additional logins. Entra admin takes precedence. If a principal is both the Entra admin and has a login, the admin permissions win. The login permissions have no additional effect. Cache propagation. Role membership and permission changes take effect on the next connection. For immediate effect, clear the auth cache with DBCC FLUSHAUTHCACHE and DBCC FREESYSTEMCACHE('TokenAndPermUserStore'). EXECUTE AS LOGIN is not supported for Entra logins on Azure SQL Database (it is supported on Managed Instance). Get started Configure a Microsoft Entra admin on your logical server Create your first Entra login and assign server roles (step-by-step tutorial) Understand the server roles and their permissions Consider enabling Entra-only authentication to eliminate SQL auth entirely Ready to migrate from SQL Authentication? If you're looking to move your existing SQL logins to Entra, check out Securing Azure SQL Database with Microsoft Entra password-less authentication - migration guide. It walks through the end-to-end journey from SQL auth to Entra, including how to identify SQL login dependencies, convert them to Entra principals, and enable Entra-only mode. Learn more Microsoft Entra server principals (logins) - full reference: syntax, permissions, limitations. Azure SQL Database server roles - role descriptions, permission matrix, examples. Microsoft Entra authentication overview - how Entra auth works with Azure SQL. Manage logins and users - login lifecycle management.Lessons Learned #540:Bulk Insert Throughput in Azure SQL Hyperscale with Partitioned Heap Tables
In this lesson learned, I would like to share an interesting scenario working on a service request where our customer was running a high-volume data load process in Azure SQL Database Hyperscale. The workload was based on a common pattern: Recreate a staging table. Load a large number of rows using bulk insert. The bulk insert showed unstable execution times and became the main area to investigate. The process was loading a very large number of rows into an Azure SQL Database Hyperscale database. The process used a staging table that was initially loaded as a heap. The main concern was the inconsistent execution time during the load process. Why Manually Adding Data Files Was Not the Right Direction In Azure SQL Database Hyperscale, the storage architecture is different from a traditional SQL Server deployment. The data layout and storage management are handled internally by the service. Because of this architecture, manually creating or pre-allocating multiple data files is not the same tuning option that we may consider in SQL Server on-premises or SQL Server running on Azure Virtual Machines. For this reason, the troubleshooting focus moved from manual file layout configuration to the actual workload pattern, waits, concurrency, batch size, and staging table design. What We Observed During the bulk insert phase, waits such as PAGELATCH_EX were observed. Since the staging table was loaded as a heap and the clustered primary key was created only after the bulk insert completed, OPTIMIZE_FOR_SEQUENTIAL_KEY was not directly applicable to the bulk insert phase. This changed the direction of the investigation. Instead of focusing on last-page insert contention on an existing clustered index, the analysis moved toward heap insert behavior, allocation contention, concurrency, batch size, and whether a different staging table design could help. First Recommendation: Start with Low-Impact Changes Before changing the table design, the first recommendation was to test the least intrusive changes: Reduce the number of concurrent bulk insert sessions. Increase the batch size, for example from 10,000 rows to 50,000 or 100,000 rows. Test TABLOCK on the dedicated heap staging table. The goal was to avoid assuming that more concurrency would always reduce the total execution time. In some high-volume load scenarios, excessive concurrency may increase contention and make the process less stable. The Interesting Design Option: Partitioned Heap Staging Table One of the most interesting design options was to evaluate a partitioned heap staging table. The idea is simple: instead of loading all rows into a non-partitioned heap staging table, the staging table can be created on the same partition scheme used by the target table, using the same partitioning column. This does not mean that a partitioned heap will always be faster. However, it can be a useful design option when: The bulk load phase is affected by allocation or latch contention. Concurrent load processes can naturally distribute rows across different partition ranges. The staging table is used only as an intermediate structure.Lessons Learned The main lessons from this scenario were: In Azure SQL Database Hyperscale, manually managing multiple data files is not the right tuning direction. PAGELATCH_EX during heap loading may point to concurrency or allocation-related contention. Reducing concurrency can sometimes improve total throughput. Larger batch sizes may provide better results than many small batches. TABLOCK on a dedicated heap staging table is a low-impact test worth evaluating. A partitioned heap staging table can be a valid second-phase design option when the load can be distributed across partition ranges. The best approach is to test small, measurable changes before introducing architectural redesigns. Final Thoughts A partitioned heap staging table can be a powerful option, but only when it is tested carefully and when the workload pattern can benefit from partition distribution.Regex support for LOB types in T-SQL—available in Azure SQL & SQL Server 2025
At a glance — Native regular expression (regex) functions in T-SQL now accept varchar(max) and nvarchar(max) inputs of up to 2 MB across all seven regex functions, including the two table-valued functions (REGEXP_MATCHES and REGEXP_SPLIT_TO_TABLE). This capability ships in SQL Server 2025 CU5 and is already available in Azure SQL Database, SQL Database in Fabric and Azure SQL Managed Instance configured with the Always-up-to-date update policy. It will reach Managed Instances on the SQL Server 2025 update policy as part of the CU5 rollout. You no longer need to split log files, HTML documents, or large JSON payloads into 8,000-byte chunks just to run a pattern match. 1. Introduction Regular expressions have long been a cornerstone of modern data processing — used for validation, parsing, transformation, and extracting structured insights from unstructured text. With SQL Server 2025 and Azure SQL, regex is now a first-class T-SQL capability, removing the historical need to rely on SQLCLR functions or application-tier processing. While the initial release made native regex broadly available, large-object (LOB) inputs were not yet supported on every function. CU5 closes that gap. Under the hood, T-SQL regex implements POSIX Extended Regular Expression (ERE) semantics, augmented by a curated set of Perl-style features, and is powered by the RE2 engine. RE2 is a linear-time, non-backtracking implementation, which means it is not susceptible to catastrophic backtracking (a class of denial-of-service issue commonly known as ReDoS). That guarantee becomes far more important when the input is a 1.8 MB log blob than when it is an 8,000-byte string. Release timeline Milestone What shipped Ignite 2025 — General Availability Regex went GA in SQL Server 2025 and Azure SQL. LOB inputs were initially supported only on REGEXP_LIKE, REGEXP_COUNT, and REGEXP_INSTR. LOB support on REGEXP_REPLACE and REGEXP_SUBSTR was deferred, and the two table-valued functions (TVFs) accepted only non-LOB string types. Azure SQL (post-GA service updates) LOB inputs enabled across all seven functions. SQL Server 2025 CU5 LOB inputs up to 2 MB enabled on all seven functions in the SQL Server. What’s new in CU5 varchar(max) and nvarchar(max) inputs are accepted on every regex function. The input string is capped at 2 MB per function call. The pattern is still capped at 8,000 bytes, which is far larger than any maintainable regular expression should ever need. Behavior is consistent between Azure SQL and SQL Server, so code you write today is fully portable. Note — The 2 MB limit applies to the input passed to a single function call, not to the column or row. A single value in a varchar(max) column can still store up to 2 GB; the constraint is that no single regex evaluation can consume more than 2 MB of that value. Prerequisites SQL Server 2025 CU5 or later, or Azure SQL Database, or SQL Database in Fabric or Azure SQL Managed Instance configured with the SQL Server 2025 / Always-up-to-date update policy. The two table-valued functions (REGEXP_MATCHES and REGEXP_SPLIT_TO_TABLE) require database compatibility level 170, unless the database-scoped configuration ALLOW_BUILTIN_TVF_IN_ALL_COMPAT_LEVELS (preview) is enabled. Note — On Azure SQL Managed Instance (Always-up-to-date), this capability is rolling out region by region. It is already live in regions where the rollout has completed and will light up in the remaining regions as the deployment finishes. Instances on the SQL Server 2025 update policy will receive it as part of the CU5 rollout — coming soon. Verify compatibility level (170 required for the TVFs) – SELECT name, compatibility_level FROM sys.databases WHERE name = DB_NAME(); -- If necessary: -- ALTER DATABASE [<your-database>] SET COMPATIBILITY_LEVEL = 170; 2. Working with LOB Data This section demonstrates the CU5 capabilities against a realistic LOB data. We build a LogEntries table whose RawPayload column holds multi-KB to multi-MB chunks of web server and application output, plus an HtmlPages table for HTML cleansing examples. 2.1 Create the sample schema and data IF OBJECT_ID('dbo.LogEntries', 'U') IS NOT NULL DROP TABLE dbo.LogEntries; IF OBJECT_ID('dbo.HtmlPages', 'U') IS NOT NULL DROP TABLE dbo.HtmlPages; CREATE TABLE dbo.LogEntries ( LogId BIGINT IDENTITY(1,1) PRIMARY KEY, Source SYSNAME NOT NULL, IngestedAt DATETIME2(3) NOT NULL DEFAULT SYSUTCDATETIME(), RawPayload VARCHAR(MAX) NOT NULL -- LOB column ); CREATE TABLE dbo.HtmlPages ( PageId INT IDENTITY(1,1) PRIMARY KEY, Url NVARCHAR(2048) NOT NULL, Body NVARCHAR(MAX) NOT NULL -- LOB column (Unicode) ); Now generate realistically large rows. The REPLICATE(CAST(... AS varchar(max)), n) pattern is required because REPLICATE returns NULL when the result would exceed 8,000 bytes unless its first argument is a max type. -- Synthetic web access-log payload (~252 KB in row 1, plus a separate ~586 KB row). DECLARE @logLine VARCHAR(500) = '127.0.0.1 - alice [21/May/2026:10:15:32 +0000] "GET /api/orders/42 HTTP/1.1" 200 1532 ' + 'user-agent="Mozilla/5.0" ip=10.0.0.7 email=alice@contoso.com card=4111-1111-1111-1234' + CHAR(10); DECLARE @bigLog VARCHAR(MAX) = REPLICATE(CAST(@logLine AS VARCHAR(MAX)), 1500) -- ~252 KB + '127.0.0.1 - mallory [21/May/2026:10:16:01 +0000] "POST /login HTTP/1.1" 500 0 ' + 'ip=203.0.113.99 ssn=123-45-6789' + CHAR(10); INSERT INTO dbo.LogEntries (Source, RawPayload) VALUES ('web-01', @bigLog), -- ~252 KB ('web-02', REPLICATE(CAST('OK ' AS VARCHAR(MAX)), 200000)); -- ~586 KB -- Synthetic HTML page (~775 KB / ~396,000 characters). DECLARE @htmlChunk NVARCHAR(MAX) = N'<div class="row"><p>Hello <b>world</b>! Contact <a href="mailto:bob@contoso.com">bob</a>.</p></div>'; INSERT INTO dbo.HtmlPages (Url, Body) VALUES (N'https://contoso.example/page-1', N'<html><head><title>Big Page</title></head><body>' + REPLICATE(@htmlChunk, 4000) + N'</body></html>'); -- Confirm payload sizes in bytes. SELECT LogId, Source, DATALENGTH(RawPayload) AS PayloadBytes FROM dbo.LogEntries; SELECT PageId, DATALENGTH(Body) AS BodyBytes, LEN(Body) AS BodyChars FROM dbo.HtmlPages; Results: LogId Source PayloadBytes 1 web-01 258,110 2 web-02 600,000 PageId BodyBytes BodyChars 1 792,124 396,062 Before CU5, feeding any of these payloads into REGEXP_REPLACE, REGEXP_SUBSTR, REGEXP_MATCHES, or REGEXP_SPLIT_TO_TABLE would have failed with a type-mismatch error or required a LEFT(RawPayload, 8000)-style truncation. The same queries now run end-to-end. 2.2 REGEXP_LIKE — Filter rows by LOB content -- Find logs that contain at least one HTTP 5xx response. SELECT LogId, Source, DATALENGTH(RawPayload) AS PayloadBytes FROM dbo.LogEntries WHERE REGEXP_LIKE(RawPayload, '"[A-Z]+\s[^"]+\sHTTP/1\.[01]"\s5[0-9]{2}\s'); REGEXP_LIKE is a Boolean predicate: it evaluates to true when the pattern matches anywhere in the input and false otherwise. Because it returns a Boolean rather than a bit, use it directly in WHERE, CASE WHEN, IIF, or CHECK constraint contexts — do not compare it with = 1 or = 0 (the parser rejects that syntax). Note — REGEXP_LIKE itself requires database compatibility level 170. The other scalar regex functions (REGEXP_COUNT, REGEXP_INSTR, REGEXP_REPLACE, REGEXP_SUBSTR) are available at all compatibility levels. Results: LogId Source PayloadBytes 1 web-01 258,110 2.3 REGEXP_COUNT — Counting at scale -- Per-row tally of GET requests, POST requests, and 5xx responses -- across the entire LOB payload. SELECT LogId, Source, REGEXP_COUNT(RawPayload, '"GET\s') AS Gets, REGEXP_COUNT(RawPayload, '"POST\s') AS Posts, REGEXP_COUNT(RawPayload, '\s5[0-9]{2}\s') AS ServerErrors FROM dbo.LogEntries; Results: LogId Source Gets Posts ServerErrors 1 web-01 1,500 1 1 2 web-02 0 0 0 2.4 REGEXP_INSTR — Locate the first error -- 1-based character position (or 0 if no match) of the FIRST 5xx response in each payload. SELECT LogId, Source, REGEXP_INSTR(RawPayload, '\s5[0-9]{2}\s', 1, 1, 0) AS FirstErrorPos FROM dbo.LogEntries; Parameter recap: REGEXP_INSTR(string, pattern, start, occurrence, return_option [, flags [, group ]]). A return_option of 0 returns the starting position of the match; 1 returns the position immediately after the last character of the match. Results: LogId Source FirstErrorPos 1 web-01 258,072 2 web-02 0 2.5 REGEXP_REPLACE — Redact sensitive data in place PII redaction over LOB payloads was one of the most-requested CU5 scenarios. Before CU5, it required a custom chunked-replace routine; it is now a single expression. -- Redact credit-card-shaped tokens, U.S. SSN-shaped tokens, and email addresses -- across the entire payload. SELECT LogId, REGEXP_REPLACE( REGEXP_REPLACE( REGEXP_REPLACE( RawPayload, '\b[0-9]{4}[- ]?[0-9]{4}[- ]?[0-9]{4}[- ]?[0-9]{4}\b', '****-****-****-****'), '\b[0-9]{3}-[0-9]{2}-[0-9]{4}\b', '***-**-****'), '\b[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}\b', '[redacted-email]' ) AS RedactedPayload FROM dbo.LogEntries; Or strip every HTML tag from an nvarchar(max) page in a single call: SELECT PageId, LEN(Body) AS OriginalLen, LEN(REGEXP_REPLACE(Body, N'<[^>]+>', N'')) AS TextOnlyLen FROM dbo.HtmlPages; Results — the ~775 KB HTML document collapses from 396,062 to 100,008 characters of plain text in a single call: PageId OriginalLen TextOnlyLen 1 396,062 100,008 2.6 REGEXP_SUBSTR — Extract a single value -- Pull the first IPv4 address out of each log payload. SELECT LogId, REGEXP_SUBSTR(RawPayload, '\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b', 1, -- start position 1, -- occurrence 'c', -- flags: case-sensitive 0 -- group: 0 returns the whole match ) AS FirstIp FROM dbo.LogEntries; To return the contents of a specific capture group instead of the entire match, pass its 1-based group number as the final argument. Results: LogId FirstIp 1 127.0.0.1 2 NULL 2.7 REGEXP_MATCHES — Every match, set-based This is where the combination of TVF and LOB delivers the largest productivity gain: extract every structured value from a megabyte of unstructured text in a single set-based query, with no client round-trips. REGEXP_MATCHES returns one row per match with these columns: Column Type Description match_id bigint Sequence number of the match (1-based). start_position int 1-based start index of the match. end_position int 1-based end index of the match. match_value same type as string_expression The entire matched substring. substring_matches json JSON array describing each capture group, with the shape [{"value":"…","start":N,"length":N}, …]. -- Every email address in every log payload, alongside its row of origin. SELECT l.LogId, m.match_id, m.match_value AS EmailFound FROM dbo.LogEntries AS l CROSS APPLY REGEXP_MATCHES( l.RawPayload, '\b[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}\b' ) AS m ORDER BY l.LogId, m.match_id; Capture groups are even more useful — you can project the parts of every log line as columns by reading from the substring_matches JSON document: -- Parse Common-Log-Format-ish entries into ip, user, status, and bytes columns. -- The pattern has four capture groups, accessed below as $[0] through $[3]. SELECT l.LogId, m.match_id, JSON_VALUE(m.substring_matches, '$[0].value') AS Ip, JSON_VALUE(m.substring_matches, '$[1].value') AS UserName, JSON_VALUE(m.substring_matches, '$[2].value') AS Status, JSON_VALUE(m.substring_matches, '$[3].value') AS Bytes FROM dbo.LogEntries AS l CROSS APPLY REGEXP_MATCHES( l.RawPayload, '^([0-9.]+)\s-\s(\S+)\s\[[^\]]+\]\s"[^"]+"\s([0-9]{3})\s([0-9]+)', 'm' -- multi-line: ^ and $ anchor to each line, not just the whole input ) AS m ORDER BY l.LogId, m.match_id; Important — Without the 'm' flag, the ^ anchor matches only at the start of the entire 250 KB input, so you would receive exactly one match for the first line. The multi-line flag is what unlocks per-line extraction. Results (first two parsed rows): LogId match_id Ip UserName Status Bytes 1 1 127.0.0.1 alice 200 1532 1 2 127.0.0.1 alice 200 1532 2.8 REGEXP_SPLIT_TO_TABLE — Shred a LOB into rows -- Project the entire log payload as one row per non-empty line. SELECT l.LogId, s.ordinal AS [LineNo], s.value AS LineText FROM dbo.LogEntries AS l CROSS APPLY REGEXP_SPLIT_TO_TABLE(l.RawPayload, '\r?\n') AS s WHERE l.LogId = 1 AND s.value <> '' ORDER BY s.ordinal; You now have a tabular projection of a multi-megabyte text blob without leaving the engine. You can feed it into a CTE, aggregate it, join it to dimension tables, or materialize it into a staging table — all set-based. Results (first three rows): LogId ordinal LineText (first 80 chars) 1 1 127.0.0.1 - alice [21/May/2026:10:15:32 +0000] "GET /api/orders/42 HTTP/1.1" 200 1 2 127.0.0.1 - alice [21/May/2026:10:15:32 +0000] "GET /api/orders/42 HTTP/1.1" 200 1 3 127.0.0.1 - alice [21/May/2026:10:15:32 +0000] "GET /api/orders/42 HTTP/1.1" 200 Tip — composing LOB regex pipelines — CROSS APPLY (and OUTER APPLY when you need to preserve rows that produce no matches) is the primary composition primitive. You can stack REGEXP_SPLIT_TO_TABLE (lines) feeding REGEXP_MATCHES (fields per line) feeding ordinary aggregates, all within a single query plan. 2.9 The 2 MB ceiling — strategies for larger inputs The 2 MB limit applies to the input string of a single regex call. If the value passed to a regex function exceeds 2 MB, the call raises an error (error number 19311, severity 16) rather than silently truncating. That is the intended behavior — silent truncation would hide correctness bugs. In practice, 2 MB is a generous ceiling: a single log file or HTML document of that size is already unusual, and most real-world LOB data sit comfortably below it. When individual values do exceed the limit, the most reliable approach is to split them into smaller logical units before they land in the column you want to query — for example, by writing one log line, one document section, or one record per row at ingestion time. Because every regex function (including the two TVFs) shares the same 2 MB ceiling, sharding at query time is not generally feasible; doing it at the load path keeps every regex call well under the limit and avoids per-query workarounds. Bytes vs. characters — The 2 MB limit is measured in bytes, not characters, and the byte count is based on the UTF-8 encoding of the input regardless of the column’s declared type. ASCII characters take 1 byte each, so plain ASCII text can run to roughly two million characters; non-ASCII characters take 2–4 bytes in UTF-8, so fewer characters fit. Keep in mind that DATALENGTH() reports storage size in the column’s own encoding, which may differ from the UTF-8 byte count used by the limit, and LEN() (which counts characters) is best avoided as a sizing check here. To measure the UTF-8 byte length that the limit actually checks, cast the value to varchar(max) under a UTF-8 collation and take its DATALENGTH: SELECT DATALENGTH( CONVERT(varchar(max), Body COLLATE Latin1_General_100_CI_AS_SC_UTF8) ) AS Utf8Bytes FROM dbo.HtmlPages; Anything above 2 * 1024 * 1024 (2,097,152) bytes will be rejected by a regex call on that value. Have a scenario that genuinely needs more than 2 MB? If your workload requires regex evaluation on individual values larger than the current 2 MB ceiling, we would like to hear about it. Please share the details — data shape, payload size, pattern, and business need — on the Azure SQL feedback portal. Customer feedback directly informs how we prioritize future limit changes. 2.10 Cleanup DROP TABLE IF EXISTS dbo.LogEntries; DROP TABLE IF EXISTS dbo.HtmlPages; 3. Summary What changed in CU5 Before CU5 — LOB inputs were accepted on REGEXP_LIKE, REGEXP_COUNT, and REGEXP_INSTR. The remaining functions — REGEXP_REPLACE, REGEXP_SUBSTR, and the two TVFs (REGEXP_MATCHES, REGEXP_SPLIT_TO_TABLE) — required non-LOB string inputs, which often meant truncating with LEFT(..., 8000) or chunking in the application tier. After CU5 (and already in Azure SQL) — All seven functions accept varchar(max) and nvarchar(max) inputs of up to 2 MB. The pattern remains capped at 8,000 bytes. Quick reference Function Returns LOB input (CU5) Common use case REGEXP_LIKE Boolean (predicate) Yes Filter rows in WHERE / CASE / CHECK predicates REGEXP_COUNT int Yes Count occurrences of a pattern REGEXP_INSTR int Yes Position of the nth match REGEXP_REPLACE string Yes Redact, cleanse, or normalize text REGEXP_SUBSTR string Yes Extract a single value REGEXP_MATCHES (TVF) (match_id, start_position, end_position, match_value, substring_matches) Yes Extract every match plus capture groups (via JSON), set-based REGEXP_SPLIT_TO_TABLE (TVF) (value, ordinal) Yes Split a LOB into rows by a regex delimiter Further reading Official documentation: REGEXP_LIKE, REGEXP_COUNT, REGEXP_INSTR, REGEXP_REPLACE, REGEXP_SUBSTR, REGEXP_MATCHES, REGEXP_SPLIT_TO_TABLE. Regular expressions overview. SQL Server 2025 CU5 release notes. Closing thought. Native regex was already a significant quality-of-life improvement when it became generally available. CU5 completes the picture: every function, every input size up to 2 MB, every shape — scalar or table-valued. The next time you are tempted to export a column out of the database in order to grep it, try one of the seven regex functions first. Happy matching. 🧠Public Preview - Data Virtualization for Azure SQL Database
Data virtualization, now in public preview in Azure SQL Database, enables you to leverage all the power of Transact-SQL (T-SQL) and seamlessly query external data from Azure Data Lake Storage Gen2 or Azure Blob Storage, eliminating the need for data duplication, or ETL processes, allowing for faster analysis and insights. Integrate external data, such as CSV, Parquet, or Delta files, with your relational database while maintaining the original data format and avoiding unnecessary data movement. Present integrated data to applications and reports as a standard SQL object or through a normal SELECT command. Data Virtualization for Azure SQL Database supports SAS tokens, Managed Identity, and User identity for secure access. Data Virtualization for Azure SQL Database will introduce and expand support for: Database Scoped Credential. External Data Source. External File Format - with support for Parquet, CSV, and Delta. External Tables. OPENROWSET. Support metadata functions and JSON functions. For enhanced security and flexibility Data Virtualization for Azure SQL Database supports three authentication methods: Shared access signature. Managed identity (system assigned managed identity and user-assigned managed identity). User identity. Key Benefits Just like in SQL Server 2022 and Azure SQL Managed Instance the key benefits of Data Virtualization for Azure SQL Database are: Seamless Data Access: Query external CSV, Parquet, and Delta Lake tables using T-SQL as if they were native tables within Azure SQL Database. Allowing for off-loading cold data while keeping it easily accessible. Enhanced Productivity: Reduce the time and effort required to integrate and analyze data from multiple sources. Cost Efficiency: Minimize the need for data replication and storage costs associated with traditional data integration methods. Real-Time Insights: Enable real-time data querying and insights without delays caused by data movement or synchronization. Security: Leverage SQL Server security features for granular permissions, credential management, and control. Example Data Virtualization for Azure SQL Database is based on the same core principles as SQL Server’s PolyBase feature. With support for Azure Data Lake Gen 2, using prefix adls:// and Azure Blob Storage, using prefix abs://. For the following example we are going to use Azure Open Datasets, more specifically NYC yellow taxi trip records open data set which allows public access. For private data sources customers can leverage multiple authentication methods like SAS Tokens, Managed Identity and User Identity. -- Create Azure Blob Storage (ABS) data source CREATE EXTERNAL DATA SOURCE NYCTaxiExternalDataSource WITH ( LOCATION = 'abs://nyctlc@azureopendatastorage.blob.core.windows.net'); -- Using OPENROWSET to read Parquet files from the external data source SELECT TOP 10 * FROM OPENROWSET( BULK '/yellow/puYear=*/puMonth=*/*.parquet', DATA_SOURCE = 'NYCTaxiExternalDataSource', FORMAT = 'parquet' ) AS filerows; -- Or using External Tables CREATE EXTERNAL FILE FORMAT DemoFileFormat WITH (FORMAT_TYPE = PARQUET); --Create external table CREATE EXTERNAL TABLE tbl_TaxiRides ( vendorID VARCHAR(100) COLLATE Latin1_General_BIN2, tpepPickupDateTime DATETIME2, tpepDropoffDateTime DATETIME2, passengerCount INT, tripDistance FLOAT, puLocationId VARCHAR(8000), doLocationId VARCHAR(8000), startLon FLOAT, startLat FLOAT, endLon FLOAT, endLat FLOAT, rateCodeId SMALLINT, storeAndFwdFlag VARCHAR(8000), paymentType VARCHAR(8000), fareAmount FLOAT, extra FLOAT, mtaTax FLOAT, improvementSurcharge VARCHAR(8000), tipAmount FLOAT, tollsAmount FLOAT, totalAmount FLOAT ) WITH ( LOCATION = 'yellow/puYear=*/puMonth=*/*.parquet', DATA_SOURCE = NYCTaxiExternalDataSource, FILE_FORMAT = DemoFileFormat ); SELECT TOP 10 * FROM tbl_TaxiRides; You can also use these capabilities in combination with other metadata functions like sp_describe_first_result_set, filename(), and filepath(). Getting started Data Virtualization for Azure SQL Database is currently available in select regions, with broader availability coming soon across all Azure regions Data Virtualization for Azure SQL Database is based on the same core principles as SQL Server’s PolyBase feature. To know more and get started with Data Virtualization for Azure SQL Database.Unexpected PITR Charges from restorableDroppedDatabases After BC → Hyperscale Migration
Why This Behavior Is by Design When migrating an Azure SQL Database from Business Critical (BC) to Hyperscale using a manual cutover, some customers notice unexpected Point-in-Time Restore (PITR) backup storage charges appearing under the following resource: /Microsoft.Sql/servers/<server>/restorableDroppedDatabases/<database> At first glance, this can be confusing—especially when: No customer-initiated drop or delete was performed The database is online and healthy post-migration Test migrations may not have shown similar charges This post explains why this happens, why it is expected by design, and how these charges naturally expire. The Observed Scenario After a BC → Hyperscale manual cutover, customers may see PITR charges tied to: restorableDroppedDatabases/<database-name> Despite the database being active and available in Hyperscale, these charges start appearing immediately after the migration cutover and gradually decrease over time. Why Does the Database Appear as “Dropped”? During a manual cutover migration, Azure SQL performs an internal platform-driven workflow to complete the transition between architectures. From a control-plane perspective: The source Business Critical logical database is internally dropped This drop is not initiated by the customer It is a required system step to complete the Hyperscale migration Telemetry confirms that the migration workflow transitions through states such as: Internal drop of the source physical and logical database Cleanup of metadata and completion of the migration This entire sequence completes within seconds and is fully platform managed. Why Are Backup Charges Generated? Although the source BC database is internally dropped, its pre-migration PITR backups are still retained according to the configured backup retention period. Here’s the key point: Backups taken before upgrading to Hyperscale are retained and billed using the dropped-database backup billing model. Because the source database is now considered dropped (from the BC perspective): The 1× database-size discount no longer applies The full data file size is added to the billable backup size Charges appear under restorableDroppedDatabases This behavior is explicitly documented as expected in internal Azure SQL billing guidance. Why Do Charges Decrease Over Time? These charges are not permanent. They: Decrease daily Continue only while the pre-migration PITR backups are retained Automatically stop once the retention window expires In practical terms: Charges stop when: days_since_migration > configured_backup_retention_days No cleanup action is required from the customer—the platform handles this automatically. Why Didn’t Test Migrations Show Similar Charges? In many reported cases, test or smaller databases migrated using the same method did not generate noticeable charges. This can be explained by two documented optimizations: Backup size threshold – very small backup footprints are not charged Low activity optimization – inactive or low-change databases generate fewer snapshots As a result, smaller or lightly used test databases may fall below the billing threshold, while larger production databases do not. Is This a Billing Error or Credit Scenario? No. Although the operation is platform-driven: The behavior is by design The charges are for temporary retention of valid PITR backups They naturally expire based on retention Therefore, this scenario is not considered a billing defect and does not typically warrant credits. How Can Customers Reduce Charges Faster? If needed, customers can: Reduce the PITR backup retention period (minimum is 1 day) Wait up to 24 hours for billing to reflect the change This shortens how long the pre-migration backups are retained and billed. FAQ – restorableDroppedDatabases Charges After BC → Hyperscale Migration Q1: Why am I seeing PITR charges for restorableDroppedDatabases when my database is still online? A: During a Business Critical → Hyperscale manual cutover, Azure SQL internally drops the source BC database as part of the migration workflow. While the Hyperscale database is active and healthy, the pre‑migration BC backups are retained and billed under restorableDroppedDatabases. Q2: Did the customer initiate a drop or delete operation? A: No. This drop is platform‑driven and required to complete the migration. It is not initiated by the customer. Q3: What exactly is being billed? A: The charges are for Point‑in‑Time Restore (PITR) backups taken before the migration. These backups are retained according to the configured backup retention period and are billed using the dropped database billing model. Q4: Why does the cost appear higher than expected? A: Once a database is considered “dropped” (from the BC perspective), the 1× database-size discount no longer applies, and the full data file size is included in the billable backup size. Q5: Will these charges continue indefinitely? A: No. The charges decrease daily and automatically stop once the pre‑migration backups expire based on the configured PITR retention period. Q6: Why didn’t this happen with smaller or test databases? A: Smaller or low‑activity databases may fall below the backup billing threshold, or benefit from low‑activity snapshot optimizations, resulting in no visible charges. Q7: Is this a billing bug or credit-worthy scenario? A: No. This behavior is by design and expected. The charges reflect valid backup retention and do not typically qualify for credits. Q8: Can the customer reduce these charges sooner? A: Yes. The customer can reduce the PITR backup retention period (minimum 1 day). Billing changes usually reflect within up to 24 hours. Key Takeaways The behavior is expected and by design Charges come from pre-migration BC backups, not the active Hyperscale database The database was internally dropped as part of migration, not by the customer Charges decrease daily and stop automatically No action is required unless the customer wants to reduce retention early Final Note As of the time of writing, this behavior is not clearly described in public customer-facing documentation, which explains why it often appears unexpected. Awareness of this mechanism can help set correct expectations when planning BC → Hyperscale manual cutover migrations.Azure SQL Hyperscale: Understanding PITR Retention vs Azure Portal Restore UI
Overview Customers using Azure SQL Database – Hyperscale may sometimes notice a discrepancy between the configured Point-in-Time Restore (PITR) retention period and what the Azure Portal displays as available restore points. In some cases: PITR retention is configured (for example, 7 days), Yet the Azure Portal only shows restore points going back a shorter period (for example, 1–2 days), And the restore UI may allow selecting dates earlier than the configured retention window without immediately showing an error. This post explains why this happens, how to validate backup health, and what actions to take. Key Observation From investigation and internal validation, this behavior is not indicative of backup data loss. Instead, it is related to Azure Portal UI behavior, particularly for Hyperscale databases. The backups themselves continue to exist and are managed correctly by the service. Important Distinction: Portal UI vs Actual Backup State What the Azure Portal Shows The restore blade may show fewer restore points than expected. The date picker may allow selecting dates outside the PITR retention window. No immediate validation error may appear in the UI. What Actually Happens Backup retention is enforced at the service layer, not the portal. If a restore is attempted outside the valid PITR window, the operation will fail during execution, even if the UI allows selection. Hyperscale backup metadata is handled differently than General Purpose or Business Critical tiers. Why This Happens with Hyperscale There are a few important technical reasons: Hyperscale backup architecture differs Hyperscale uses a distributed storage and backup model optimized for scale and fast restore, which affects how metadata is surfaced. Some DMVs are not supported Views like sys.dm_database_backups, commonly used for backup visibility, do not support Hyperscale databases. Azure Portal relies on metadata projections The portal restore experience depends on backend projections that may lag or behave differently for Hyperscale, leading to UI inconsistencies. How to Validate Backup Health (Recommended) Instead of relying solely on the Azure Portal UI, use service-backed validation methods. Option 1: PowerShell – Earliest Restore Point You can confirm the earliest available restore point directly from the service: # Set your variables $resourceGroupName = "RG-xxx-xxx-1" $serverName = "sql-xxx-xxx-01" $databaseName = "database_Prod" # Get earliest restore point $db = Get-AzSqlDatabase -ResourceGroupName $resourceGroupName -ServerName $serverName -DatabaseName $databaseName $earliestRestore = $db.EarliestRestoreDate Write-Host "Earliest Restore Point: $earliestRestore" Write-Host "Days Available: $([math]::Round(((Get-Date) - $earliestRestore).TotalDays, 1)) days" This reflects the true PITR boundary enforced by Azure SQL. Option 2: Internal Telemetry / Backup Events (Engineering Validation) Internal monitoring confirms: Continuous backup events are present. Coverage aligns with configured PITR retention. Backup health remains ✅ Healthy even when the portal UI appears inconsistent. Key takeaway: Backup data is intact and retention is honored. Is There Any Risk of Data Loss? No. There is no evidence of backup loss or retention policy violation. This is a visual/UX issue, not a data protection issue. Recommended Actions For Customers ✅ Trust the configured PITR retention, not just the portal display. ✅ Use PowerShell or Azure CLI to validate restore boundaries. ❌ Do not assume backup loss based on portal UI alone. For Support / Engineering Capture a browser network trace when encountering UI inconsistencies. Raise an incident with the Azure Portal team for investigation and fix. Reference Hyperscale-specific behavior during troubleshooting. Summary Topic Status PITR retention enforcement ✅ Correct Backup data integrity ✅ Safe Azure Portal restore UI ⚠️ May be misleading Hyperscale backup visibility ✅ Validate via service tools Final Thoughts Azure SQL Hyperscale continues to provide robust, reliable backup and restore capabilities, even when the Azure Portal UI does not fully reflect the underlying state. When in doubt: Validate via service APIs Rely on enforcement logic, not UI hints Escalate portal inconsistencies appropriatelyAnnouncing Preview of 160 and 192vCore Premium-series Options for Azure SQL Database Hyperscale
We are excited to announce the public preview of 160 and 192vCore compute sizes for Premium-series hardware configuration in Azure SQL Database Hyperscale. Since the introduction of Premium-series hardware configurations for Hyperscale in November 2022, many customers have successfully used larger vCore configurations to consolidate workloads, reduce shard counts, and improve overall application performance and stability. This preview builds on the Premium-series configuration introduced previously for Hyperscale, extending the maximum scale of a single database and elastic pools from 128vCores to 192vCores to support higher concurrency, faster CPU performance, and larger memory footprints, for more demanding mission critical workloads. With this preview, customers running largescale OLTP, HTAP, and analytics-heavy workloads can evaluate even higher compute ceilings without rearchitecting their applications. Premium-Series Hyperscale Hardware Overview Premium-series Hyperscale databases run on latest-generation Intel and AMD processors , delivering higher per core performance and improved scalability compared to standard-series (Gen5) hardware. With this public preview release, Premium-series Hyperscale now supports larger vCore configurations, extending the scaleup limits for customers who need more compute and memory in a single database. Getting started Customers can enable the 160 or 192vCore Premium-series options when creating a database, or when scaling up existing Hyperscale databases in supported regions (where preview capacity is available). As with other Hyperscale scale operations, moving to a larger vCore size does not require application changes and uses Hyperscale’s distributed storage and compute architecture. Resource Limits & Key characteristics Link to Azure SQL documentation on resource limits Single Database Resource Limits Cores Memory (GB) Tempdb max data size (GB) Max Local SSD IOPS Max Log Rate (MiB/s) Max concurrent workers Max concurrent external connections per pool Max concurrent sessions 128 (Current Limit) 625 4,096 544,000 150 12,800 150 30,000 160 (New preview limit) 830 4,096 680,000 150 16,000 150 30,000 192 (New preview limit) 843* 4,096 816,000 150 19,200 150 30,000 *Memory values will increase for 192 vCores at GA. Elastic Pool Resource Limits Cores Memory (GB) Tempdb max data size (GB) Max Local SSD IOPS Max Log Rate (MiB/s) Max concurrent workers per pool Max concurrent external connections per pool Max concurrent sessions 128 (Current Limit) 625 4,096 409,600 150 13,440 150 30,000 160 (New preview limit) 830 4,096 800,000 150 16,800 150 30,000 192 (New preview limit) 843* 4,096 960,000 150 20,160 150 30,000 *Memory values will increase for 192 vCores at GA. Premium-series Hyperscale can now scale up to 160 vCores & 192 vCores in public preview regions. High performance CPUs optimized for compute-intensive workloads. Increased memory capacity proportional to vCore scale Up to 128 TiB of data storage, consistent with Hyperscale architecture Full compatibility with existing Hyperscale features and capabilities Performance Improvements with 160 and 192 vcores Strong scale-up efficiency observed beyond 128 vCores: Moving from 128 → 160 → 192 vCores shows consistent performance gains, demonstrating that Hyperscale Premium-series continues to scale effectively at higher core counts. 160 vCores delivers a strong balance of single-query and concurrent performance. 192 vCores is ideal for customers prioritizing maximum throughput, high user concurrency, and large-scale transactional or analytical workloads TPC-H Power Run (measures single-stream query performance) improves from 217 (128 vCores) to 357 (160 vCores) and remains high at 355 (192 vCores), delivering a +64% increase from 128 → 192 vCores, indicating strong single-query execution and CPU efficiency at larger sizes. TPC-H Throughput Run (measures multi-stream concurrency) increases from 191 → 360 → 511 QPH, resulting in a +168% gain from 128 → 192 vCores, highlighting significant benefits for highly concurrent, multi-user workloads. Performance case study (Zava Lending example) If the player doesn’t load, open the video in a new window: Open video Zava Lending scaled Azure SQL Hyperscale online as demand increased—supporting more users and higher transaction volume with no downtime. Throughput scaled linearly as compute increased, moving cleanly from 32 → 64 → 128 → 192 vCores to match real workload growth. 192 vCores proved to be the optimal operating point, sustaining peak transaction load without over‑provisioning. Azure SQL Hyperscale handled mixed OLTP and analytics workloads, including nightly ETL, without becoming a bottleneck. Every scale operation was performed online, with no service interruption and no application changes. Preview scope and limitations During preview, Premium-series 160 and 192 vCores are supported in a limited set of initial regions (Australia East, Canada Central, East US 2, South Central US, UK South, West Europe, North Europe, Southeast Asia, West US 2), with broader availability planned over time. During preview: Zone redundancy and Azure SQL Database maintenance window are not supported for these sizes Preview features are subject to supplemental preview terms, and performance characteristics may continue to improve through GA Customers are encouraged to use this preview to validate scalability, concurrency, memory utilization, query parallelism, and readiness for larger single database deployments. Next Steps This public preview is part of our broader investment in scaling Azure SQL Hyperscale for the most demanding workloads. Feedback from preview will help inform GA configuration limits, regional rollout priorities, and performance optimizations at extreme scale.Managed Identity Support for Azure SQL Database Import & Export services (preview)
Today we’re announcing a public preview that lets Azure SQL Database Import & Export services authenticate with user-assigned managed identity. Now Azure SQL Databases can perform import and export operations with no passwords, storage keys or SAS tokens. With this preview, customers can choose to use either a single user-assigned managed identity (UAMI) for both SQL and Storage permissions or assign separate UAMIs, one for the Azure SQL logical server and another for the Storage account, for full separation of duties. At a glance: Run Import/Export using a user-assigned managed identity (UAMI). Use one identity for both SQL and Storage, or split them if you prefer tighter scoping. Works in the portal, REST, Azure CLI, and PowerShell. Why this matters: Managed identity support makes SQL migrations simpler and safer, no passwords, storage keys, or SAS tokens. By leveraging managed identity when integrating Import/Export into a pipeline, you streamline access management and enhance security: permissions are granted directly to the identity, reducing manual credential handling and the risk of exposing sensitive information. This keeps operations efficient and secure, without secrets embedded in scripts You’ve got two straightforward options: One UAMI for everything (simplest setup). Two UAMIs, one for SQL and one for Storage, recommended if you wish to maintain more strictly defined permissions. Getting started: Create a user-assigned managed identity (UAMI) Decide up front whether you want one identity end-to-end, or two identities (SQL vs Storage) for separation of duties. Attach the UAMI to the Azure SQL logical server On the server Identity blade, add the UAMI so the Import/Export job can run as that identity. Set the server’s Microsoft Entra ID admin to the UAMI In Microsoft Entra ID > Set admin, select the UAMI. This is what lets the workflow authenticate to SQL without a password. Grant Storage access Use Storage Blob Data Reader for import and Storage Blob Data Contributor for export, assigned in Access control (IAM). If you can, scope the assignment to the container that holds the .bacpac. Pass resource IDs (not names) in your calls In REST/CLI/PowerShell, you pass the UAMI resource ID as the value of administratorLogin (SQL identity) and storageKey (Storage identity), and set authenticationType / storageKeyType to ManagedIdentity. administratorLogin → UAMI resource ID used for SQL auth storageKey → UAMI resource ID used for Storage authauthenticationType / storageKeyType → ManagedIdentity Run the import/export job Kick it off from the portal, REST, Azure CLI, or PowerShell. From there, the service uses the identity you selected to reach both SQL and Storage. Portal experience In the Azure portal, you can choose Authentication type = Managed identity and select the user-assigned managed identity to use for the operation. Figure 1: Azure portal Import/Export experience with Managed identity authentication selected. Notes This preview supports user-assigned managed identities (UAMIs). For least privilege, scope Storage roles to the specific container used for the .bacpac file and use two user-assigned managed identities (UAMIs), one for SQL and one for the storage. Sample 1: REST API — Export using one UAMI: $exportBody = "{ `n `"storageKeyType`": `"ManagedIdentity`", `n `"storageKey`": `"${managedIdentityServerResourceId}`", `n `"storageUri`": `"${storageUri}`", `n `"administratorLogin`": `"${managedIdentityServerResourceId}`", `n `"authenticationType`": `"ManagedIdentity`" `n}" $export = Invoke-AzRestMethod -Method POST -Path "/subscriptions/${subscriptionId}/resourceGroups/${resourceGroupName}/providers/Microsoft.Sql/servers/${serverName}/databases/${databaseName}/export?api-version=2024-05-01-preview" -Payload $exportBody # Poll operation status Invoke-AzRestMethod -Method GET $export.Headers.Location.AbsoluteUri Sample 2: REST API — Import to a new server using one UAMI: $serverName = "sql-mi-demo-target" $databaseName = "sqldb-mi-demo-target" # Same UAMI for SQL auth + Storage access $importBody = "{ `n `"operationMode`": `"Import`", `n `"administratorLogin`": `"${managedIdentityServerResourceId}`", `n `"authenticationType`": `"ManagedIdentity`", `n `"storageKeyType`": `"ManagedIdentity`", `n `"storageKey`": `"${managedIdentityServerResourceId}`", `n `"storageUri`": `"${storageUri}`", `n `"databaseName`": `"${databaseName}`" `n}" $import = Invoke-AzRestMethod -Method POST -Path "/subscriptions/${subscriptionId}/resourceGroups/${resourceGroupName}/providers/Microsoft.Sql/servers/${serverName}/databases/${databaseName}/import?api-version=2024-05-01-preview" -Payload $importBody # Poll operation status Invoke-AzRestMethod -Method GET $import.Headers.Location.AbsoluteUri Sample 3: PowerShell — Export using two UAMIs: # Server UAMI for SQL auth, Storage UAMI for storage access New-AzSqlDatabaseExport -ResourceGroupName $resourceGroupName -DatabaseName $databaseName -ServerName $serverName -StorageKeyType ManagedIdentity -StorageKey $managedIdentityStorageResourceId -StorageUri $storageUri -AuthenticationType ManagedIdentity -AdministratorLogin $managedIdentityServerResourceId Sample 4: PowerShell — Import to a new server using two UAMIs: New-AzSqlDatabaseImport -ResourceGroupName $resourceGroupName -DatabaseName $databaseName -ServerName $serverName -DatabaseMaxSizeBytes $databaseSizeInBytes -StorageKeyType "ManagedIdentity" -StorageKey $managedIdentityStorageResourceId -StorageUri $storageUri -Edition $edition -ServiceObjectiveName $serviceObjectiveName -AdministratorLogin $managedIdentityServerResourceId -AuthenticationType ManagedIdentity Sample 5: Azure CLI — Export using two UAMIs: az sql db export -s $serverName -n $databaseName -g $resourceGroupName --auth-type ManagedIdentity -u $managedIdentityServerResourceId --storage-key $managedIdentityStorageResourceId --storage-key-type ManagedIdentity --storage-uri $storageUri Sample 6: Azure CLI — Import to a new server using two UAMIs: az sql db import -s $serverName -n $databaseName -g $resourceGroupName --auth-type ManagedIdentity -u $managedIdentityServerResourceId --storage-key $managedIdentityStorageResourceId --storage-key-type ManagedIdentity --storage-uri $storageUrib For more information and samples, please check Tutorial: Use managed identity with Azure SQL import and export (preview)Why PITR Restore for Azure SQL Hyperscale Can Take Longer Than Expected
Azure SQL Database Hyperscale is designed to deliver fast, storage‑optimized backup and restore operations. According to Microsoft documentation, most Point‑in‑Time Restore (PITR) operations for Hyperscale should complete within 10 minutes, regardless of database size, because the service uses metadata-based restore techniques rather than copying full data files. However, some real‑world scenarios can lead to unexpectedly long restore times, even when the source database is Hyperscale and even when no obvious configuration changes are made. This article explains one such scenario, outlines what happened, and provides guidance to avoid similar delays in the future. Expected Behavior for Hyperscale PITR Hyperscale databases use a unique architecture that separates compute and storage. Backups are taken from storage snapshots and do not require data copying during a typical PITR restore. From Microsoft Learn: “Most restores complete within minutes, even for large databases.” Ref: https://learn.microsoft.com/azure/azure-sql/database/hyperscale-automated-backups-overview?view=azuresql#backup-and-restore-performance This performance expectation applies as long as the Backup Storage Redundancy remains the same between the source DB and the target restore. Customer Scenario Overview A customer initiated PITR restore operations for Hyperscale DB: Source DB: Hyperscale (SLO: HS_PRMS_64) Target DB: Hyperscale (SLO: HS_PRMS_128) Same logical server Source DB Backup Storage Redundancy: Standard_RAGRS Customer enabled Zone Redundancy for the target database during restore The customer therefore expected the restore to finish within the normal Hyperscale window (~10 minutes). Instead, the restore took significantly longer. Why the Restore Took Longer Although the source database used Standard_RAGRS, enabling Zone Redundancy at restore time introduced a configuration change that affected the underlying Backup Storage Redundancy (BSR) for the newly restored database. 🔍 Key Point: Changing BSR Creates a Full "Size-of-Data" Restore When the target DB uses a different BSR type than the source DB, Azure SQL Database cannot perform a fast metadata-based restore. Instead, it must perform a full data copy, and the restore becomes proportional to the database size: More data → longer restore Effectively behaves like a physical data movement operation This overrides Hyperscale’s normally fast PITR workflow This behavior is documented here: https://learn.microsoft.com/azure/azure-sql/database/hyperscale-automated-backups-overview?view=azuresql#backup-and-restore-performance In the customer’s case: Customer-enabled Zone Redundancy changed the restore workflow. As a result, the system selected a backup storage redundancy configuration different than the source: Restore workflow chose: Standard_RAGZRS Source database actually used: Standard_RAGRS (non‑zone‑redundant) This mismatch triggered a size-of-data restore, leading to the observed delay. Summary of Root Cause ✔ Hyperscale PITR is fast only when BSR is unchanged ✔ Customer enabled Zone Redundant configuration during restore ✔ This resulted in a different Backup Storage Redundancy from the source ✔ Target restore had to perform a full data copy, not metadata-based restore ✔ Restore time scaled with database size → leading to long restore duration Key Takeaways 1. Do not change Backup Storage Redundancy during PITR unless necessary Any change (e.g., RAGRS → RAGZRS) converts the restore into a size‑of‑data operation. 2. Restores that involve cross‑region or cross‑redundancy conversions always take longer This applies equally to: PITR restore Restore to another server Restore with SLO changes Restore involving ZRS/RA‑GZRS transitions 3. Hyperscale PITR is extremely fast—when configuration is unchanged If the source and target BSR match, Hyperscale restores usually complete in minutes. 4. Enabling Zone Redundancy is valid, but do it after the restore If the customer wants ZRS for the restored DB: Perform PITR first (fast restore) Then update redundancy post‑restore (online operation) Conclusion While Hyperscale PITR restores are typically very fast, configuration changes during the restore—especially related to Backup Storage Redundancy—can trigger a full data copy and significantly increase restore duration. To get the best performance: Keep the same BSR when performing PITR Apply redundancy changes after the restore completes Use metadata-based restores whenever possible Understanding these nuances helps ensure predictable recovery times and aligns operational processes with Hyperscale’s architectural design.
Introducing the Azure SQL hub: A simpler, guided entry into Azure SQL
Choosing the right Azure SQL service can be challenging. To make this easier, we built the Azure SQL hub, a new home for everything related to Azure SQL in the Azure portal. Whether you’re new to Azure SQL or an experienced user, the hub helps you find the right service quickly and decide, without disrupting your existing workflows. For existing users: Your current workflows remain unchanged. The only visible update is a streamlined navigation pane where you access Azure SQL resources. For new users: Start from the Azure SQL hub home page. Get personalized recommendations by answering a few quick questions or chatting with Azure portal Copilot. Or compare services side by side and explore key resources, all without leaving the portal. This is one way to find it: Searching for "azure sql" in main search box or marketplace is also efficient way to get to Azure SQL hub Answer a few questions to get our recommendation and use Copilot to refine your requirements. Get a detailed side-by-side comparison without leaving the hub. Still deciding? Explore a selection of Azure SQL services for free. This option takes you straight to the resource creation page with a pre-applied free offer. Try the Azure SQL hub today in the Azure portal, and share your feedback in the comments!
