Data Migration - Collation mismatch error during an offline migration
This post explores a collation mismatch error during offline migration from SQL Server Express to Azure SQL. It identifies the root cause—AUTO_CLOSE being enabled—and offers a simple fix to ensure smooth migration using Azure DMS.
DMS - Support for Managed Identity for Azure SQL Managed Instance migration
Azure Database Migration Service (DMS) has introduced a new feature that supports the use of Managed Identity for migrating to Azure SQL Managed Instance. This enhancement simplifies the migration process and ensures secure and seamless integration with Azure Database Migration services. In this blog post, we will dive into the prerequisites, permissions or role(s) required, and how to use this associated Managed identity for migrating to Azure SQL Managed Instance. Currently, this feature is supported through Azure Portal, PowerShell, and Az cmdlets. Prerequisites Before you begin the migration to Azure SQL Managed Instance using Managed identity, ensure that following prerequisites are in place: 1. The Target Azure SQL Managed Instance's associated Managed Identity: Azure Database Migration Service only supports Managed Identity that is associated with the target Azure SQL Managed Instance. How to identify the associated Managed Identity? Once you start the migration to Azure SQL Managed Instance using Azure Database Migration Service and on second page, select the target Azure SQL Managed instance, its associated Managed Identity will be displayed if "Use Managed Identity" is selected (default), as highlighted below. Alternatively, you can follow these steps: a) Go to the target Azure SQL Managed Instance's home page. b) On the left menu, under Security > Identity: If User-assigned Managed Identity is present, the associated Managed Identity will be same as selected under the Primary Identity. If there is no User-assigned Managed Identity and only System-assigned Managed Identity is enabled, the associated Managed Identity will be System-assigned Managed Identity and have the same name as the Azure SQL Managed Instance's name. For example, for ABCSQLMI - Azure SQL Managed Instance the System-assigned Managed Identity will be "ABCSQLMI". 2) Permissions: Assign the "Storage Blob Data Reader" role on the storage account to the target instance's associated Managed Identity. Steps to Assign Permissions In the Azure portal, go to the storage account that will be used in migration for keeping the backup files. On the left menu under Access Control (IAM), click on "+Add" > Add role assignment Select or search for builtin role "Storage Blob Data Reader", click Next. Assing this role access to Managed Identity by selecting the associated Managed Identity identified in the previous step as the member. Note: Ensure that the storage account has the "Allow storage account key access" enabled. How to use associated Managed identity for migration? Upon initiating the migration to Azure SQL Managed Instance using Azure Database Migration Service, navigate to the second page and select the target Azure SQL Managed Instance. If the "Use Managed Identity" option is selected (default), the associated Managed Identity will be displayed and used for the migration (as shown in the first image above). Once Managed identity is used for the migration, DMS will utilize this Managed identity for reading the backup files on the Azure blob storage and thus removing the need for SAS keys. Limitations: Azure Database Migration Service supports Managed Identity that is associated with the target Azure SQL Managed Instance only. It can be either User assigned, or System assigned Managed identity. Currently, this feature is supported through Azure Portal only. Ensure that the storage account has the "Allow storage account key access" enabled. Benefits of using Managed Identity: Using Managed Identity for Azure SQL Managed Instance migrations offers several security benefits: Enhanced Security: Managed identities eliminate the need to use SAS key, reducing the risk of SAS key token exposure. Simplified Management: As associated Managed Identity of the target Azure SQL MI is used, it allows for seamless integration with Azure Database Migration services, making it easier to manage access permissions and roles. Improved Efficiency: The streamlined authentication process speeds up migrations and reduces the complexity of managing SAS keys. Improved Compliance: By using Managed Identity, user can ensure that they adhere to security best practices and compliance requirements, as it is managed securely by Azure. All the above benefits make Managed Identity better than SAS key token. Learn more. Conclusion The new feature supporting Managed Identity in Azure Database Migration Service for Azure SQL Managed Instance migrations offers a secure and efficient way to manage permissions during the migration process. By following the steps outlined above and leveraging the security benefits of Managed Identity, you can ensure a smooth and secure migration to Azure SQL Managed Instance.Enhanced SQL Migration Tracking & Bringing SQL Server Arc Assessments to Azure Data Studio
In the ever-evolving landscape of data management, ensuring seamless and efficient migrations is crucial for businesses. Migration is a multi-step process and typically requires multiple tools to complete the migration. To enhance the migration experience, we are bringing a new feature that enhances the migration tracking experience across tools. In this blog post we delve into the benefits this feature offers to streamline the migration tracking process. This feature is available for Azure Data studio users via the latest Azure SQL Migration extension v1.5.8 The Azure SQL migration extension now offers 2 new features in ADS to help users in their migration journey. Ability to view Arc assessments, SKU recommendation in Azure Data Studio Ability to track the migration via SQL Server instance (At no additional cost!) Viewing Arc assessments in Azure Data Studio for SQL Server enabled by Azure Arc If you are planning to migrate SQL Server instances that are Arc-enabled, The Azure Data studio now helps you jump start the migration by providing the ability to view the pre-computed Arc assessments. ADS now provides a link to the pre-computed assessments in the Arc experience Azure portal, which provides migration readiness assessment, SKU recommendation and pricing information (coming soon). Users can continue with rest of their migration journey in the Arc experience. To view the pre-computed assessments computed by Arc, users have to select Yes to the ‘Is your SQL Server instance tracked in Azure? ‘and fill in the Azure resource details of the SQL Server Instance enabled by Azure Arc. The pre-computed assessments and SKU recommendations will be generated as a navigation link to users like below: Ability to streamline the migration tracking process For SQL Server instances which are not Arc-enabled, this feature provides an ability to track the migration by creating an Azure resource with no additional cost associated. Once this resource has been created, users can select this migration resource which is created for their successive migrations that take place for the same source & avail the assessment and readiness benefits. This migration tracking ability is available for both Azure Data Studio and Database migration service portal. Below images show the experience for this ability in Azure DMS portal:Release Announcement of SQL Server Migration Assistant (SSMA) v 10.1
Overview SQL Server Migration Assistant (SSMA) Access, DB2, MySQL, Oracle, and SAP ASE (formerly SAP Sybase ASE) allow users to convert a database schema to a Microsoft SQL Server schema, deploy the schema, and then migrate data to the target SQL Server (see below for supported versions). What’s new? Enhanced monitoring experience for migrations using DMS [ SSMA-Oracle] Migrating Oracle workloads using Azure Database Migration service is in preview, please refer for more details. For Oracle workloads migrating using DMS in SSMA, we are bringing an enhanced monitoring experience through tabular user interface where the users can now view live list of migrations that are in progress or completed phase. Each entry represents a migration activity along with the start time of migration, DMS used for migration and status. Users can view quick brief migration information of individual tables to know more information like Table name, Schema name, copy duration, No. of Rows copied & status real-time. For the sake of users requiring more granular monitoring information about their migration activity, SSMA provides a link to the Azure Database Migration service portal webpage (View Comprehensive Report) where they can view details like Data read, writes, Rows copies, Throughput along with copy duration, etc. Code Conversion Improvements [DB2, Oracle] Conversion enhancements for identity column from Db2 z/OS to SQL Server 2019 Improve conversion of Db2 stored procedure WITH RETURN clause to Azure SQL Database Improve database objects load for Db2 Appropriate error handling for conversion of identifier REPLACE(STRING, CHAR,CHAR) in Db2 Detection of CHAR length in Oracle VARCHAR2 datatype Downloads SSMA for Access SSMA for DB2 SSMA for MySQL SSMA for Oracle SSMA for SAP ASE Supported sources and target versions Source: For the list of supported sources, please review the information on the Download Center for each of the above SQL Server Migration Assistant downloads. Target: SQL Server 2016, SQL Server 2017, SQL Server 2019, Azure SQL Database, an Azure SQL Database managed instance Resources SQL Server Migration Assistant documentationPublic Preview announcement - Unified migration experience in Azure DMS
We are excited to announce that Azure Database Migration Service (DMS) now supports seamless migration of your MySQL on-premises or Virtual Machine (VM) workloads to Azure Database for MySQL - Flexible Server. This new feature, now available in public preview, allows you to use physical backup files of the MySQL server for migration. By restoring your physical data files directly to your target Flexible Server, you can migrate multi-terabyte workloads quickly and effortlessly with minimal downtime ensuring a smooth and efficient transition to Azure Database for MySQL - Flexible Server, enabling you to take full advantage of the platform's capabilities. To migrate your workloads using the Physical Online Data Migration option in Azure DMS, you need to take backups of your workload on the source server using Percona Xtrabackup utility. After taking a backup, upload the backup files to Azure Blob Storage. DMS can read the uploaded backup files from Azure Blob Storage and apply them on the target flexible server for rapid movement of large workloads to MySQL flexible server. To get started, go to your DMS project and choose "[Preview] Physical Online Data Migration" for migrating your workloads from on-premises or VMs. Limitations: You must create and configure the target Flexible server prior to migrating your physical backup files. Migration for encrypted backups isn't supported. Migration cancellation during the import operation is not supported. For more information about using physical online migration with Azure DMS please follow our detailed step-by-step instructions in our documentation: https://aka.ms/dmsPhysicalImportOnlineMigration If you have any feedback or questions about the information provided above, please leave a comment below or email us at AskAzureDBforMySQL@service.microsoft.com. Thank you!
Leveraging GitHub Copilot for T-SQL Code Conversion: A Deep Dive into Code Explainability
Converting database code between different relational database management systems (RDBMS) is an essential part of database migration , especially when moving from Oracle’s PL/SQL to SQL Server’s T-SQL. Both Oracle and SQL Server have unique syntax, functions, and programming conventions that require significant adjustments when converting complex SQL queries, stored procedures, or functions. While manual conversion can be a painstaking process, GitHub Copilot, an AI-powered code assistant, can ease this burden by offering real-time suggestions and automating many aspects of the conversion. In the first part of this two-part blog, we explore how GitHub Copilot can be a powerful tool for code conversion, helping developers quickly adapt code from one language or framework to another. By providing context-aware suggestions and completions, Copilot simplifies the process of rewriting and refactoring code. In the second part, we dive deeper into Copilot’s explainability feature, showcasing how it enhances the code conversion process from Oracle to Azure SQL. Code explainability in GitHub Copilot refers to the tool’s ability to offer suggestions and comments that help developers understand what a given piece of code is doing. When dealing with database migrations, such as converting Oracle SQL code to Azure SQL (SQL Server), GitHub Copilot’s ability to explain, suggest, and refactor code can ease the transition. It can assist in making the conversion process smoother, more efficient, and less error-prone by explaining the logic behind Oracle-specific queries and suggesting corresponding changes in Azure SQL’s syntax We'll go through multiple examples, analyze the differences between the two languages, and show how GitHub Copilot handles these challenges. Understanding the Key Differences: PL/SQL vs T-SQL Before jumping into examples, it's important to understand few of the fundamental differences between PL/SQL (Oracle) and T-SQL (SQL Server): Syntax: While both are procedural SQL dialects, there are key differences in the way they handle variables, control structures, and flow control. Functions: Each platform has its own set of built-in functions (e.g., string manipulation, date handling, etc.), and these need to be mapped correctly during conversion. Error Handling: Error handling and exception management differ between the two, with PL/SQL using EXCEPTION blocks and T-SQL using TRY...CATCH. Cursor Handling: While both support cursors for iterating through results, their syntax differs. Leveraging GitHub Copilot for Oracle PL/SQL to SQL Server T-SQL Code Conversion: A Deep Dive into Complex Examples with Explainability Converting database code between different relational database management systems (RDBMS) is an essential part of database migration , especially when moving from Oracle’s PL/SQL to SQL Server’s T-SQL. Both Oracle and SQL Server have unique syntax, functions, and programming conventions that require significant adjustments when converting complex SQL queries, stored procedures, or functions. While manual conversion can be a painstaking process, GitHub Copilot, an AI-powered code assistant, can ease this burden by offering real-time suggestions and automating many aspects of the conversion. In the first part of this two-part blog, we explore how GitHub Copilot can be a powerful tool for code conversion, helping developers quickly adapt code from one language or framework to another. By providing context-aware suggestions and completions, Copilot simplifies the process of rewriting and refactoring code. In the second part, we dive deeper into Copilot’s explainability feature, showcasing how it enhances the code conversion process from Oracle to Azure SQL. Code explainability in GitHub Copilot refers to the tool’s ability to offer suggestions and comments that help developers understand what a given piece of code is doing. When dealing with database migrations, such as converting Oracle SQL code to Azure SQL (SQL Server), GitHub Copilot’s ability to explain, suggest, and refactor code can ease the transition. It can assist in making the conversion process smoother, more efficient, and less error-prone by explaining the logic behind Oracle-specific queries and suggesting corresponding changes in Azure SQL’s syntax We'll go through multiple examples, analyse the differences between the two languages, and show how GitHub Copilot handles these challenges. Understanding the Key Differences: PL/SQL vs T-SQL Before jumping into examples, it's important to understand few of the fundamental differences between PL/SQL (Oracle) and T-SQL (SQL Server): Syntax: While both are procedural SQL dialects, there are key differences in the way they handle variables, control structures, and flow control. Functions: Each platform has its own set of built-in functions (e.g., string manipulation, date handling, etc.), and these need to be mapped correctly during conversion. Error Handling: Error handling and exception management differ between the two, with PL/SQL using EXCEPTION blocks and T-SQL using TRY...CATCH. Cursor Handling: While both support cursors for iterating through results, their syntax differs. Example 1: Complex Stored Procedure Conversion Let’s start with a complex example that involves handling parameters, cursors, and error handling. We'll look at a PL/SQL stored procedure in Oracle that processes employee records and outputs their details. CREATE OR REPLACE PROCEDURE GetEmployeeDetails (emp_id IN NUMBER) IS CURSOR emp_cursor IS SELECT employee_id, first_name, last_name, hire_date FROM employees WHERE employee_id = emp_id; emp_record emp_cursor%ROWTYPE; emp_full_name VARCHAR2(100); BEGIN OPEN emp_cursor; LOOP FETCH emp_cursor INTO emp_record; EXIT WHEN emp_cursor%NOTFOUND; -- Concatenate first and last name emp_full_name := emp_record.first_name || ' ' || emp_record.last_name; -- Output employee details DBMS_OUTPUT.PUT_LINE('Employee ID: ' || emp_record.employee_id); DBMS_OUTPUT.PUT_LINE('Full Name: ' || emp_full_name); DBMS_OUTPUT.PUT_LINE('Hire Date: ' || TO_CHAR(emp_record.hire_date, 'YYYY-MM-DD')); END LOOP; CLOSE emp_cursor; EXCEPTION WHEN NO_DATA_FOUND THEN DBMS_OUTPUT.PUT_LINE('No employee found with the given ID.'); WHEN OTHERS THEN DBMS_OUTPUT.PUT_LINE('An error occurred.'); END; Oracle PL/SQL Example In Oracle, we might have a procedure that uses a cursor to fetch employee records and print them. The procedure also includes error handling for potential issues during execution. Key Points in Oracle PL/SQL: Cursor: PL/SQL uses CURSOR for iterating through rows. Concatenation: Oracle uses || for string concatenation. Error Handling: PL/SQL uses EXCEPTION blocks for handling errors such as NO_DATA_FOUND. Converting to SQL Server T-SQL with GitHub Copilot When you type the PL/SQL code into an IDE with GitHub Copilot enabled (e.g., Visual Studio Code), Copilot will start suggesting the equivalent SQL Server T-SQL code. Here’s how Copilot converts the procedure. Copilot’s Suggested SQL Server T-SQL Code CREATE PROCEDURE GetEmployeeDetails (@emp_id INT) AS BEGIN DECLARE @emp_id INT, @first_name NVARCHAR(50), @last_name NVARCHAR(50); DECLARE @emp_full_name NVARCHAR(100); DECLARE emp_cursor CURSOR FOR SELECT employee_id, first_name, last_name, hire_date FROM employees WHERE employee_id = @emp_id; OPEN emp_cursor; FETCH NEXT FROM emp_cursor INTO @emp_id, @first_name, @last_name, @hire_date; WHILE @@FETCH_STATUS = 0 BEGIN -- Concatenate first and last name SET @emp_full_name = @first_name + ' ' + @last_name; -- Output employee details PRINT 'Employee ID: ' + CAST(@emp_id AS NVARCHAR); PRINT 'Full Name: ' + @emp_full_name; PRINT 'Hire Date: ' + CONVERT(NVARCHAR, @hire_date, 23); FETCH NEXT FROM emp_cursor INTO @emp_id, @first_name, @last_name, @hire_date; END CLOSE emp_cursor; DEALLOCATE emp_cursor; BEGIN TRY -- Error handling code (example) END TRY BEGIN CATCH PRINT 'An error occurred.'; END CATCH END; Key Points in T-SQL: Cursor: SQL Server uses DECLARE CURSOR to declare a cursor, similar to PL/SQL but with different syntax for opening and fetching. Concatenation: SQL Server uses + for string concatenation. Error Handling: SQL Server uses TRY...CATCH for error handling, unlike PL/SQL’s EXCEPTION block. GitHub Copilot provides a T-SQL equivalent for an Oracle PL/SQL stored procedure. Explanation of the Conversion Process Cursor Declaration and Fetching: PL/SQL uses CURSOR with OPEN, FETCH, and CLOSE to process rows. T-SQL uses DECLARE CURSOR to define the cursor and fetches rows with FETCH NEXT. Both approaches are similar, but T-SQL requires explicit DEALLOCATE to release the cursor. Error Handling: PL/SQL uses EXCEPTION blocks to handle errors like NO_DATA_FOUND. T-SQL uses TRY...CATCH blocks to capture and handle errors, a major difference in error handling between the two systems. String Concatenation: PL/SQL uses || for concatenating strings, while T-SQL uses +. This is a simple yet crucial change when converting between the two dialects. Example 2: Complex Query with Date Handling Let’s now look at a query that involves date handling and conditional logic. These are often tricky when converting between PL/SQL and T-SQL due to differences in how each platform handles dates. Oracle PL/SQL Query Example SELECT employee_id, first_name, hire_date, CASE WHEN hire_date < TO_DATE('2000-01-01', 'YYYY-MM-DD') THEN 'Veteran' ELSE 'New Hire' END AS employee_status FROM employees WHERE hire_date BETWEEN TO_DATE('1990-01-01', 'YYYY-MM-DD') AND SYSDATE; Key Points in PL/SQL: TO_DATE: Used to convert strings into dates. SYSDATE: Returns the current system date. CASE: Used for conditional logic to categorize employees as either "Veteran" or "New Hire". Copilot’s Suggested SQL Server T-SQL Code SELECT employee_id, first_name, hire_date, CASE WHEN hire_date < CONVERT(DATE, '2000-01-01', 23) THEN 'Veteran' ELSE 'New Hire' END AS employee_status FROM employees WHERE hire_date BETWEEN CONVERT(DATE, '1990-01-01', 23) AND GETDATE(); Key Points in T-SQL: CONVERT: SQL Server uses CONVERT for date formatting, similar to Oracle’s TO_DATE, but with different syntax and style codes. GETDATE(): Equivalent to Oracle’s TO_DATE In conclusion, GitHub Copilot streamlines code conversion by offering both quick suggestions and detailed explanations. The first part highlighted its ability to assist with code transitions, while the second part focused on how its explainability feature enhances the Oracle to Azure SQL migration, providing guided, efficient, and error-free conversion. This combination accelerates development and reduces potential issues during migration.Release: Azure SQL Migration extension for Azure Data Studio v1.5.6
We're delighted to announce the release of the latest version of the Azure SQL Migration extension for Azure Data Studio, v1.5.6. This release provides you with Azure Database Migration Service’s new features like: 1) Support for Next-gen General Purpose service tier for Azure SQL Managed Instance. 2) Target Provisioning based upon SKU recommendation (using ARM templates) - Public Preview. 3) Enhanced login migration experience - Public Preview. What is new in Azure SQL Migration extension v1.5.6? 1) Support for Next-gen General Purpose service tier for Azure SQL Managed Instance: The Next-gen General Purpose service tier is an architectural upgrade to the existing General Purpose service tier that can be used for new and existing instances. Now, the Azure Data Studio extension for Azure Database migration service – Azure SQL Migration support Next-gen General Purpose as SKU recommendation for Azure SQL Managed Instance. For details, refer here. This service tier provides better performance, throughput, greater storage capacity and support more than 100 databases on a single instance. 2) Target Provisioning based upon SKU recommendation (using ARM templates) - Public Preview: With Azure SQL Migration v1.5.6, now users can generate ARM templates directly based upon the SKU recommendation generated using performance data collected from the source. User can use these ARM templates for all the Azure SQL offerings – Azure SQL VM, Azure SQL MI and Azure SQL DB and easily create the Azure SQL Target for the migrations. To create and deploy the Azure SQL Target, users have two options: a) Copy or save the ARM template in JSON and use Azure CLI, PowerShell and other deployment operations. b) Using Deploy-to-Azure button, then provide the Azure blob storage account details to store the template and deploying it though the Azure Portal. This feature is in Public Preview and will help you to streamline the Azure SQL target creation using ARM templates, can automate deployments and use the practice of infrastructure as code, deploy them quickly and CI/CD integration. 3) Enhanced login migration experience - Public Preview: After completing the data migration, the next critical step is to setup the authentication and authorization for the databases and thus login migration becomes the critical step in the migration journey. Azure SQL Migration extension supports Login migration (Public Preview) and now we have enhanced its experience by adding Pre-requisites validation checks to ensure all the requirements are in place for successful login migrations. Currently, only Azure SQL Managed Instance and SQL Server on Azure Virtual Machines targets are supported. Resources For more information about the extension and Azure Database Migration Service, see the following resources. Azure Database Migration Service documentation Migrate databases using the Azure SQL Migration extension One-click SQL Migration PoC environment Architecture of Azure Database Migration Service | Microsoft Community HubGitHub Copilot and SSMA: Strap a GenAI conversion booster to your Oracle to SQL Migrations
In this blog, we’ll explore how GitHub Copilot and SQL Server Migration Assistant (SSMA) for Oracle can supercharge your code conversion journey from PL/SQL to T-SQL. Learn how both these tools bring comprehensive conversion rule set and Generative AI capabilities together to simplify and expedite your migration journey from Oracle to Azure SQL.
Advanced Document Management System (DMS) for sharepoint ?
Does anyone know of a solution which enhances the Document Management Features of Sharepoint 365 and makes it a "proper DMS" ? Are there vendors out there which build their DMS solution on the sharepoint platform, rather than developing their own solution from scratch ? We would love to keep all our documents within the office 365 ecosystem, but provide our users with everything a modern DMS has to offer these days. Any suggestions and experiences welcome.
