Introducing the Windows Server Storage Migration Service

First published on TECHNET on Apr 12, 2018

Update! We shipped Windows Server 2019!!! Go to https://aka.ms/storagemigrationservice for all production support info on SMS! You don't need this blog post anymore. 😊

Hi Folks, Ned here again with big news: Windows Server 2019 Preview contains an entirely new feature!

The Storage Migration Service helps you migrate servers and their data without reconfiguring applications or users.

• Migrates unstructured data from anywhere into Azure & modern Windows Servers


• It’s fast, consistent, and scalable


• It takes care of complexity


• It provides an easily-learned graphical workflow

My team has been working on this critter for some time and today you’ll learn about what it can do now, what it will do at RTM, and what the future holds.

Did I mention that it comes in both Standard and Datacenter editions and has a road map that includes SAN, NAS, and Linux source migrations?

Come with me…

Why did we make this?

You asked us to! No really, I dug through endless data, advisory support cases, surveys, and first party team conversations to find that our #1 issue keeping customers on older servers was simply that migration is hard, and we don’t provide good tools. Think about what you need to get right if you want to replace an old file server with a new one, and not cause data loss, service interruption, or outright disaster:

• All data must transfer


• All shares and their configuration must transfer


• All share and file system security must transfer


• All in-use files must transfer


• All files you, the operator, don’t have access to must transfer


• All files that changed since the last time you transferred must transfer


• All use of local groups and users must transfer


• All data attributes, alternate data streams, encryption, compression, etc. must transfer


• All network addresses must transfer


• All forms of computer naming, alternate naming, and other network resolution must transfer


• Whew!

If you were to wander into this Spiceworks data on market share (a little old but still reasonably valid), you’ll see some lopsided ratios:

( source )

A year and a half later, there are a few million Windows Server 2016 nodes in market that have squeezed this balloon, but there’s even more Windows Server 2012 and still plenty of 2008 families, plus too much wretched, unsupported Windows Server 2003. Did you know that Windows Server 2008 Support ends in January of 2020 ? Just 20 months away from the end of life for WS2008 and we still have all this 2003!

The Storage Migration Service (Updated: August 30, 2018)

Important: This section of the blog post is going to change periodically, as with the Windows Insider preview system and Windows Admin Center’s preview extension system, I can give you new builds, features and bug fixes very rapidly. You’ll want to check back here often.

In this first version, the feature copies over SMB (any version). Targets like Azure File Sync servers, IaaS VMs running in Azure or MAS, or traditional on-prem hardware and VMs are all valid targets.

The feature consists of an orchestrator service and one or more proxy services deployed. Proxies add functionality and performance to the migration process, while the orchestrator manages the migration and stores all results in a database.

Storage Migration Service operates in three distinct phases:

1. Inventory – an administrator selects nodes to migrate and the Storage Migration Service orchestrator node interrogates their storage, networking, security, SMB share settings, and data to migrate


2. Transfer – the administrator creates pairings of source and destinations from that inventory list, then decides what data to transfer and performs one more or transfers


3. Cutover – the administrator assigns the source networks to the destinations and the new servers take over the identity of the old servers. The old servers enter a maintenance state where they are unavailable to users and applications for later decommissioning, while the new servers use the subsumed identities to carry on all duties.

Walk through

Requirements

You’ll need the following to start evaluating this feature:

• At least two Windows Server 2019 Preview computers or VMs with the Storage Migration Service features installed ( Build 17744 or later ). One will operate as the orchestrator and one as the destination of the migration. Note: it is supported to make the orchestrator and destination the same computer, such as in a small environment with a single server to migrate. A larger environment will usually have a single Orchestrator and many destinations, however, so that's how the steps are documented below.


• The Windows Admin Center installed on some computer, such your laptop or desktop


• The Storage Migration Service preview extension for Windows Admin Center installed

Supported source operating systems VM or hardware (to migrate from) :

• Windows Server 2003


• Windows Server 2008


• Windows Server 2008 R2


• Windows Server 2012


• Windows Server 2012 R2


• Windows Server 2016


• Windows Server 2019 Preview

Supported destination operating system VM or hardware (to migrate to ):

• Windows Server 2012 R2


• Windows Server 2016


• Windows Server 2019 Preview*

* Windows Server 2019 Preview will have double the data transfer performance due to its inclusion of the SMS Proxy service.
Security:

• All computers domain-joined


• You must provide a migration account that is an administrator on selected source computers


• You must provide a migration account that is an administrator on selected destination computers


• The following firewall rules must be enabled INBOUND on source and destination* computers:
  
  
  
  • “File and Printer Sharing (SMB-In)”
  
  
  • “Netlogon Service (NP-In)"
  
  
  • "Windows Management Instrumentation (DCOM-In)"
  
  
  • "Windows Management Instrumentation (WMI-In)"
  
  
  
  

* Windows Server 2019 with the SMS Proxy installed will automatically open and close correct firewall ports during migrations

• The following firewall rule must be enabled INBOUND on orchestrator computer only if you wish to download reports:
  
  
  
  • “File and Printer Sharing (SMB-In)”
  
  
  
  

Install

Use the Windows Admin Center, Server Manager, or PowerShell to install the Storage Migration Service. The SMS extension allows this with a single click:

The features to install on the orchestrator node are:

• “Storage Migration Service”


• “Storage Migration Service Proxy” (installs automatically when orchestrator selected)


• “Storage Migration Service tools” ( installs automatically when orchestrator selected, under Remote Server Administration Tools, Feature Administration Tools )

The feature to install on the Destination nodes where you intend to migrate to :

• “Storage Migration Service Proxy”

Run your first test inventory and transfer

Now you’re ready to start migrating.

1. Logon to your Windows Admin Center instance and connect to the orchestrator node as an administrator.

2.  Ensure the latest Storage Migration Service extension is in the Tools menu (if not, install the extension ) and click it.

3. Observe the landing page. There is a summary that lists all active and completed jobs. Jobs contain one or more source computers to inventory, transfer, and cutover as part of a migration.

4. You are about to begin the Inventory. Click New Job . Enter a job name, click Next.

5. Enter Source Credentials that are an administrator on your source (to be migrated from) computers and click Next.

6. Click Add Device and add one or more source computers. These must be Windows Server and should contain SMB Shares with test data on them that you want to migrate.

7. Click Start Scan and wait for the inventory to complete.

8. Observe the results. You can open the Details at the bottom of the page with the caret in the lower right. When done reviewing, click Finish Inventory .

9. You are now in the Transfer You are always free to return to the Inventory phase and redefine what it gathers, get rid of the job, create a new job, or proceed forward to data transfer. As you can see, each phase operates in a similar fashion by providing credentials, setting rules, defining nodes, then running in a result screen.

10. Provide credentials, destination computers mapped to the source computers, ensure each server you wish to migrate is set to Included in transfer, review your settings, validate the proposed transfer, then proceed with Start Transfer .

11. Observe the migration. You will see data transfers occur in relative real time (periodically refreshed) as the orchestrator copies data between source and destination nodes. When complete, examine the destination server and you’ll find that Storage Migration Service recreated all shares, folders, and files with matching security, attributes, characteristics (see Known Issues below for not-yet-released functionality here). . Note the Export options that allows you to save a complete database dump of the transfer operations for auditing purposes.

12. Move to the cutover phase.

13. Note that the destination credentials are preserved (but can be changed).

14. Pair the network interfaces between source and destination NICs so that IP addresses can be moved. You have the option to move the old source computer to DHCP or use a new static IP address. You also have the option to randomly rename the old source computer or specify a new name, as the destination will take over the old name as part of cutover.

15. Validate the cutover preparedness, then begin the cutover and allow the destination computer to take over the network and names of the old source. Both source and destination will reboot several times apiece. The progress bar will show how far until the operation completes.

Cutover time to completion depends on:

a. Server reboot times

b. AD Replication time (for domain joins and computer accounts being known to all users)

c. DNS replication time

16. When cutover completes, migration is done. Your old servers are renamed and have new network configurations so that they are inaccessible to users and applications, but still retain all data and shares. The new servers now assume all duties of the computers they replaced, and users cannot tell that anything changed.

Known issues

Important! During the preview phase we need you to always run latest binaries before reporting bugs. To get the latest:

1. Windows Server 2019 Preview 17744 or later - download from https://www.microsoft.com/en-us/software-download/windowsinsiderpreviewserver


2. Storage Migration Service extension for Windows Admin Center 17744 or later - download using the extension manager https://docs.microsoft.com/en-us/windows-server/manage/windows-admin-center/configure/using-extensions . If you already have a previous extension installed, uninstall it , then install the latest extension.

• Missing Menu Option - if you do not see the SMS tool listed in the left-hand menu of Windows Admin Center even with the extension installed, ensure that your Orchestrator server is running build 17744 or later

Applies to:

• Windows Server 2019 Insider Preview Build 17744


• Storage Migration Service Extension Version 0.0. 17744

Roadmap

What does the future hold? We have a large debt of work already accumulated for when we complete the SMB and Windows Server transfer options. Things on the roadmap – not promised, just on the roadmap 😊:

1. Network range and AD source computer scanning for inventory


2. Samba, NAS, SAN source support


3. NFS for Windows and Linux


4. Block copy transfers instead of file-level


5. NDMP support


6. Hot and cold data detection on source to allow draining untouched old files


7. Azure File Sync and Azure Migrate integration


8. Server consolidation


9. More reporting options

Feedback and Contact

Please make sure the issue isn’t already noted above before filing feedback and bugs!

• Use the Feedback Hub tool include in Windows 10 to file bugs or feedback. When filing, choose Category “ Server, ” subcategory “ ” It helps routing if you put Storage Migration Service in the title.


• You can also provide feature requests through our UserVoice page at https://windowsserver.uservoice.com/forums/295056-storage . Share with colleague and industry peers in your network to upvote items that are important to you.


• If you just want to chat with me privately about feedback or a request, send email to smsfeed@microsoft.com . Keep in mind that I may still make you go to feedback hub or UserVoice.

Now get to testing and keep an ear out for updates. We plan to send out new builds of Windows Server 2019 and the Storage Migration service very often until we get close to RTM. I will post announcements here and on twitter .

Here’s some background music to help

https://youtu.be/Dyx4v1QFzhQ

- Ned “reel 2 real” Pyle