Cutover Timeout recommended settings

[John Wildes]

Hello

I'm trying to migrate a server to Azure.  I'm in the cutover phase, it has a default timeout of 2880 minutes, which is 48 hours.  Personally I can't have a file server "offline" for 48 hours during cutover.  It's a simple server, so I changed this to 60 minutes.  It's 2016 to 2016 on prem to Azure. It's been running for more than an hour, and stuck at 44%.  When would this cancel, since it's reached it's timeout threshold?  How would I manually cancel this job, as there's no UI for that?  

thanks

john

[#2]

Hi John. Can you tell me what the "State" column shows on the WAC cutover UI when it's stuck? 

If you chose static IP for the destination, cutover to Azure isn't going to work automatically. Azure does not support L2 networks, it's just a whole new network - but then if you put on an on-prem IP address on that, the VM will become unreachable. We are adding an option in a coming update to skip static network migration to the source for this very reason. And we will give better hints and advice in the UI before you accidently shoot yourself. 

The 60 minutes expiring should have ended this, that's just a bug on our end, perhaps in the UI itself. You can use Stop-SMSCutover to end this wedged cutover state. If you attempt the cutover again (until we provide the update to make Azure IaaS work properly), once you get to 44% you can just go set the VMs IP address to DHCP in Azure itself so that the cutover can resume and complete.

- Ned

[#3]

Ned,

Thanks for the reply!

It was stuck on "44% adding source computer to the domain".  It did time out, a few minutes after 60, with FAILED status.  I have the logs from the Orchestrator computer and the destination computer. 

We checked on the source box, and it had been renamed to what we specified during the cutover wizard process. It had been removed from the domain, it was workgroup status.  It was placed as DHCP, which we also specified during the cutover process (admittedly I thought this setting was what was going to be set on the Azure machine, and not change the source machine).  In our case I'd prefer we not change the IP of the source box, and allow the Azure VM to retain what is currently set within Azure.  Currently the Azure VM is set to DHCP. 

We logged in with the local administrator account on the source box, renamed it back to the old machine name, and joined it to the domain.  Users could connect to it, and the shares so we "recovered" and would like to try this again.

When we ran the validate step the process did give a PASS check mark on it.  However it only ran the FIRST of what looked like 8 different checks, the other checks had NOT RUN on them. 

[#4]

Ah good - I was worried our cancel at the 1 hour work was totally being ignored. I think our UI could be much more clear here, that’s good feedback.

We intentionally prevent the source from keeping the same IP address because many apps/scripts/users will continue to talk to the decommissioned source after cutover. In the new feature where you can skip migration of the network we will still require that. But we’re thinking of adding a 3rd “do nothing at all here and let admins operate at their own risk” mode.

If you try this again, if you get to the 44% phase, set the destination VMs IP back to working and let it register DNS, then the migration should start working within the dns replication and timeouts. We have that overall cutover timeout so high because many environments will not have converged their AD and DNS for many hours.

Do you have a screenshot of what you mean about the validate?

[#5]

Ok, I'll test it again, 

I've attached screens of the Validate source and destination devices step of the cutover wizard.  it says PASS.  

When looking further into it the MIGRATION TEST RESULTS screen that comes up when you click validation shows a bunch of things NOT RUN, but the first item that says WE CAN MIGRATE is marked as PASS.

[#6]

I can see why you would do what you do by default with the changing of the IP address.  In my case I would power off the old server after cutover to preserve state in case you had to roll back.  I do think having a 3rd option of letting the admin proceed at their own choosing, highlighting some risks of proceeding without the recommended changes would be good practice.  Allows for flexibility.

[#7]

Oh that's busted. We should not be showing pass after not running! Does that repro every time you run validate?

[#8]

Those screen shots are fresh.  Happens every time...

[#9]

Hmm. Is a cutover already running when you ran this validate?

[#10]

No, there's no cutover running, when I took those screen shots.  However it was the same "job" that had failed.  I was just starting over at the beginning of the CUTOVER Wizard again.  

I've started back at the TRANSFER portion of the job, and that has just completed.  I'm now at the cutover step, and have received a pass on the validate step.  All portions of this have PASS in them now.

Went back through all the settings to make sure that all appropriate firewall ports were opened on source / destination / orchestrator.  All things are set.

I opened a ticket for windows server through the Azure portal.  Support staff on this one have said they were going to escalate to a specialist team as they haven't had the training on SMS yet.  Whatever that means ;) 

Let me know if you need anything else.

[#11]

Well that’s interesting. Can you share your logs?

[#12]

Ned, lemme know where you want me to send them.  I just started a new Get-SMSLOGS

I uploaded a set of logs for ticket 119012925004530 earlier today.  If you can somehow get them internally.  

john

[#13]

I cannot get to those, for PII control reasons. But being with support is the right answer.

We’re theorizing the cutover flag wasn’t returned and we’re going to file a bug on our end to investigate.

[#14]

Ned,

How is this supposed to work.  I've been able to repro my issue EVERY TIME now.  All settings according to the documentation are correct from firewall ports and user account privileges.  Here's what I see happening.

To recap, I am taking an on prem file server, and cutting over to an IaaS VM in Azure which is supported by the documentation.

My VM in Azure is on a completely different subnet, and you CANNOT specify an IP address on the physical NIC, WITHIN windows, and have the IaaS VM be accessible.  So unless I'm cutting over an entire subnet of addresses that this on prem VM belongs to, MOVING the on prem vm address to the IaaS VM is going to fail.  It's going to fail regardless because the process sets an IP address within windows, which breaks Azure.

1.) at the beginning of the cutover process during the ADJUST SETTINGS part of the Wizard I cannot proceed unless I choose DHCP as my option for the SOURCE computer.  If I specify an address, the same address that the on prem NIC is set to, it won't continue.  I continue on this step with DHCP

2.) the process has removed the computer from the domain, and the process had changed the ip address on the server to DHCP and this is where it appears to fail, as the server gets a new address, but does not register with DNS.  DHCP at my customer location is handed out by a router in their network and not a Windows DHCP server.  DNS is handled by AD.  the DHCP server affixes the appropriate suffix to the computer when it gets a DHCP address, AND appropriate AD DNS servers, yet still does not register in DNS.

3.) The process reports 44% waiting to join the source computer to the domain.

4.) I login to the console of the VM through Hyper-V manager using a local account, and check the status of the box.  It is in a state of "this computer will be renamed on reboot" with the new name that I've specified as the name the computer will become.  I reboot the computer manually.

5.) I login to the console of the VM with a local account again. The VM does nothing.  I manually add it to the domain and it reboots.

6.) the progress indicator has now moved on the process, and proceeds to 72% "mapping network interfaces on destination computer"  This will fail because my machine is in Azure.  

NOW ONTO THE DESTINATION VM IN AZURE

7.) I attempt to connect to the VM  using the IP address dynamically assigned to the NIC from the Azure VNET. I cannot RDP into this box, my assumption is that the process has changed the ip address on the ONLY nic on this box to match the on prem IP address.

8.) I create a new nic, shut down my VM, attach the new nic, boot the VM, and am Able to connect to the box via RDP on the new NIC ip address which has been dynamically assigned by Azure.

9.) The machine is in a state of

a - removed from the domain

b - primary nic has assumed the ip configuration for the on prem network (which is not valid in azure)

c - machine name has not been renamed to the on prem VM name.

10.) I mark the NIC within Windows to be DHCP.  I restart the box.  It is appropriately set to DHCP

11.) The machine name is still set to it's current name, so I manually rename it to the on prem file server name.  Join it to the domain and restart.

12) the machine comes up renamed as the old file server on prem. Joined to the domain, the SMS cutover job stays at 72% and does not complete.

13) Upon each and every reboot of the server in Azure, the IP address of the first NIC assumes the IP Configuration of the on prem server within Windows.  

14.) My job times out at 60 min.  I'm unable to complete the process of cutting over from an on prem vm to an Azure VM. 

[#15]

Yes, until we give you the option to skip networking that I mentioned before (coming in an update in a month or two), IaaS VM is not something SMS currently supports. Anything you do right now is hacking. 

For the source computer though, that is something entirely unrelated and should be working fine here. Is the DNS  server IP address(s) being assigned by DHCP or set statically? I need to understand exactly how to repro this. That it's not registering with DNS is the real issue here - did your customer disable automatic DNS registration?

[#16]

Thanks, for the reply Ned

I'll check on the DNS registration.  From what I see in AD right now, it is set for nonsecure and secure updates, so the devices that get suffix and ip information and cannot be domain joined should show up appropriately.  This would be one of those boxes.  I'll check on it though.

So for this particular project i won't be able to wait until the skip the networking feature is available.

What exactly happens in the cutover piece?  I think I remember reading in the documentation that I could stop after transfer if I didn't want to actually assume the identity of the old file server on the new one.  If all I have to do is shut down the old file server, and rename the new one manually and I'm done then I'll do that manually.  If there are other steps that I'm missing out on with the cutover process can you share what happens?

[#17]

Ned,

Thanks for jumping in on this thread.  I have started another one around SMS Inventory process, and need assistance there as well, as I have three servers that are "failing" inventory.  They each take over 24 hours to progress very little. No indication in the logs as to what is wrong, and everything I end up manually cancelling.  Same environment as this one server that did inventory properly.

I would appreciate any thoughts you can share on this thread, if you have time.

I'm 0 for 3 on the support technicians for this case currently they're passing it to another group.

r/

john

[#18]

What I'd recommend instead for Azure IaaS currently is:

1. Migrate to an on-prem VM

2. Use ASR to move that VM to Azure, and then turn off ASR for that VM so it's not continuously protected (i.e. lift and shift), so you don't pay for anything. https://azure.microsoft.com/en-us/services/azure-migrate/

But you'll need to figure out why DNS registration is broken, or manually register the DNS entries so that cutover can complete (it will if you do that; we've had one lab here with no DNS auto-registration allowed and that was my solution there).

[#19]

Thanks for the help Ned.