Hello!
My name is Herbert Fuchs and together with other members of the Customer Success Unit and the Customer Service & Support Organization we want to help our Customers with This Blog-Series. We gathered information and put our field and support experience into this. Special thanks to our contributors, reviewers and content-writers, Wilhelm Kocher, Anthony Fontanez, Emilian Bucur, Pavel Yurenev, Anderson Cassimiro and Madalina Zamfir.
In this Blog we want to explain what is necessary to change the Operating System for a Server which host the Configuration Manager by a Disaster Recovery Procedure.
We all know this sentence from the past – never touch a running system – but frankly speaking Configuration Manager is not different to any other Service. And a Service must be able to be recovered. So do not fear this. Be serious and cautious and know what is necessary to restore your Service. We advise and recommend that you test your Backup & Recovery Procedure on a regular basis. If those tests are with production data even better – because only a test with production data is a valid test.
Backup Methods
Before we continue with the Recovery – we first clarify our Backup. Here we have two Options we can use the Built-in-Backup or we use a native SQL-Backup – this makes also a difference in the Recovery-Wizard.
Backup sites - Configuration Manager | Microsoft Learn
So, what is the difference between those.
Built-In-Backup:
Here we talk about a Site-Maintenance-Task. A simple fire and forget checkbox. Activate the Task. Define the Schedule. Define the Location.
What does the Backup include – here we find the cd.latest to your current Configuration Manager Version. The Database and Transaction File. A Copy of your inboxes and an export of Registry-Values. Everything in one place.
However, there is no backup of your Package-Sources or the Content-Library. There is no backup of other Database like SUSSDB, Report Server. And there is no archive out of the box – the Backup-Location will be wiped, and Backup-Files recreated. You can modify/extend a few things through the SMS-Backup-Control-File which can be found here: "<ConfImgr-Install-Dir>\inboxes\smsbkup.box\smsbkup.ctl"
Native SQL Backup:
With this method you need to configure everything on your own. There are built-in-Maintenance-Task to configure Backup and Cleanup of the SQL-Backup-Files. Here you can backup each Database on your SQL – Master, MSDB, SUSDB, ReportServer, MDT. One big difference to the Built-In-Backup by Configuration Manager is the compression of the File which can up to 70 % of the original Database. You have archives over longer period – depending on your configuration.
Also this method has advantages if you only have to recover the Database in a Disaster. However, you also need to setup Notifications on your own if the Backup was successful or failing.
BUT Always keep in mind that the Database alone is not enough to restore a Site – you require always the cd.latest in the Version of your Configuration Manager Site! So you cannot use the Baseline Version MECM 2203 to recover a Site which is running MECM 2207/2211. You also need to take care that this aligns.
Disaster-Scenarios:
Now, where we know how we can Backup and what is included – we close this chapter with a Mind-Map about the Disasters. There are couple of options and different Recovery-Steps are possible.
|
# |
Operating System |
Site Server |
Database |
Scenario |
|
1 |
|
|
|
OS and Site Server stable, but SQL broken |
|
2 |
|
|
|
OS and SQL stable, but Site Server broken |
|
3 |
|
|
|
OS stable, but Site Server and SQL broken |
|
4 |
|
|
|
OS and Site Server broken, only SQL stable |
|
5 |
|
|
|
OS broken, Site Server and SQL stable – This is only possible if you run High Availability |
|
6 |
|
|
|
OS, Site Server, and SQL broken |
Change Site Server Operating System
Depending on your Configuration Manager Design – you will run in one of the Scenarios 4, 5 or 6. Or simplified said each Scenario where the Operating System is affected.
Review your Disaster-Recovery-Plan – if you do not have one – we encourage you the create one. In a real Disaster there is some kind of panic mode usually. There is a lot of pressure on the ConfigMgr-Admins from the Service Desk and Leadership Team. And sometimes you face tiny issues which are blocking you from making the recovery, like do we have License-Keys? Where are our Installation-Medias? Do we have access to them? Where to find the Passwords for the Service-Accounts? And many others – you will be only aware when you do and train this exercise regular.
In our case we have a “planned” Disaster – so we can do preparation and define a schedule. Always review Microsoft Learn and look at the Supported Configurations for Microsoft Configuration Manager. Is the Configuration Manager Version we are running supported with a higher Operating System or SQL-Version. Maybe it is necessary to upgrade the Site first before we can achieve our goal to change/replace the Operating System Version.
IMPORTANT:
When you use this procedure also to upgrade your SQL-Server to latest supported Version – Verify if you are using the Configuration Manager Bitlocker Feature and if you also configured encryption of the Database. If your SQL Server is 2014 and below – you will face issues with the encryption Keys. The reason for SQL 2016 and higher we use a stronger Cipher for the Encryption Certificate, and it will be necessary to escrow the Keys again. If you are running in this Scenario open a Ticket with Microsoft Customer Service & Support to get detailed instructions.
“Fortune favors prepared mind.”
Here a quick Reference for the Preparation and the Disaster-Recovery Day. We differentiate
here two scenarios – a Configuration Manager Site Standalone and a Configuration Manager Site with a dedicated SQL-Server-Instance:
Configuration Manager Standalone:
PreWork:
- Setup the new Site Server
- Uninstall CCMAgent (on new Box if present)
- NO_SMS_ON_DRIVE.SMS placed
- Create Backup of the SQL-Database (MECM, SUSDB)
- Create Backup of Custom Reports
- Create Backup of the SSRS-Encryption Key
- Copy Pkg/App/Image Sources to the new Server (Recommendation to have central File-Share for Sources)
- Create a Backup of the Share-Permissions (Built-In shares will recreated but maybe you configured custom)
- Create a Backup of Scheduled Task
- Copy cd.Latest-Folder to the new Server
- Copy SQL-ISO to the new Server
- Copy/Download SSRS-Binarys
- Copy/Download WADK-Binarys
- Review Windows-Logins and Permissions
- Review SQL-Logins and Permissions
- Review SQL-Jobs and Maintenance Plans
- Verify that the Service Accounts have correct Privileges – (https://docs.microsoft.com/en-us/troubleshoot/windows-server/deployment/error-install-windows-intern...)
- Have the MECM-Product-Key
- Have the SQL 2019 Product-Key (needed for SSRS)
- Note the SiteCode and the Installation-Path of MECM - SiteCode XYZ - D:\Apps\Microsoft Configuration Manager
- Set a Local Administrator Password for the new Server
- Create a PreStage-Content-File / Backup of the Content-Library
Disaster-Recovery-Day:
- Join the new Server to a Workgroup
- Shutdown the old Site Server
- Reset the AD-Computer-Account of the old Site Server
- Join the new Server again to the Domain with the Hostname of the old Site Server
- Change IP-Address (This is not a hard dependent or requirement)
- Move the Server to the new OU-Unit for Windows Server 2019/2022
- Install SQL
- Install SSMS
- Restore SQL-Databases
- Install Roles & Features
- Install WADK
- Install SSRS
- Configure SSRS, point to the restored Database and apply the SSRS-Encryption-Key
- Install Third Party Apps & Tools
- Apply Custom-Unique Configurations (for Instance Host-File-Config)
- Start Recovery-Process of Microsoft Configuration Manager
- Review Site Components & Status Messages
- Check Inboxes for a Back-Log
- Update Package Sources / Import Prestage-Content-File / Restore Content-Library
- Sync WSUS and verify Synchronisation success
- Regenerate Boot-Image-Medias
- Regenerate CMG-Secret-Key
- Regenerate DP-Certificates (Http/eHttp Environments)
- Test-Deployments (TaskSequence, Applications,…)
Configuration Manager Site with a dedicated SQL-Instance:
PreWork:
- Setup the new Site Server
- Uninstall CCMAgent (on new Box if present)
- NO_SMS_ON_DRIVE.SMS placed
- Copy Pkg/App/Image Sources to the new Server (Recommendation to have central File-Share for Sources)
- Create a Backup of the Share-Permissions (Built-In shares will recreated but maybe you configured custom)
- Create a Backup of Scheduled Task
- Copy cd.Latest-Folder to the new Server
- Copy/Download WADK-Binarys
- Review Windows-Logins and Permissions
- Verify that the Service Accounts have correct Privileges – (https://docs.microsoft.com/en-us/troubleshoot/windows-server/deployment/error-install-windows-intern...)
- Have the MECM-Product-Key
- Note the SiteCode and the Installation-Path of MECM - SiteCode XYZ - D:\Apps\Microsoft Configuration Manager
- Set a Local Administrator Password for the new Server
- Create a PreStage-Content-File / Backup of the Content-Library
Disaster-Recovery-Day:
- Join the new Server to a Workgroup
- Shutdown the old Site Server
- Reset the AD-Computer-Account of the old Site Server
- Join the new Server again to the Domain with the Hostname of the old Site Server
- Change IP-Address (This is not a hard dependent or requirement)
- Move the Server to the new OU-Unit for Windows Server 2019/2022
- Install Roles & Features
- Install WADK
- Install Third Party Apps & Tools
- Apply Custom-Unique Configurations (for Instance Host-File-Config)
- Start Recovery-Process of Microsoft Configuration Manager – ( in the Wizard use a site database that has been manually recovered)
- Review Site Components & Status Messages
- Check Inboxes for a Back-Log
- Update Package Sources / Import Prestage-Content-File / Restore Content-Library
- Sync WSUS and verify Synchronisation success
- Regenerate Boot-Image-Medias
- Regenerate CMG-Secret-Key
- Regenerate DP-Certificates (Http/eHttp Environments)
- Test-Deployments (TaskSequence, Applications,…)
So, this general Guidance should help you to do a successful Restore and running Configuration Manager with a current Operating System. The full documentation you find here:
Site recovery - Configuration Manager | Microsoft Learn
We hope this Blog-Post was helpful and give you a better understanding what is necessary and to do and what to take care.
Disclaimer
The sample scripts are not supported under any Microsoft standard support program or service. The sample scripts are provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample scripts and documentation remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample scripts or documentation, even if Microsoft has been advised of the possibility of such damages