cannot open skype control panel 2019 after migration side by side from Lync 2013

[Elazaro]

Existing Environment-
We have Lync server 2013 with pool enterprises with 3 Front-end.
I just upgraded from Lync 2013 to Skype for Business 2019. (the new Existing Environment on windows server 2019 and we doing migration side by side ).
Everything seems to be working fine except I cannot log into the Control Panel from anywhere the web interface (PoolName/CSCP) or from the app : ”skype for business control panel”.
When I connect I get - Error 503 service unavailable.
Also the new MACP (the new control panel with Html 5 not working).
Thanks for the help!

[#2]

Hi @Elazaro,

503 seems to be backend related.

• However, did you also try to go onto one frontend server and try to open it via https://localhost/macp or /cscp, too?
• If that's not working, too, did you check event logs on the frontend servers?
• Are all services up and running (services.msc)
• Is there any load balancer involved in front of the pool which might not allow the traffic?
• Did you try to re-run the configuration wizard installation process?
• Did you check the IIS virtual directory settings?
  Especially the URL rewriting rules and bindings? (actually that's nothing you should have to deal with because the bootstrapper configures it the right way).

[#3]

@Erik365Online 

Hi Erik,

• However, did you also try to go onto one frontend server and try to open it via https://localhost/macp or /cscp, too? I try its get the same error 503.
• If that's not working, too, did you check event logs on the frontend servers?I get this event ID 9009 - The Application Host Helper Service encountered an error trying to delete the history directory 'C:\inetpub\history\CFGHISTORY_0000000064'. The directory will be skipped and ignored. Note that the directory may still get deleted in the future if the service restarts. The data field contains the error number. I don't know if it's related to the issue because I get this Event In general.
• Is there any load balancer involved in front of the pool which might not allow the traffic?No, only I have 3 A record of the pool that points to Each FE server separately.
• Did you try to re-run the configuration wizard installation process?YES, I reinstall the IIS and WEB Components (also I try to reinstall the pool and install again and it's shown again 503 services.
• Did you check the IIS virtual directory settings?yes, all settings are correct.
• Especially the URL rewriting rules and bindings? (actually that's nothing you should have to deal with because the bootstrapper configures it the right way).
  yes its also configure correctly I do the bootstrapper.
  You have any suggestions I'd be glad to help

[#4]

yes its also configure correctly I do the bootstrapper.

[#5]

Hi @Elazaro ,

to me this seems like the IIS configuration was messed up anyhow.

• You could try to set the permissions for the 'C:\inetpub\history' or the 'C:\inetpub\history\CFGHISTORY_0000000064' manually?
  Or delete the folder manually or just renamed it...
  However, I don't think that the event ID 9009 is the actual root cause.
• Is there any antivirus scanner on the servers?
  In case there is did you configure exclusions or try to turn it off in order to check if than works?
• Are there any workloads / migrated users already on the server? I'd guess no, right?
• Did you consider to start from scratch? Would that be an option?
  • Either a new pool on new/fresh Windows Servers or 
  • you try to get the existing Windows Servers in good shape
    • CMS, conf. directories etc. should be on the 2013 
    • Stopping all SFB 2019, local SQL ... on the 2019 frontends
    • Uninstall the complete IIS role on the Windows Server
    • Manually checking the IIS directories (file directories in C:\...) are removed (if not I'd remove these)
    • Re-Install IIS (SFB 2019 prerequisites as described on docs.microsoft.com; I'd just run the powershell prereq script/cmdlet)
    • Install Windows Server updates ...
    • Re-Run SFB 2019 setup/bootstrapper (all steps, so that it installs IIS vDirs and stuff as needed)
    • Test if the errors still occurs or is vanished

Please note, the above are just some ideas and thoughts and might be incomplete. You might want to open a support case at Microsoft Support for more in-depth support.

[#6]

Hi Erik,

I want to update you on the changes we made yesterday.

We Removed the Pool from The Topology builder and cleaned the SQL Database.

We installed new servers (Server 2019) and create new pool in Topology builder.

We didn’t install CU, we want clean environment. (So now we don’t have the MACP).

After we finished we try to open the CSCP and the issue 503 service unavailable still happening.

I  checked the logs in CLSloger. 

But I don't see the Web Infra logs.

They are empty.

I want to clarify that the issue his with all the application pools in IIS. (503 service unavailable).

[#7]

Hi @Elazaro, 

that's a very odd behaviour especially that you set up a new pool including fresh servers.

Just some basic things I'd re-check although you probably have done that already.

• Did you open the IIS console and check the applications and virtual services?
• Is there cscp available at all?
• Is it possible to open https://localhost/meet/ or other vDirectories/contents?
• You could re-run the pre-req script maybe something did not work properly, maybe something did not install as it should (https://docs.microsoft.com/de-de/skypeforbusiness/plan/system-requirements#software-that-should-be-i...:(

Add-WindowsFeature RSAT-ADDS, Web-Server, Web-Static-Content, Web-Default-Doc, Web-Http-Errors, Web-Asp-Net, Web-Net-Ext, Web-ISAPI-Ext, Web-ISAPI-Filter, Web-Http-Logging, Web-Log-Libraries, Web-Request-Monitor, Web-Http-Tracing, Web-Basic-Auth, Web-Windows-Auth, Web-Client-Auth, Web-Filtering, Web-Stat-Compression, Web-Dyn-Compression, NET-WCF-HTTP-Activation45, Web-Asp-Net45, Web-Mgmt-Tools, Web-Scripting-Tools, Web-Mgmt-Compat, Windows-Identity-Foundation, Server-Media-Foundation, Telnet-Client, BITS, ManagementOData, Web-Mgmt-Console, Web-Metabase, Web-Lgcy-Mgmt-Console, Web-Lgcy-Scripting, Web-WMI, Web-Scripting-Tools, Web-Mgmt-Service

• Check if .NET is up-to-date https://support.microsoft.com/de-de/help/3186497/the-net-framework-4-7-offline-installer-for-windows
• afterwards I'd might re-run the bootstrapper and reboot the server or at least run iisreset /force in cmd as admin

[#8]

Thanks for the response, Erik.

I checked the applications and virtual services all configured correctly. 

• Did you open the IIS console and check the applications and virtual services?

no, the Cscp of skype 2019, not available.

only the old cscp Lync 2013 ( different environment in the pool).

• Is there cscp available at all?

no, all get error 503.

• Is it possible to open https://localhost/meet/ or other vDirectories/contents?

Already checked.   

• Add-WindowsFeature RSAT-ADDS, Web-Server, Web-Static-Content, Web-Default-Doc, Web-Http-Errors, Web-Asp-Net, Web-Net-Ext, Web-ISAPI-Ext, Web-ISAPI-Filter, Web-Http-Logging, Web-Log-Libraries, Web-Request-Monitor, Web-Http-Tracing, Web-Basic-Auth, Web-Windows-Auth, Web-Client-Auth, Web-Filtering, Web-Stat-Compression, Web-Dyn-Compression, NET-WCF-HTTP-Activation45, Web-Asp-Net45, Web-Mgmt-Tools, Web-Scripting-Tools, Web-Mgmt-Compat, Windows-Identity-Foundation, Server-Media-Foundation, Telnet-Client, BITS, ManagementOData, Web-Mgmt-Console, Web-Metabase, Web-Lgcy-Mgmt-Console, Web-Lgcy-Scripting, Web-WMI, Web-Scripting-Tools, Web-Mgmt-Service​

• Check if .NET is up-to-date - I have .NET 4.8 and I also try the repair tool for framework Microsoft.

[#9]

Hi Erik,

thanks a lot for your help.

the issue solved by install 3 new Servers 2016.

and install in the existing pool 3 new FE and Remove the (Old 2019 from the pool).

it seems like the server 2019 with the last update was the issue.

[#10]

Good that you could fix it by setting it up from scratch including the windows servers (2016 instead of 2019). Well done. :thumbs_up:

[#11]

@Elazaro 

just update - 

this issue happened if you are installing Windows Admin Center 2019 on your Windows Server 2019 machine.

admin center was installed on my Server.

I removed the Admin Center before the installation of skype.

but this not fixes the issue.

because some who port 443 still used.

I previously had Windows Admin Center installed, and now nothing else can use the same TCP/IP port

Manually run these two commands in an elevated command prompt:

netsh http delete sslcert ipport=0.0.0.0:443
netsh http delete urlacl url=https://+:443/

and please don't forget to run - IISreset after you running those commands.

or

when you install the Admin Center if you going to install skype on this machine.

please change the port when you install the Admin Center.

Microsoft adds this note lately -

more details - 

If you are installing Windows Admin Center 2019 on your Windows Server 2019 machine, it will prompt you for a port to listen on. There's a likelihood you might choose port 443, but if that machine has Skype for Business Server 2019 installed on it, or is going to have Skype for Business Server 2019 installed on it, then you must choose a different port number.

Why is this the case? If Windows Admin Center 2019 is running on port 443, you will not be able to connect to the server using the Skype for Business Control Panel, nor will you be able to connect to any internal web service running on the server (Address Book Web Service, Autodiscover Service, WebTicket Service, etc). In fact, you will not be able to connect to any Internal Web Service URL. Please choose a different port, in the event you need or want to put Windows Admin Center 2019 on a server with Skype for Business Server 2019.