Provider Hosted App (PHA) application errors (TLS errors and 401 errors).

[SPDev_Support]

Note: Please take extreme caution before making any changes in Production. Make sure you test the changes in a test environment first.

Recently due to a spate of updates to various endpoints in SharePoint, Azure, and the AAD auth login endpoints, we are seeing projects compiled with version of .NET before 4.6 cause TLS errors which don’t always show as TLS errors in the PHA. They are intermittent sometimes due to the updating of load balanced servers. In PHAs, it sometimes shows as a 401 error due to the way the token helper PHA code works. The actual error is this:

"The underlying connection was closed” or “System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host”

This error is most likely due to deprecation of TLS1 1.0 & 1.1 while .NET 4.5 apps don't use TLS 1.2 without an explicit code change.

Please see:

Preparing for TLS 1.2 in Office 365 and Office 365 GCC - Microsoft 365 Compliance | Microsoft Docs

Enable TLS 1.2 on servers - Configuration Manager | Microsoft Docs

TLS 1.0 and 1.1 deprecation - Microsoft Tech Community

Transport Layer Security (TLS) best practices with the .NET Framework | Microsoft Docs

The updates were communicated in the Office 365 message center.

• MC218794 - July 17, 2020 | TLS 1.0 and 1.1 retirement date in Office 365 to be October 15, 2020
• MC240160 - Feb 16, 2021 | Reminder: Disabling TLS 1.0 and TLS 1.1 in Microsoft 365

The deprecation of TLS 1.0 and 1.1 on all SharePoint Online servers will be completed by June of 2021. There are no rollbacks to this security update.

Additionally, these are the only ciphers that will be supported for TLS 1.2 connections-

• TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
• TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
• TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
• TLS_DHE_RSA_WITH_AES_128_GCM_SHA256

Additionally, Windows Server versions 2012 R2 and before have a bug in TLS_DHE* cipher suites that may cause them to fail, and there is no fix for those versions. For Server 2016 and later you can disable TLS_DHE* ciphers since they support the 2 TLS_ECDHE* ciphers, but earlier server versions will not work reliably. More information here-
Errors when applications try to connect to SQL Server in Windows - Windows Server | Microsoft Docs

Windows Server 2008 and 2008 R2 extended security updates | Microsoft Docs

The recommendation is to upgrade to at least server 2016.

Checking which TLS and ciphers are used

To see which version of TLS, and which ciphers are supported by the client machine, you can take a network trace with a tool like Wireshark which we are using here, or any others that can take .cap captures. Get the resolved IP address of your SharePoint server by using ping as follows-

Make a note of that IP.

Next take your network capture while causing the failing network call to happen. In Wireshark you start by selecting your network interface-

After reproducing the problem with the trace running, stop the trace, and use the SharePoint online IP noted above to filter the network trace as follows-

Take note of the TLS version and the ciphers.

If the connection is being rejected due to TLS version/protocol issues, you will see a response with the RST flag set, in Wireshark it is highlighted in red. The flags are displayed as [RST, ACK].

You can compare this with the supported cipher list mentioned earlier in this article.

Windows 2016 and up have the proper ciphers enabled, but many admins will disable them.

One way the ciphers are disabled is by disabling RSA support with this key-

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\KeyExchangeAlgorithms\PKCS]

    "Enabled" = dword:00000000

If this key is present, change the value of 'Enabled' to 0xffffffff.

If this isn't the issue, then you may need to change cipher settings in windows with these articles-
Manage Transport Layer Security (TLS) | Microsoft Docs
(using group policy on the local or domain policy to set which ciphers and their order)

TLS Module | Microsoft Docs

(using Get-TlsCipherSuite and Enable-TlsCipherSuite commands in Powershell on Windows server 2016+)

More info on cipher suites-
Cipher Suites in TLS/SSL (Schannel SSP) - Win32 apps | Microsoft Docs

Scenarios

In all scenarios on a VM or physical server

you first need to add the following registry key settings on any web servers or hosts/workstations running .NET executables:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v4.0.30319]

"SystemDefaultTlsVersions" = dword:00000001

"SchUseStrongCrypto" = dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\v4.0.30319]

"SystemDefaultTlsVersions" = dword:00000001

"SchUseStrongCrypto"=dword:00000001

Note: You will need to restart your server(s)

After this, you will need to do one of the following depending on the application type (web app, executable, WCF).

If the app web is hosted on-premise in IIS

After the registry steps above, modify the targetFramework attribute on the httpRuntime element as follows-

<httpRuntime targetFramework="4.7.1"/>

Any version equal or greater than .NET 4.6+ will work, but we recommend at least 4.7+.

If the App web is hosted on Azure

Log into the Azure portal (portal.azure.com) with an account with admin rights on the web app in question.

Once you open the App Services and select the web app hosting the PHA site, click on the App Service Editor (Preview).

This will open the editor, then select the web.config file and change the circled targetFramework attribute to 4.7 preferably (any setting higher than 4.6 will work too). Note the status in the upper right will say ‘DIRTY’ for a bit, then it should auto-save and change back to ‘SAVED’-

At this point you should be set, refresh the page with the PHA and all should be good.

If your application is an Azure web job, Executable (e.g. console, WPF, Windows Forms, etc.), or WCF service

The .NET registry keys mentioned above need to be applied on the host machine that runs the application (except for the Azure case).

Then the application project in Visual Studio needs to be retargeted to 4.6+ (4.7.1+ recommended) by changing the target framework setting in the application properties and re-compiling. The Azure web job executable will need to be updated with the re-compiled executable.

More here-

Targeted .NET frameworks - Visual Studio | Microsoft Docs

If you want to read up on known issues for specific re-targeting scenarios, you can select the from/to versions and see the known issues-

Runtime and retargeting changes - .NET Framework | Microsoft Docs

Note: You can’t use the config file for the exe to re-target the same way we can for a web application.

If you are running an app on Java

You can find more information here- 

Configure Oracle's JDK and JRE Cryptographic Algorithms (java.com)

If you want to continue to use .NET 4.5.x apps (NOT RECOMMENDED)

This is NOT recommended since it takes the same time to retarget the framework and re-compile, and it requires adding a line of code. Only use this if there is a valid reason to not re-target to .NET 4.6+. This line will enable TLS 1.1 and 1.2, you can add or remove protocols that are in the SecurityProtocolType enumeration-

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;

I emphasize, this should not be used except as a last resort to an impediment to re-targeting to .NET 4.6+

[Mike Morawski]

Is this something that would be intermittent? Or would it be something that once it breaks ends up being constant? As well, in the HTTP response body is there any message indicating the cause as TLS so we can verify if this is indeed the issue?

[SPDev_Support]

Yes. We have seen this would be both intermittent and constant. In the response body "The underlying connection was closed: An unexpected error occurred on a send." is the most consistent message we have seen. We have also seen Tls stream error messages in the stack trace.

[WestleyH]

Make sure to put in the strong crypto key for 64 and 32 bit compilation, as VS projects default to prefer 32 bit, and if that key is missing (the Wow6432Node one), those programs will still use the older TLS-
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v4.0.30319]
"SchUseStrongCrypto"=dword:00000001
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\v4.0.30319]
"SchUseStrongCrypto"=dword:00000001

[bphillips86]

If we do the retargeting and also explicitly set this following in global.asax, would there be any issues?

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;

[WestleyH]

It will work fine, but might keep it from using TLS13 in the future when that protocol is introduced or enforced.

Hard-coding the protocols should be avoided.

Since the code is in the global.asax, it isn't as bad since you can edit it and it will re-compile; you could also put the settings in a web.config and that would be a little better.

Leaving it up to .NET and the OS is best, then you just make sure the OS has the support installed/enabled, and you are ready to go with no changes to your application.

[bphillips86]

Thank you for the reply!

What would these changes look like in web.config?

I currently have customers reporting errors that resolve to "The underlying connection was closed: An unexpected error occurred on a send.". And, while I've made most of the changes included in this article, namely upgrading to .NET 4.7.2 and retargeting. When that didn't work, I added the line of code to Global.asax. However, it still yields the same outcome. It happens when using CSOM.

Is there something special that needs to be done to make CSOM calls after making these TLS changes? I upgraded it to 16.1.0.0 hoping that would help.

[WestleyH]

The changes in the web.config would be app config entries with the protocols, and you would need some code to pick them up and set the TLS.

The other issue that people are hitting is that only 4 ciphers in TLS 1.2 are supported in the changes happening now, and if you have disabled these, the TLS 1.2 connections will fail. I have updated the above article to include some of this info.

[manjeetjagtapgmailco]

I had 2 issues and was getting the same error. 

The first thing is my client Id and Client secret was expired. If you create app through /_layouts/15/appregnew.aspx it's expiry is 1 year. If you want to increase expiry by more than 1 year you can use the PowerShell script given in the below link.

https://docs.microsoft.com/en-us/sharepoint/dev/sp-add-ins/replace-an-expiring-client-secret-in-a-sh...

Secondly, I have a 4.5 framework console application and able to reproduce this issue on windows server 2012 R2. Even after I get updated client Id and secret and  I changed the .Net framework to 4.6+ or 4.7 still able to reproduce this issue. So I have to have the following line of code before calling a method to get an access token.

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;

This resolved my issue at least for now.  

[WestleyH]

You don't need to hardcode, unless your executable is compiled with .NET 4.5. If you recompiled to 4.6 or 4.7 (you can't use the config file for this, you have to re-compile in VS) then the registry keys should work.