I finally got my EWS in our DAG working again. Thank you yvess2355 for your excellent tip!
So basically I used these cmdlets, servernames are simple, not FQDN:
New-SettingOverride -Name "Adding learning location ClientExtensionCollectionFormatter" -Server SERVER1 -Component Data -Section DeserializationBinderSettings -Parameters @("LearningLocations=ClientExtensionCollectionFormatter") -Reason "Deserialization failed"
Get-SettingOverride -Identity "Adding learning location ClientExtensionCollectionFormatter" | Set-SettingOverride -Server SERVER1,SERVER2,SERVER3,SERVER4
And then:
Get-ExchangeDiagnosticInfo -Process Microsoft.Exchange.Directory.TopologyService -Component VariantConfiguration -Argument Refresh
On each individual Server, followed by a IISRESET.