About add-in problem, we use Symantec's "Report Email" and KnowBe4's "Phish Alert" add-in.
It seems KnowBe4's "Phish Alert" caused the issue so we removed the add-in, users can see add-ins now.
We had to recycle "MXExchangeServicesAppPool" to see the add-ins page from ECP again, any user try to see add-in page will fubar the add-in page for all!
Here is an error I got when I tried to manage the app from powershell when KnowBe4 button was still intact
PS C:\Windows\system32> get-app
WARNING: An unexpected error has occurred and a Watson dump is being generated: Deserialization of type System.MarshalByRefObject blocked due to InDeny at location ClientExtensionCollectionFormatter.
get-app : Deserialization of type System.MarshalByRefObject blocked due to InDeny at location ClientExtensionCollectionFormatter.
At line:1 char:1
+ get-app
+ ~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Get-App], BlockedDeserializeTypeException
+ FullyQualifiedErrorId : Microsoft.Exchange.Diagnostics.BlockedDeserializeTypeException,Microsoft.Exchange.Management.Extension.GetApp