Outlook 2016 deleted internet calenders keep reappearing

I found the bug after hours of painstaking error searching! When using a shared internet calendar, there is an option (marked yellow in the screenshot) to display the calendar on all computers with the default account in Outlook. 
Having this marked seriously messes up Outlook. After I turned it off on my work computer, the errors when away. Scary enough, this is selected by default. 

So, solution: Review all computers with Outlook and remove/disable all internet calendars. Add them back one by one and ensure you disable the "Display this calendar on other computers with the account:" 
As an added security, I went into the folder C:\Users\xxxx\AppData\Local\Microsoft\Outlook and deleted the internet caledar pst file too to ensure it's completely removed.