TaxonomyHiddenList not properly provisioned in Group sites

We are provisioning a lot of O365 Groups and sometimes the TaxonomyHiddenList in the Group connected site does not have the needed fields populated.

In some cases it's possible to re-create the list by deleting it and re-activate the TaxonomyFieldAdded feature which is responsible for provisioning the list - but in other cases activating the feature simply results in a "Access Denied".

This i quite annoying as the only other fallback I can see is to delete the Group and try again! 

Any suggestions would be appreciated?