Allow only specific apps to be added to teams

Hi,

I am trying to configure it so that users can only add one specific app to any team in my organization (working on a dev tenant to test this out). 

I went into Teams Admin -> Teams apps - Permission Policies and edited the Global (Org-wide default) policy. I set it to allow Specific apps and block all others. I then added Smartsheet to the allowed apps list. 

I then logged in to the Teams client with a test user (no admin access) and tried to add a tab. It still listed all the apps there , and it did let me add an 'AgilePolly' tab (a 3rd party app).

So how do I allow Specific apps and block all others?