Seems like PowerBI is finally getting some attention from the rest of Microsoft's services teams. First we got the Office 365 Content pack (preview), and now we also have a content pack for Azure AD.
The official announcement is here: https://blogs.technet.microsoft.com/enterprisemobility/2017/01/20/admins-rejoice-azure-active-direct...
The setup instructions can be found here: https://powerbi.microsoft.com/en-us/blog/azure-active-directory-meets-power-bi/preview/. In my case the login only worked with OAuth, not sure why.
This only works if you have Azure AD Premium
Yup, that's the part I didn't like :(