Azure AD App Registration - Permision request to Read and Write to “All” Site Collections too broad

When setting up an Azure App Registration for the Microsoft Graph or the SharePoint Online APIs, the only option is to grant read and write to "ALL" site collections either as delegated or app permis...