What entitlements users receive when they are enabled with Copilot Premium Licenses ?

Having Copilot Premium/Microsoft 365 Copilot doesn't simply give the user full Power Apps or Power Automate licensing. Agent Builder is part of the Microsoft 365 Copilot experience, and agents can be copied from Agent Builder into Copilot Studio. Copilot Studio itself runs on Power Platform environments and supports Power Platform connectors, including custom connectors.

I'd therefore treat this primarily as a Copilot Studio/Power Platform governance issue rather than trying to remove Power Automate or Power Apps entitlements. Control who has maker access to environments, restrict the Default environment, and use Power Platform data policies to block or constrain connectors and other Copilot Studio capabilities. You should also review Copilot Studio credit usage to establish exactly which capability is consuming the tenant's credits.