Securing outbound traffic with Azure Data Factory's outbound network rules

Does the ability to enable/disable the outbound rule feature, map to an action within the Microsoft.DataFactory resource provider ?

We would like to enable the feature but our users are generally ADF contributors of their own subscriptions, so from what I understand, we can enable Azure policy containing the allowed FQDNS on their Sub/Resource group/ADF, but then its optional to enable or disable the setting inside the ADF portal. 

If users are contributors, they could enable/disable the feature to get around the restriction. 

I was thinking a custom RBAC policy would do it, containing most of the contributor roles, but disalowing the enable/disablement of the feature.... but cant see anything in the https://learn.microsoft.com/en-us/azure/role-based-access-control/resource-provider-operations#microsoftdatafactory that might be one we wanted to restrict.