How to revoke elevation rights quickly in Endpoint Privilege Management?

We plan to use Endpoint Privilege Management for our users to get temporary rights to elevate access on their local client via my access packages. I have created 2 settings policies to achieve this: One that denies all elevations with all users assigned to and a second one that allows elevation for a specific group of users. This specific group of users is excluded from the "deny" group in the corresponding deny-policy.

When a user is in the "grant" group he/she is able to elevate, perfect. But when I remove the corresponding user from the "grant" group, it takes hours until the elevation access is revoked. Any ideas what I can do to speed up this? Sometimes, a reboot helps, but not always.

Thanks for your support!