You could do this via a playbook/logic app
If you had an incident created from an Azure AD Identity Protection alert which had the AAD Object ID as a mapped Account entity you could create a playbook called closed-identityprotection-alert or something. Use the Sentinel and Azure AD Identity Protection logic apps to dismiss the user and close the incident. Then instead of closing the incident in the Sentinel dashboard, just trigger the playbook instead.
See example below
