Allow editing People sharing links Permission via Graph Api

There doesn't seem to be a way of removing access to a particular user to a Share Link via the Graph Api. Trying to update a permission's users (in my case removing some) results in an error : "Invalid input: No Information provided to update the specifed permission". Which means there is currently a way of adding users via the permission/grant api, but no way of removing them.