Removed external meeting participant can easily rejoin

We are in the process of switching over to Teams but encountered an odd issue in testing meetings with external guests.  We have it set up so guests start out in the lobby and then we approve them to enter.  If we remove them at some point during the meeting they are correctly forced out and on their screen see the message that they were removed and get two options, Rejoin or Dismiss.  If they immediately click Rejoin they bypass the lobby and are right back in the meeting.  It feels like this is not intended.  If they dismiss and close their browser then go back to the invitation and click the link to enter the meeting again they get put back in the lobby and have to wait to be let back in.  This is with the guest attending the meeting using a web browser.  I tried this with both Edge and Chrome, I have not yet tried this using the Teams app.