Group permissions and Sub Sites

I have created a main site and added sub sites with unique permissions. I did this be stopping inheritance for each subsite. I then created a unique permission group for each subsite. I did not send out the email invitation for the groups as we are still setting up the site. Running tests with some key users, if I give them the main site address they are able to get to the main site but get a message  that they are not allowed on the subsite.