Importing Team Members from Outlook

I'm testing out Team and likely have my Outlook address book set up improperly since when I try to add a new member to a team it recognizes only four out of the hundreds of contacts in my list.