People field in a list with some non-Active Directory people

I am making a SharePoint list roster that lists everyone by individual, then all the memberships (different committees and groups) and membership types (lead, alternate, etc) that person has.

I am hoping to use the People field for data integrity and to avoid hand entering all contact info. The problem is, there are a handful of out-of-org people that are not in Active Directory. I can't figure out a good way to deal with those non-AD people and still be able to use the smart People field for everyone else. 

Any ideas? I was trying to think through using a lookup list somehow, or putting the non-AD ones in a different field, but the first doesn't really work and the second is messy. Or possible some sort of workflow solution I'm not thinking of? Any ideas welcome!