User Accounts Not Populating with Filters in Directory with PnP V4

I'm working on building a staff directory for my org. I'm leveraging the PnP Modern Search Web Parts for this directory. I am having issues with 5 user accounts not populating in the Search Results Web Part with the department or office filters applied to the SearchTerms box. I have RefinableString03 set to People:Department and People:SPS-DEPARTMENT. RefinableString09 is set to People:OFFICE and People:SPS-Location. 

If I set the SearchTerms box to *, the 5 missing user accounts populate along with other accounts that needed filtered out. The location/office and department attributes are set for each of these accounts. They show in the hovercard for each account. It seems that the department and office/location attributes are not being picked up by the SearchTerm filter when the RefinableString03 and or RefinableString09 are applied to the Search Results Web Part. User account attributes are synced from our on-prem AD environment.

What piece am I missing here?