We designed a hierarchy based on internal teams that we want to use for access to document library resources.
For example, in a simplified version we might want:
- Exec Team to have access to all resources ↓
- Payroll → limited to that group and Exec
- Managers access to almost all but payroll and and manager HR locations ↓
- Senior Team Leads to team docs, Growth and HR folders except for Team Leads and above ↓
- Nuts & Bolts to admin and finance docs ↓
- Growth Team access to multiple team docs, theirs and all experience teams ↓
- Experience Teams access to all customer data
So we created AAD Security Groups starting with Experience permission group and then Growth. The Growth permission group gets a group membership to Experience etc up to the Exec permission group which has a group membership to all the security permission groups.
We don't want SP site membership to define access to docs so we delete the inherited permissions and add a Limited Access Group to the document library settings for the lowest hierarchical group then all above should have access.
The idea is then if we add a Manager user, they are not added to multiple sites as Members, instead, access permissions are conferred via their permission group
Managers belong to Senior Team Leads which belongs to Growth and so on
When checking permissions on an Experience library for someone assigned to Manager for example, it shows they have multiple permission levels conferred by their group and all the groups that the Managers permission group belongs to on a downward cascading basis. BUT, the individuals only have access if Direct Members in their own group 
Anyone have thoughts on why that would be?
And if the above isn't possible, what do other people do to assign a person to have access to multiple libraries? We have not found the perfect model for our operating needs.
Thanks a TON in advance!