Introducing Conditional Access by Network Location for SharePoint and OneDrive for Business

For those who are looking for mroe granular options, i.e. by Site Collection, why would that make a difference? I'm trying to figure this out with a few of my customers, and I'm stuck on the "Authentication" piece being managed by Conditional Access, and the "Authorization" piece being managed, still, via permissions on site collections.  
I think you would want to use Conditional Access to enable network access based on location as determined by Who the User Is, and then, subsequently, use permissions to determine which Site Collections that person would have access to.  I struggle to see how  the same user identity would have different access to different site collection depending on where they are. I think having the one lever of conditional access to handle authentication and the second lever of permissions should be enough, shouldn't it?

I expect there will be more conversations around this, so I'd like to understand why conditional access at the site collection level would be a requirement. 
Thanks - 
Owen Allen,
Cloud Productivity TSP