Restrict Document Library so that only specific folders shared with a user are visible to them

I want to thank you both for the replies.  This is a legal requirement to preserve our communications being that we are a regulated industry.  In addition, we did not want to lose institutional knowledge.  
We did strip inheritance from every individual folder and then re-permission with direct access based on an audit log we were provided.  It was incredibly cumbersome and time consuming, and any kind of management of specific folder permissions at that scale is a huge burden.  So we decided to never manage folders.  We scripted the permissions, we set them in place, and permissions on the site will never be altered.  I am now focusing on creating indexed columns to improve searchability.