SharePoint Document Library speed?

Hi,   I am wondering if having a larger SharePoint document library can result in slower access. I am aware of the 5000 item list restriction, and it will not apply to our scenario as all files wil...