What these guys said, however, there is also some things that would require the link, the users using that link are external users they won't have access to the Shared with me link etc. So it kind of depends on who is using the link originally.
If you share using an anon link or "Org link", the first time someone clicks the link they will get added to the permissions of the source if they are an org user, but until then they must use the link, unless you use "Specific people" option which will add direct permission to the use at time of creation.
So basically, to answer your question, it depends on how it was shared.
