Remove from shared list error "couldn't remove item from shared list"

I was trying to remove outdated files in my "Shared with me" list.  I kept getting the error "Couldn't remove item from shared list - Either item hasn't been shared or isn't shared with you.   I veri...