Sync shared mailbox contacts on Android

Hello,

I have a shared mailbox with over 4000 contacts I would like to sync on my Android device. 

I added the mailbox on the outlook app and turned on contact sync but it's not working, I also tried multiple Android device but it doesn't sync.

Is there a way to fix this ?