Unable to exclude assigned members for tasks from board

After assigning a member for any tasks, you have to add them to the complete board which has all the other projects/tasks which is challenging as you don't like to highlight all the work to them. How can be this be executed on Planner