azure devops pipeline build very slowly

we have a Privatized deployment Azure devops environment.   There are approximately 230+ repos available.   Use yaml file to concatenate all repos ,forming a cascading pipeline of multiple stages, about 400+ stages.  we have three agent pool, a windows pool with 20 agents, two linux pool with 16 agents and 9 agents.

Now,when we run the complex pipeline,it took very long time to end. Execution time is twice than normal,or more.

Based on our observations,time is wasted on scheduling agent,but our agent is very idle.

how to reduce the execution time?  

Anybody can help me? If you need any information, please let me know.  thanks a lot